04
Key concepts
8 min read · Chapter 4 of 10
01
Entity and identity
What is the unit of record, and how do we know it is the same one?
The governed unit depends on who's asking. US bank guidance governs a "model", which the April 2026 text narrowed to complex quantitative methods built on statistical, economic or financial theory, leaving out spreadsheet arithmetic, deterministic rules and generative AI. Canada's OSFI E-23 widens "model" to include AI and machine learning methods and judgment-based tools. The UK's PRA covers vendor models "regardless of technology". The EU AI Act, NIST and ISO 42001 talk about an "AI system", and most company registers track a use case. I'd key the register on the use case (purpose, users, data, decisions affected) and link models, vendors, prompts and datasets to it, because one model serves many use cases and one use case can swap models.
Roles carry identity too. Banks name a model owner, developer, validator and user. The EU separates the provider, who places a system on the market under its own name, from the deployer, who uses it professionally, and those roles attach per system and can flip. US law keys on the decision-maker's legal role (creditor, employer, insurer, health care provider) and now pulls vendors in as the employer's "agent" (Workday, California's rules), as a consumer reporting agency (a theory in a 2026 suit against Eightfold) or under housing law (SafeRent). For each use case I'd write down which entity decides and which vendor a court could call its agent.
02
State and lifecycle
What states exist, and what moves an entity between them?
A use case moves through discovered, proposed, in assessment, approved with conditions, live, under review or suspended, changed and re-validated, retired and archived. The April 2026 US guidance is silent on retirement; Canada's E-23 has an explicit decommissioning stage. Vendors run their own states on their own clocks (active, legacy, deprecated, retired, meaning slightly different things at each lab and cloud), and those states drive yours.
Laws have lifecycles too: Colorado's AI Act was enacted in 2024, delayed, paused by a court stipulation, then repealed and replaced in May 2026. Most AI lawsuits in 2026 sit between discovery and class certification, and an ISO 42001 certificate lasts three years, with a surveillance audit each year.
03
System of record and ledger
Who owns the truth, and how do systems reconcile?
The inventory is the ledger. US bank guidance wants enough information to understand model risk "at the individual and aggregate levels", and E-23 lists required fields. The fields I'd keep:
| Field | Why it matters |
|---|---|
| ID, name, owner | Someone answers for it |
| Purpose, users, decisions affected | Sets the tier and which rules apply |
| Vendor, model and version | Tells you what to re-test when a vendor changes something |
| Data used | Privacy and bias questions |
| Approval, conditions, review date | What was agreed and when it expires |
| Validation dates, findings, open exceptions | US guidance asks banks to track "recommendations, responses, and exceptions" |
| Monitoring metrics and incidents | Evidence it still works |
| Retirement date and archive location | What you'd show a court later |
Even banks keep it in modest tools: in Deloitte's 2025 survey of 87 banks in Europe, the Middle East and Africa, 24% still used Excel as their main model risk tool. The evidence itself lives in eval runs, traces, tickets and logs.
Litigation turns these records into evidence. Cigna produced about 2.1 million pages in a suit over its claim-review algorithm, and Workday was ordered to produce its federal equal-employment reports. Retention rules set the floor: six months for EU deployer logs of high-risk systems, four years for automated hiring data in California. The test I'd use: if a court asked for every input and output for one applicant from 2024, could we produce them?
04
Rules and policy
What logic decides outcomes, and who can change it?
The policy stack runs from an AI policy and an acceptable-use policy down to tiering rules, an approval matrix and vendor standards; in a bank it sits next to the model risk policy. The operational pieces lag: in a 2026 survey by Schellman, which sells ISO 42001 audits, 44% of US companies with 500 or more staff had incident procedures specific to AI.
Tiering combines an outside screen with an inside score. US bank guidance measures materiality as exposure plus purpose, with inherent risk on top; E-23's inherent risk rating sets review intensity, documentation, the approver and monitoring; the EU tiers by use.
The binding rule often sits in a consent order or a contract instead of a statute. Massachusetts' settlement with Earnest, a student lender, requires an AI inventory, risk assessments, testing, documentation and an oversight team, which reads like a bank model risk program. Labs' usage policies require human review in credit, employment, insurance, housing and health, and customer contracts add their own conditions on top.
05
Effective dating
Which version of the rule applied at that moment?
Dates I'd keep on a calendar as of October 2026:
| Change | Effective | Status (Oct 2026) |
|---|---|---|
| EU AI Act prohibited practices | February 2, 2025 | In force; no fines found |
| California rules on automated hiring decisions | October 1, 2025 | In force |
| SR 26-2 replaces SR 11-7 for US bank model risk | April 17, 2026 | In force; not enforceable; excludes generative and agentic AI |
| Colorado insurance AI rules, auto and health compliance | July 1, 2026 | In force |
| Disparate impact removed from Reg B | July 21, 2026 | In force |
| EU Article 50 transparency duties | August 2, 2026 | In force |
| Connecticut: relying on AI is no defense to discrimination | October 1, 2026 | In force |
| EU marking grace ends; new prohibition on generated intimate images | December 2, 2026 | Upcoming |
| Colorado's replacement AI law; California automated-decision compliance | January 1, 2027 | Upcoming; Colorado may slip |
| OSFI E-23 for Canadian banks and insurers | May 1, 2027 | Upcoming |
| Connecticut pre-decision notices in employment | October 1, 2027 | Upcoming |
| EU high-risk duties for Annex III uses (credit, hiring, insurance pricing) | December 2, 2027 | Upcoming; moved from August 2, 2026 |
| EU high-risk duties for regulated products | August 2, 2028 | Upcoming; moved from August 2, 2027 |
Dates here slip often: Colorado's moved twice, the EU's high-risk date by 16 months, and the Commission's guidelines on classifying high-risk systems, due by February 2, 2026, came out as a draft in May. Vendors' retirement dates are often "not sooner than" dates.
06
Interfaces and standards
What format and protocol do counterparties speak?
| Standard | What it is | Who asks for it |
|---|---|---|
| NIST AI RMF and its generative AI profile (2024) | Voluntary US framework; inventory is GOVERN 1.6; under revision, no 2.0 by October 2026 | US buyers and regulators, as shared language |
| ISO/IEC 42001 | Certifiable AI management system | Enterprise procurement |
| ISO/IEC 42005 and 42006 (2025) | Guidance for impact assessments; rules for the bodies that certify to 42001 | Assessment teams; buyers checking a certificate |
| CSA AI-CAIQ and STAR for AI | A standard AI questionnaire; Level 1 self-assessed, Level 2 validated and requiring 42001 (from November 20, 2025) | Enterprise buyers |
| SIG questionnaire | Shared Assessments' vendor questionnaire: 128 questions (Lite), 627 (Core), 1,936 (Detail) | Enterprise buyers |
| SOC 2 | No AI-specific criteria found by October 2026; AI systems are scoped into ordinary reports | Enterprise buyers |
| NYC bias audit | Impact ratios by sex, race and ethnicity, and their intersections | NYC employers |
| EU code on marking AI content (June 10, 2026) | Voluntary layered marking, such as metadata plus a watermark, and a common EU icon | EU regulators |
The interface that matters most is the crosswalk between frameworks (NIST publishes one to ISO/IEC 23894), and vendors sell "map once, comply many". Questionnaires still travel as spreadsheets and portal forms. Model cards, labs' system cards and the EU's technical file describe a model, and none of them is a validation report.
07
Networks and counterparties
Who sits between us and the outcome, and what do they want?
Inside the company, banks use the "three lines" model, and other companies copy it loosely:
| Line | In a bank | Elsewhere |
|---|---|---|
| 1st: build and run | Model owner, developer, user | Product owner and the ML or platform team |
| 2nd: challenge | Model risk management: validation, policy, inventory oversight | AI governance office, often in privacy, risk or legal |
| 3rd: check the checkers | Internal audit, which evaluates the program without redoing validation | Internal audit if there is one; a certification body for 42001 |
Outside banks, privacy teams and legal or compliance teams each own AI governance in about a fifth of companies, ahead of IT, in the IAPP's 2025 survey. Around the company sit labs and clouds, SaaS vendors with AI inside, assurance firms, certification bodies and their accreditors, regulators, customers' procurement teams, plaintiffs' lawyers and insurers. Supply is concentrated: in the Bank of England and FCA's 2024 survey, the top three model providers made up 44% of all the model providers UK financial firms named.
08
Regulatory layering
Jurisdiction × activity × entity type: is it a license or a certification?
US bank
- Sector layer
- SR 26-2 (generative AI out); third-party risk guidance; fair lending
- Horizontal layer
- State laws; EU AI Act if it serves people in the EU
- Voluntary and contractual layer
- Examiners' expectations; ISO and NIST as language
EU bank
- Sector layer
- Financial-services governance, which the AI Act accepts for most quality and monitoring duties
- Horizontal layer
- AI Act; GDPR
- Voluntary and contractual layer
US insurer
- Sector layer
- NAIC bulletin or state rules (Colorado, New York)
- Horizontal layer
- State privacy and anti-discrimination law
- Voluntary and contractual layer
- Vendor audit rights
US employer
- Sector layer
- Federal anti-discrimination statutes
- Horizontal layer
- NYC, Illinois, California, Colorado, Connecticut
- Voluntary and contractual layer
- Vendor contract
B2B AI vendor
- Sector layer
- Its customers' sector rules, flowed down by contract
- Horizontal layer
- EU provider duties if its system is high-risk
- Voluntary and contractual layer
- Questionnaires, ISO 42001, AI addenda
One hiring decision in New York by a company based in the EU can touch the AI Act, GDPR, three federal statutes, state and city human-rights law, the city's bias-audit law, FCRA and the vendor's contract.
09
Exceptions and reversals
What goes wrong, and how is it undone?
| Exception | Who starts it | What happens |
|---|---|---|
| Use before validation | The business, for urgent need | Allowed under US bank guidance with limits and closer monitoring; the conditions may never close |
| Out of scope by definition | The guidance | A US bank's generative AI isn't a "model"; each bank decides whether to validate it anyway |
| Use beyond intended purpose | The business | New analysis; in the EU, repurposing into a high-risk use makes you the provider |
| Vendor won't share internals | The vendor | Behavior tests, outcome monitoring, contract terms on change notice and audit |
| Forced change from outside | Lab retirement, cloud auto-upgrade, a settlement | Unplanned re-testing and re-approval |
| Rejected use case | The governance team | Often continues on personal accounts |
| Incident | Monitoring, a complaint, a regulator | Contain, notify, find the cause; EU high-risk providers report serious incidents within 15 days |
| Rule reversed | A government or court | CFPB circulars withdrawn, EEOC guidance removed, an FTC order set aside, Illinois rules withdrawn, Italy's fine on OpenAI annulled |
A rule withdrawn in Washington doesn't remove the statute behind it, and states and private plaintiffs can still sue.
10
Liability allocation
When it fails, who pays?
| Failure | Who absorbs it | Mechanism |
|---|---|---|
| Biased screening or scoring of people | The deployer by default; increasingly the vendor that scores | Anti-discrimination law; California's "agent" rule; Connecticut's no-defense rule; Workday and SafeRent |
| Automated denial without real human review | Patients first, then the insurer in litigation | Contract and good-faith claims; state laws requiring physician review |
| Chatbot misstatement | The company running it | Air Canada |
| Misleading claims about an AI product | The vendor | FTC and state attorneys general (Pieces, accessiBe, DoNotPay) |
| Vendor model changed or retired | The deployer | Notice periods, no compensation |
| Discrimination claim tied to a vendor's tool | Split by contract | Vendor caps of about 12 months of fees; whether discrimination is carved out isn't public |
| AI loss outside insurance cover | The insured company | Generative-AI exclusions in liability forms since January 2026 |
The law reaches whoever makes or shapes the decision about a person: the deployer by default, and more and more the application vendor whose tool scores, ranks or screens. Model labs sit behind usage policies and caps of about a year of fees, and I found no case where a lab was held liable for a deployer's decision about a person. So the evidence splits: the deployer keeps the inventory, oversight design, notices, outcome monitoring and records; the vendor keeps testing, documentation and intended-use limits; contracts decide who pays between them. Vendors' certificates don't move this: AWS says customers aren't "automatically certified by association" with its ISO 42001 certificate.