The Platform PM

Key concepts

Field Guide

AI governance and model risk

04

Key concepts

8 min read · Chapter 4 of 10

01

Entity and identity

What is the unit of record, and how do we know it is the same one?

The governed unit depends on who's asking. US bank guidance governs a "model", which the April 2026 text narrowed to complex quantitative methods built on statistical, economic or financial theory, leaving out spreadsheet arithmetic, deterministic rules and generative AI. Canada's OSFI E-23 widens "model" to include AI and machine learning methods and judgment-based tools. The UK's PRA covers vendor models "regardless of technology". The EU AI Act, NIST and ISO 42001 talk about an "AI system", and most company registers track a use case. I'd key the register on the use case (purpose, users, data, decisions affected) and link models, vendors, prompts and datasets to it, because one model serves many use cases and one use case can swap models.

Roles carry identity too. Banks name a model owner, developer, validator and user. The EU separates the provider, who places a system on the market under its own name, from the deployer, who uses it professionally, and those roles attach per system and can flip. US law keys on the decision-maker's legal role (creditor, employer, insurer, health care provider) and now pulls vendors in as the employer's "agent" (Workday, California's rules), as a consumer reporting agency (a theory in a 2026 suit against Eightfold) or under housing law (SafeRent). For each use case I'd write down which entity decides and which vendor a court could call its agent.

More on Entity and identity →

02

State and lifecycle

What states exist, and what moves an entity between them?

A use case moves through discovered, proposed, in assessment, approved with conditions, live, under review or suspended, changed and re-validated, retired and archived. The April 2026 US guidance is silent on retirement; Canada's E-23 has an explicit decommissioning stage. Vendors run their own states on their own clocks (active, legacy, deprecated, retired, meaning slightly different things at each lab and cloud), and those states drive yours.

Laws have lifecycles too: Colorado's AI Act was enacted in 2024, delayed, paused by a court stipulation, then repealed and replaced in May 2026. Most AI lawsuits in 2026 sit between discovery and class certification, and an ISO 42001 certificate lasts three years, with a surveillance audit each year.

More on State and lifecycle →

03

System of record and ledger

Who owns the truth, and how do systems reconcile?

The inventory is the ledger. US bank guidance wants enough information to understand model risk "at the individual and aggregate levels", and E-23 lists required fields. The fields I'd keep:

FieldWhy it matters
ID, name, ownerSomeone answers for it
Purpose, users, decisions affectedSets the tier and which rules apply
Vendor, model and versionTells you what to re-test when a vendor changes something
Data usedPrivacy and bias questions
Approval, conditions, review dateWhat was agreed and when it expires
Validation dates, findings, open exceptionsUS guidance asks banks to track "recommendations, responses, and exceptions"
Monitoring metrics and incidentsEvidence it still works
Retirement date and archive locationWhat you'd show a court later

Even banks keep it in modest tools: in Deloitte's 2025 survey of 87 banks in Europe, the Middle East and Africa, 24% still used Excel as their main model risk tool. The evidence itself lives in eval runs, traces, tickets and logs.

Litigation turns these records into evidence. Cigna produced about 2.1 million pages in a suit over its claim-review algorithm, and Workday was ordered to produce its federal equal-employment reports. Retention rules set the floor: six months for EU deployer logs of high-risk systems, four years for automated hiring data in California. The test I'd use: if a court asked for every input and output for one applicant from 2024, could we produce them?

More on System of record and ledger →

04

Rules and policy

What logic decides outcomes, and who can change it?

The policy stack runs from an AI policy and an acceptable-use policy down to tiering rules, an approval matrix and vendor standards; in a bank it sits next to the model risk policy. The operational pieces lag: in a 2026 survey by Schellman, which sells ISO 42001 audits, 44% of US companies with 500 or more staff had incident procedures specific to AI.

Tiering combines an outside screen with an inside score. US bank guidance measures materiality as exposure plus purpose, with inherent risk on top; E-23's inherent risk rating sets review intensity, documentation, the approver and monitoring; the EU tiers by use.

The binding rule often sits in a consent order or a contract instead of a statute. Massachusetts' settlement with Earnest, a student lender, requires an AI inventory, risk assessments, testing, documentation and an oversight team, which reads like a bank model risk program. Labs' usage policies require human review in credit, employment, insurance, housing and health, and customer contracts add their own conditions on top.

More on Rules and policy →

05

Effective dating

Which version of the rule applied at that moment?

Dates I'd keep on a calendar as of October 2026:

ChangeEffectiveStatus (Oct 2026)
EU AI Act prohibited practicesFebruary 2, 2025In force; no fines found
California rules on automated hiring decisionsOctober 1, 2025In force
SR 26-2 replaces SR 11-7 for US bank model riskApril 17, 2026In force; not enforceable; excludes generative and agentic AI
Colorado insurance AI rules, auto and health complianceJuly 1, 2026In force
Disparate impact removed from Reg BJuly 21, 2026In force
EU Article 50 transparency dutiesAugust 2, 2026In force
Connecticut: relying on AI is no defense to discriminationOctober 1, 2026In force
EU marking grace ends; new prohibition on generated intimate imagesDecember 2, 2026Upcoming
Colorado's replacement AI law; California automated-decision complianceJanuary 1, 2027Upcoming; Colorado may slip
OSFI E-23 for Canadian banks and insurersMay 1, 2027Upcoming
Connecticut pre-decision notices in employmentOctober 1, 2027Upcoming
EU high-risk duties for Annex III uses (credit, hiring, insurance pricing)December 2, 2027Upcoming; moved from August 2, 2026
EU high-risk duties for regulated productsAugust 2, 2028Upcoming; moved from August 2, 2027

Dates here slip often: Colorado's moved twice, the EU's high-risk date by 16 months, and the Commission's guidelines on classifying high-risk systems, due by February 2, 2026, came out as a draft in May. Vendors' retirement dates are often "not sooner than" dates.

More on Effective dating →

06

Interfaces and standards

What format and protocol do counterparties speak?

StandardWhat it isWho asks for it
NIST AI RMF and its generative AI profile (2024)Voluntary US framework; inventory is GOVERN 1.6; under revision, no 2.0 by October 2026US buyers and regulators, as shared language
ISO/IEC 42001Certifiable AI management systemEnterprise procurement
ISO/IEC 42005 and 42006 (2025)Guidance for impact assessments; rules for the bodies that certify to 42001Assessment teams; buyers checking a certificate
CSA AI-CAIQ and STAR for AIA standard AI questionnaire; Level 1 self-assessed, Level 2 validated and requiring 42001 (from November 20, 2025)Enterprise buyers
SIG questionnaireShared Assessments' vendor questionnaire: 128 questions (Lite), 627 (Core), 1,936 (Detail)Enterprise buyers
SOC 2No AI-specific criteria found by October 2026; AI systems are scoped into ordinary reportsEnterprise buyers
NYC bias auditImpact ratios by sex, race and ethnicity, and their intersectionsNYC employers
EU code on marking AI content (June 10, 2026)Voluntary layered marking, such as metadata plus a watermark, and a common EU iconEU regulators

The interface that matters most is the crosswalk between frameworks (NIST publishes one to ISO/IEC 23894), and vendors sell "map once, comply many". Questionnaires still travel as spreadsheets and portal forms. Model cards, labs' system cards and the EU's technical file describe a model, and none of them is a validation report.

More on Interfaces and standards →

07

Networks and counterparties

Who sits between us and the outcome, and what do they want?

Inside the company, banks use the "three lines" model, and other companies copy it loosely:

LineIn a bankElsewhere
1st: build and runModel owner, developer, userProduct owner and the ML or platform team
2nd: challengeModel risk management: validation, policy, inventory oversightAI governance office, often in privacy, risk or legal
3rd: check the checkersInternal audit, which evaluates the program without redoing validationInternal audit if there is one; a certification body for 42001

Outside banks, privacy teams and legal or compliance teams each own AI governance in about a fifth of companies, ahead of IT, in the IAPP's 2025 survey. Around the company sit labs and clouds, SaaS vendors with AI inside, assurance firms, certification bodies and their accreditors, regulators, customers' procurement teams, plaintiffs' lawyers and insurers. Supply is concentrated: in the Bank of England and FCA's 2024 survey, the top three model providers made up 44% of all the model providers UK financial firms named.

More on Networks and counterparties →

08

Regulatory layering

Jurisdiction × activity × entity type: is it a license or a certification?

US bank

Sector layer
SR 26-2 (generative AI out); third-party risk guidance; fair lending
Horizontal layer
State laws; EU AI Act if it serves people in the EU
Voluntary and contractual layer
Examiners' expectations; ISO and NIST as language

EU bank

Sector layer
Financial-services governance, which the AI Act accepts for most quality and monitoring duties
Horizontal layer
AI Act; GDPR
Voluntary and contractual layer

US insurer

Sector layer
NAIC bulletin or state rules (Colorado, New York)
Horizontal layer
State privacy and anti-discrimination law
Voluntary and contractual layer
Vendor audit rights

US employer

Sector layer
Federal anti-discrimination statutes
Horizontal layer
NYC, Illinois, California, Colorado, Connecticut
Voluntary and contractual layer
Vendor contract

B2B AI vendor

Sector layer
Its customers' sector rules, flowed down by contract
Horizontal layer
EU provider duties if its system is high-risk
Voluntary and contractual layer
Questionnaires, ISO 42001, AI addenda

One hiring decision in New York by a company based in the EU can touch the AI Act, GDPR, three federal statutes, state and city human-rights law, the city's bias-audit law, FCRA and the vendor's contract.

More on Regulatory layering →

09

Exceptions and reversals

What goes wrong, and how is it undone?

ExceptionWho starts itWhat happens
Use before validationThe business, for urgent needAllowed under US bank guidance with limits and closer monitoring; the conditions may never close
Out of scope by definitionThe guidanceA US bank's generative AI isn't a "model"; each bank decides whether to validate it anyway
Use beyond intended purposeThe businessNew analysis; in the EU, repurposing into a high-risk use makes you the provider
Vendor won't share internalsThe vendorBehavior tests, outcome monitoring, contract terms on change notice and audit
Forced change from outsideLab retirement, cloud auto-upgrade, a settlementUnplanned re-testing and re-approval
Rejected use caseThe governance teamOften continues on personal accounts
IncidentMonitoring, a complaint, a regulatorContain, notify, find the cause; EU high-risk providers report serious incidents within 15 days
Rule reversedA government or courtCFPB circulars withdrawn, EEOC guidance removed, an FTC order set aside, Illinois rules withdrawn, Italy's fine on OpenAI annulled

A rule withdrawn in Washington doesn't remove the statute behind it, and states and private plaintiffs can still sue.

More on Exceptions and reversals →

10

Liability allocation

When it fails, who pays?

FailureWho absorbs itMechanism
Biased screening or scoring of peopleThe deployer by default; increasingly the vendor that scoresAnti-discrimination law; California's "agent" rule; Connecticut's no-defense rule; Workday and SafeRent
Automated denial without real human reviewPatients first, then the insurer in litigationContract and good-faith claims; state laws requiring physician review
Chatbot misstatementThe company running itAir Canada
Misleading claims about an AI productThe vendorFTC and state attorneys general (Pieces, accessiBe, DoNotPay)
Vendor model changed or retiredThe deployerNotice periods, no compensation
Discrimination claim tied to a vendor's toolSplit by contractVendor caps of about 12 months of fees; whether discrimination is carved out isn't public
AI loss outside insurance coverThe insured companyGenerative-AI exclusions in liability forms since January 2026

The law reaches whoever makes or shapes the decision about a person: the deployer by default, and more and more the application vendor whose tool scores, ranks or screens. Model labs sit behind usage policies and caps of about a year of fees, and I found no case where a lab was held liable for a deployer's decision about a person. So the evidence splits: the deployer keeps the inventory, oversight design, notices, outcome monitoring and records; the vendor keeps testing, documentation and intended-use limits; contracts decide who pays between them. Vendors' certificates don't move this: AWS says customers aren't "automatically certified by association" with its ISO 42001 certificate.

More on Liability allocation →