The Platform PM

Rules and risks

Field Guide

AI governance and model risk

07

Rules and risks

7 min read · Chapter 7 of 10

How the rules work

EU AI Act. It sorts uses into four tiers. Prohibited practices, banned since February 2, 2025, include social scoring, manipulative techniques causing significant harm, emotion recognition at work and in education, and untargeted scraping for facial recognition databases, with fines up to €35 million or 7% of worldwide turnover. High-risk uses include employment, education, credit scoring, life and health insurance pricing, biometrics and critical infrastructure (Annex III), plus AI inside regulated products (Annex I). Article 50 transparency duties cover chatbots, synthetic content and deepfakes, and the rest is minimal risk.

The Digital Omnibus (Regulation (EU) 2026/1744, in force since July 27, 2026) moved Annex III high-risk duties from August 2, 2026 to December 2, 2027, and Annex I to August 2, 2028. Article 50 kept August 2, 2026, with a grace period to December 2, 2026 for machine-readable marking on systems already on the market. The omnibus also added a prohibition, from December 2, 2026, on AI that generates non-consensual intimate images or child sexual abuse material.

Most high-risk duties sit with the provider: risk management, documentation, a quality management system, conformity assessment (for credit and hiring, the provider's own internal check), registration, post-market monitoring and serious-incident reports. The deployer, the lender in our example, must follow the instructions, assign competent human oversight, check its input data, monitor, keep logs for at least six months and tell people when a high-risk system is used in decisions about them. Public bodies and deployers doing credit scoring or life and health insurance pricing also owe a fundamental rights impact assessment, and can reuse a GDPR assessment for it. A deployer that rebrands a system, modifies it substantially or repurposes a general-purpose model for a high-risk use becomes its provider. Breaching the deployer or Article 50 duties can cost up to €15 million or 3% of turnover (the lower of the two for small companies). As of October 9, 2026 no AI Act fine had been found, and the Commission's guidelines on what counts as high-risk were still a draft.

US banks. On April 17, 2026 the Federal Reserve (SR 26-2), the OCC (Bulletin 2026-13) and the FDIC replaced the 2011 guidance, SR 11-7, with 12 pages. The new text says it doesn't set enforceable standards and that not following it "will not result in supervisory criticism", though unsafe or unsound practices can still be acted on. It's "most relevant" for banks above $30 billion in assets, drops fixed validation cycles and detailed independence rules, keeps effective challenge, and keeps vendor models in scope even when the vendor won't share code or data. Footnote 3 says generative and agentic AI models "are not within the scope of this guidance", and that the bank's broader risk management should guide controls for them.

For the lender's assistant, that means no supervisory guidance requires a model risk validation of it any more. The lender can still run it through model risk management, as some large banks reportedly did by default for generative AI, Copilot included, before the change. Either way the assistant needs controls under the bank's general risk framework, and examiners can still act on unsafe practices. The agencies promised a request for information on AI, generative and agentic included, "in the near future"; none had appeared in the Federal Register by mid-September 2026. Vendor AI relationships now fall under the guidance on outside vendors, whose replacement the agencies proposed on September 11, 2026, with comments due November 10. Banks elsewhere doubt the fit too: only 29% in Deloitte's survey thought their frameworks fully adequate for AI.

Canada and the UK. OSFI's E-23 was finalized on September 11, 2025 and takes effect on May 1, 2027. It covers every federally regulated bank, insurer and trust and loan company, defines models to include AI and machine learning, requires an inventory of models with more than negligible risk, lets the risk rating drive review and approval, and ends with decommissioning; generative AI isn't named. The UK's PRA SS1/23, in force since May 17, 2024, applies to banks with internal-model approval, covers vendor models regardless of technology and names a senior manager as responsible.

US credit. The Reg B rules on adverse action (specific principal reasons, within 30 days) didn't change. The CFPB's final rule effective July 21, 2026 removed the disparate-impact "effects test" from Reg B, over about 64,500 comments. Disparate treatment and proxies remain unlawful, and state fair-lending law still reaches disparate impact, which is how Massachusetts reached Earnest.

US hiring. An April 2025 executive order told agencies to eliminate disparate-impact liability "to the maximum degree possible", and the EEOC withdrew its AI guidance under Title VII and the ADA. The statutes and private suits are unchanged, and the action moved to cities and states. New York City has required an independent bias audit less than a year old, a public summary of impact ratios and notice to candidates since July 2023. Illinois made discriminatory AI in employment decisions a civil-rights violation from January 1, 2026. California's rules, from October 1, 2025, treat a vendor that screens or ranks for an employer as its agent and require four years of records. Connecticut makes relying on AI no defense to a discrimination claim from October 1, 2026. Colorado replaced its 2024 AI Act with a narrower law effective January 1, 2027 (notice, an explanation of adverse outcomes within 30 days, correction and human review), which drops the old law's exemption for small companies and its defense for following NIST or ISO. Enforcement is paused by agreement in a suit xAI brought and the Justice Department joined.

US insurance and health care. The NAIC's model bulletin of December 2023 asks insurers for a written AI program, governance and vendor oversight with audit rights, and about 25 states had adopted it by mid-2026, by law-firm and vendor counts. Colorado's regulation is binding for life, auto and health insurers. No enforcement under an adopted bulletin had been reported by early 2026. In health care, an HHS rule has required hospitals and insurers since May 1, 2025 to make reasonable efforts to find and reduce discrimination in patient care decision tools, and at least seven states passed laws in 2026 barring AI as the sole basis for prior-authorization denials.

The FTC and preemption. The FTC's 2023 order against Rite Aid faulted it for not asking its two vendors how their facial recognition was tested. Since 2025 the FTC has kept bringing AI deception cases and set aside a broader order against Rytr. No federal law preempted state AI laws by October 9, 2026: a December 2025 executive order set up a Justice Department task force against them, and a House draft would preempt state rules on model development for three years. Even that draft leaves rules on use alone, so I'd expect the deployer-facing laws to stay.

What mistakes cost

Let's say an employer with 5,000 staff and 60,000 applicants a year, hiring in New York City, Illinois, California, Colorado and Connecticut, buys the HR software company's ranking feature, which runs on a lab's model. My rough arithmetic, on stated assumptions:

  • Running it properly: the module at $50,000-250,000 a year, a bias audit at perhaps $10,000-60,000 a year (no prices are published), and $20,000-100,000 once for notices, opt-outs and four-year record keeping. The lab's share is a fraction of a cent per applicant.
  • If the tool skews against applicants over 40: New York City penalties of $500-1,500 per violation, each day without an audit counting separately; damages under Illinois law; and, if a court accepts the theory that the vendor is a consumer reporting agency, $100-1,000 per willful violation, a theoretical $6-60 million on 60,000 applicants. Real settlements have been far smaller: iTutorGroup paid $365,000 to more than 200 applicants, about $1,800 each, and SafeRent $2.3 million. Defense through class certification could run from the high six figures to the low eight.
  • Who pays: the employer always, because it made the decision, and in Connecticut relying on the tool is no defense. The HR software company is now a defendant in its own right; its contract probably caps its liability at 12 months of fees, $50,000-250,000 here, unless discrimination claims are carved out, which I found no public data on. The lab pays close to nothing, since its usage policy required human review, and general liability insurance increasingly excludes AI.

The record so far:

CaseWhat happenedWho paid
EEOC v. iTutorGroup, September 2023Software rejected women aged 55 and over and men aged 60 and overThe employer: $365,000 and monitoring
FTC v. Rite Aid, December 2023Thousands of false facial-recognition matches; no testing before launchThe deployer: a five-year ban and deleted models, no money
Texas v. Pieces Technologies, September 2024A health AI vendor claimed a "critical hallucination rate" below 0.001%The vendor: five years of accuracy disclosures, no money
Louis v. SafeRent, November 2024Tenant-screening scores challenged under the Fair Housing Act, though landlords made the final callThe vendor: $2.3 million and no scores for voucher holders for five years
Italy's data protection authority, 2024-2025GDPR fines of €15 million on OpenAI and €5 million on Replika's makerOpenAI's fine was annulled by a Rome court in March 2026
Massachusetts v. Earnest, July 2025AI underwriting models with alleged disparate impact on Black, Hispanic and non-citizen applicantsThe lender: $2.5 million and a governance program
Mobley v. Workday, ongoingApplicants allege age, race and disability bias in Workday's screening toolsNobody yet. On June 22, 2026 the court held Workday can be directly liable under California's FEHA for its own conduct on employers' behalf, rejected its argument that its liability depends on its customers', let disability proxy claims proceed and struck some new race claims. No liability finding or settlement
UnitedHealth, Humana, Cigna, 2023 onwardSuits over algorithms used in claim denialsPending: claims survived dismissal; Cigna's class-certification motion is due October 29, 2026

Regulators' payments in AI cases have been small, from nothing to $2.5 million. The cost so far is defense, remediation and multi-year compliance programs, and the evidence a company kept is what lowers both. Shadow AI has a price too: in IBM's 2025 breach study, companies with heavy shadow-AI use paid about $670,000 more per breach, a correlation but a large one.