02
How it works
4 min read · Chapter 2 of 10
How a use case gets approved, step by step
Here's the lender's assistant, from proposal to retirement. The riskier the use, the more testing and oversight it gets before launch. After launch the model can change under you, through drift or a vendor's update, so monitoring never ends until the use case is retired.
- Proposed. A business owner submits the use case: purpose, users, data, which decisions it touches, which vendor and model. It gets an ID, an owner and a status in the inventory. Someone also has to find the AI already in use (SaaS features, browser extensions, personal accounts), and that list usually comes from security tools. What can go wrong: most AI never gets proposed. In a 2026 survey by OneTrust, which sells governance software, a third of companies said staff used unapproved AI because the approved route was too slow, so a slow intake creates the shadow AI it's meant to catch.
- Assessed. The use case is risk tiered. Is it prohibited? High-risk under the EU AI Act (credit scoring is, from December 2, 2027)? A consequential decision under a state law? How material is it, by exposure (how many decisions, how much money) and purpose? Then come impact assessments (the EU's fundamental rights assessment for credit scoring, a GDPR assessment, or ISO/IEC 42005), vendor due diligence on data use and notice of model changes, and legal review. The exception on the diagram, Blocked, happens here: the use is prohibited, such as emotion recognition at work in the EU, or too risky for the controls on offer. In a US bank there's an extra fork: under the April 2026 guidance a generative AI assistant isn't a "model", so it goes to a separate AI register. What can go wrong: the use case is under-tiered and grows quietly; the same tool counts as a model under one definition and not another; a rejected use case carries on through personal accounts.
- Validated. The use case is tested: task evals on real loan files with known answers, bias tests on the draft letters, red teaming for prompt injection and data leaks, and a sample of outputs checked by people. In banks this is "effective challenge", critical review by people with the expertise, independence and standing to force a change. A foundation model can't be validated like a credit score; one large bank's AI head told Risk.net "You cannot validate a foundation model", so the work shifts to testing behavior. What can go wrong: validation becomes box-ticking; the vendor won't share what proper testing needs; tests pass and nobody records which model version they ran on.
- Deployed. The use case is approved, usually with conditions: a human reviews each letter, volumes are capped, monitoring thresholds and a review date are set. Launch brings the notices (adverse-action reasons under Reg B, an AI disclosure under EU Article 50 for a chatbot), user training and logging; EU deployers of high-risk systems keep logs for at least six months. US bank guidance lets a model go live before validation finishes when the need is urgent, with limits and closer monitoring. What can go wrong: the conditions never get closed; the human reviewer approves everything; the conditional launch becomes permanent.
- Monitored. The use case is watched: quality and drift metrics, how often underwriters override the draft, complaints, incidents, cost, and the vendor's change and retirement notices. That's the diagram's note, because models change under you. In a 2023 study, GPT-4's accuracy at identifying prime numbers fell from 84% to 51% between its March and June versions; critics said that measured a shift in default answers, but the same model name behaved differently either way. Lab retirement notices run from about 60 days to six months, and every change means re-testing and sometimes re-approval. What can go wrong: the vendor retires the model and nobody re-tests; an incident isn't escalated (in the same OneTrust survey, only 27% of companies said they had slowed or paused a deployment in response to AI incidents).
Retired is the other exception, when the use case is replaced or failed. The owner tells users, switches to a fallback, archives the model version, prompts, eval results and logs, and watches for effects downstream. Canada's OSFI E-23 is the only regulator text I found that spells this out, down to keeping the retired model as a benchmark or fallback, which is impossible for a vendor model you never held. Records outlive the model: EU deployer logs, California's four-year rule for hiring data, any court order to reproduce a decision.
Cheat sheet: which rules reach which use (October 2026)
Credit decisions (US)
- In force now
- Reg B adverse-action reasons; state fair-lending law; FCRA notices
- Coming
- Colorado (January 1, 2027, may slip); California automated-decision rules (January 1, 2027)
- Who carries it
- The lender
Credit scoring or life and health insurance pricing (EU)
- In force now
- GDPR automated-decision rights; Article 50 if a chatbot faces customers
- Coming
- High-risk duties and a fundamental rights impact assessment (December 2, 2027)
- Who carries it
- Deployer: oversight, logs, assessment. Provider: conformity
Hiring and promotion
- In force now
- NYC bias audits; Illinois; California's rule treating vendors as agents; Connecticut's "AI is not a defense"; federal statutes through private suits
- Coming
- California and Colorado (January 1, 2027); Connecticut notices (October 1, 2027); EU (December 2, 2027)
- Who carries it
- The employer, and the vendor as its agent
Insurance pricing and claims (US)
- In force now
- NAIC bulletin in about 25 states; Colorado's regulation; New York guidance; state limits on AI-only prior-authorization denials
- Coming
- NAIC evaluation tool
- Who carries it
- The insurer
Customer chatbot
- In force now
- EU Article 50 disclosure; FTC deception rules; the Air Canada ruling
- Coming
- EU marking grace ends December 2, 2026
- Who carries it
- The company running it
Health care decision support (US)
- In force now
- HHS rule on discrimination in patient care tools; FDA change plans for AI devices
- Coming
- Who carries it
- The hospital or insurer
Internal assistant, no decision about a person
- In force now
- Company policy, vendor terms, customer contracts; in a US bank, "broader risk management"
- Coming
- OSFI E-23 for Canadian banks and insurers (May 1, 2027)
- Who carries it
- The company