AI coding agents: writing, reviewing and shipping code

Picture a company that sells invoicing and payments software to other businesses, with 200 engineers. A developer there hands a task to a coding agent: upgrade the PDF library and fix whatever breaks. The agent might sit in the developer's editor, run in a terminal or work in the cloud from a GitHub issue. Wherever it runs, it sends the task and the code it reads to a model from a lab such as Anthropic, OpenAI or Google, and pays in tokens (the chunks of text a model reads and writes). It edits the company's repo, where the code, the tests and the rules files written for agents live, and opens a pull request, a proposed change that people review before it's merged. Tests and reviewers then decide what ships, and that's where the work now piles up. After the deploy, errors and traces from production can flow back to the agent as its next task.

Writing code got cheap quickly: in July 2026 Microsoft said one in three pull requests on GitHub involves an agent. Checking that code hasn't got cheaper at the same rate.

What I'd want a new PM in this space to take away:

1. Review: review and CI are the new bottleneck. Agent pull requests merge less often than people's, and the gap isn't closing. A July 2026 study of about 9,400 agent pull requests in 489 open-source Python repos found that, adjusted for repo and task, about 65% merged against about 85% for human ones. Reviewers often abandon them, and the ones that don't merge are bigger and fail CI (continuous integration, the automated builds and tests every change must pass) more often. Amazon reportedly now requires senior sign-off on AI-assisted changes by junior and mid-level engineers after a run of outages, though it disputes the link to AI. Nobody has a causal 2026 study of agents inside companies: METR paused its trial design in February 2026, and DORA, Google's long-running research program on software delivery, had published no 2026 AI report by October 9. See Do coding agents make teams faster: what's real.
2. Environment: the model sets the ceiling, and the environment decides how much of it you reach: a repo the agent can build, tests it can trust and run quickly, and permissions scoped to the task. Executable checks do more than written instructions: in a February 2026 study, AGENTS.md files (instructions for agents kept in the repo) gave no significant gain in task success and raised inference cost by about a fifth. Public benchmarks are breaking down too: OpenAI dropped SWE-bench Verified in February 2026 and withdrew its recommendation of SWE-bench Pro in July. The only hard gate is the pull request on the repo host, where approvals and required checks live.
3. Pricing: pricing moved to usage. Since June 1, 2026 GitHub has charged Copilot's AI Credits at each model's published API rate, and Microsoft said in July that the change improved its margins. A $19 seat now buys roughly one and a half to three and a half hours of heavy agent work on a model like Anthropic's Opus 5.5, and past that you're buying tokens. For the invoicing company my estimate runs from about $50,000 a year (seats only, agents capped) to about $1.8 million (agents for everyone, with a heavy tail of power users), or 0.1-5% of its engineering payroll. See How the money moves.
4. Power: labs sell their own agents (Claude Code, Codex, Jules, Antigravity) to the same buyers as the app companies that pay them for tokens, and they use control of model supply. Anthropic cut Windsurf's direct access to Claude in June 2025 while OpenAI was trying to buy it. In 2026 SpaceX bought Cursor in a deal reported at $60 billion, and OpenAI then said it would stop supplying its models to Cursor on November 12, 2026, according to press reports. App companies escape by routing work to their own or cheaper models, or by selling. See Who holds the power.
5. Risk: generated code is no more secure than a year ago. In Veracode's 2026 test, AI-written code passed its security checks about 56% of the time, against 55% in 2025. Attackers now go after agents running in CI, through issue text the agent reads as instructions (PromptPwnd, Clinejection), and after agents installed on laptops (the Nx malware). Agents have deleted real data, from Replit's production database to a user's files with Gemini CLI. On copyright, the Ninth Circuit rejected the DMCA claim over Copilot's output on September 16, 2026. Indemnities cover intellectual property only, and damage from an agent's actions falls on the customer under liability caps of about 12 months of fees, as What mistakes cost shows.

How agents run inside production apps (orchestration, tool security, evals) is in AI agents: orchestration platforms, and I don't repeat it here. Production feedback, including Dynatrace's Bluebox, is in Observability, code and supply-chain scanning in Cloud security (CNAPP), and agents that talk on the phone in Voice AI. How a leadership team rolls these tools out and measures them is in Becoming AI native.

The main players

These are the companies behind the diagram's parties, layer by layer, in no particular order.

How they make money, and who's moving:

• Labs sell tokens, subscriptions and enterprise seats plus usage. Anthropic said in February 2026 that Claude Code had passed $2.5 billion in annualized revenue, more than double since January 1, and it bought Bun, the JavaScript runtime, in December 2025. OpenAI says Codex passed 5 million weekly active users by June 2026, and it agreed to buy Ona (formerly Gitpod) for persistent cloud environments. Alphabet said in July 2026 that more than 2.4 million people use Antigravity each week, a year after Google paid $2.4 billion to license Windsurf's technology and hire its leaders. SpaceXAI's Grok models now sit in the same company as Cursor, and Mistral sells Devstral 2, an open-weight coding model, at about $0.40 per million input tokens. The user and revenue figures here are the companies' own.
• Coding agents and editors charge a seat with some usage included, then usage at or near the lab's API rates. Microsoft said GitHub Copilot had 50 million users in July 2026, with 4.7 million paid subscribers in January, and it now routes some Copilot work to its own model, MAI-Code-1-Flash. Cursor reportedly passed $2 billion of annualized revenue in February 2026 and $4 billion by June, according to people familiar with it. AWS is moving Amazon Q Developer users to Kiro: Q Developer closed to new signups on May 15, 2026 and loses support on April 30, 2027. Open-source agents such as opencode, Codex CLI and Gemini CLI, each with over 100,000 GitHub stars, set a free floor.
• Background agents charge a seat plus usage at API rates. Cognition, which makes Devin and bought what was left of Windsurf in July 2025, was reportedly valued at about $47-48 billion in September 2026. GitHub's Agent HQ, in public preview since February 2026, lets a Copilot customer hand the same issue to Copilot, Claude or Codex.
• App builders sell credits on top of a subscription. Lovable reportedly reached about $500 million of annualized revenue in June 2026 and raised $400 million at $13.3 billion in August; Replit raised $400 million at $9 billion in March 2026; I found no 2026 figure for Bolt. They pay the labs list prices, and their margins showed it: in 2025 Replit's gross margin reportedly swung from 36% to minus 14%, and Lovable's and Bolt's sat at about 35-40%.
• Repo hosts and CI charge seats, CI minutes and security add-ons. GitLab reported $955 million of revenue for its 2026 fiscal year and called its AI revenue "modest". Cursor opened an early beta of its own git host, Origin, in August 2026.
• Review and security sell per developer or per pull request author, with usage on top. CodeRabbit raised $143 million at $1.5 billion in August 2026, Cursor bought Graphite in December 2025, and Socket and Aikido each reached $1 billion valuations in 2026. Snyk, the incumbent, reportedly cut about 90 jobs in June 2026. The labs are moving in: Anthropic's Claude Security and OpenAI's Codex Security entered research preview in February and March 2026, and Google DeepMind announced CodeMender in October 2025.

As of October 2026. Most revenue and valuation figures here are self-reported or come from press coverage of funding rounds, so treat the list as a map to check before relying on it.

Back to the invoicing company. After a deploy, Sentry's Seer agent traces a spike of errors in the PDF export and hands the fix to Copilot's cloud agent, which calls a model from Anthropic or OpenAI, builds the repo in a sandbox and runs the tests until they pass. On its pull request CodeRabbit comments, Snyk scans the new dependency, GitHub Actions runs CI, and a senior engineer who didn't ask for the change approves. After the next deploy, Bluebox checks the traces. One small fix touches six or seven companies, each with its own meter, and only the engineer's approval is a hard gate (the vendors in the story are illustrative).

How an agent's task gets done, step by step

Here's the PDF-library upgrade, run by a background agent from a GitHub issue. An agent is only as good as its definition of done. Where tests and CI are strong, it can loop until they pass, and where they're weak a person has to catch what it got wrong, sometimes only after it reaches production.

1. Task given: an issue, a prompt, a chat thread, a production alert or a schedule starts the work. On GitHub only someone with write access can start Copilot's cloud agent, and comments from anyone else never reach it. Breaks: an underspecified issue leaves the agent to guess what done means (Devin's docs ask for explicit completion criteria); an automation with nobody behind it gets its pull request credited to whoever set it up, who then can't approve it.
2. Planned: with the context gathered (AGENTS.md or CLAUDE.md, a search of the repo, the issue), the agent drafts a plan of files and steps. Some products stop for approval here: Jules shows its plan before changing any code, Kiro writes requirements, design and task files, and Claude Code has a read-only plan mode. Breaks: the agent misses the file that matters; a long session compacts its history and drops an early instruction; in Claude Code's experimental agent teams the lead agent approves its teammates' plans, which is a plan gate with no person at it.
3. Changed: the agent writes code and produces a diff in a sandbox, either an ephemeral container or virtual machine in the cloud or the developer's own machine, after a setup script installs the dependencies. What it can reach differs: Codex keeps the network off by default while the agent works and gives secrets only to the setup step, Copilot's agent works behind a firewall with an allowlist, and Cursor's cloud agents have the network on, with optional domain limits, and get secrets when they start. Breaks: the build fails on a private registry, a missing credential or a database the agent can't reach, which is the usual first failure; Copilot's sessions stop at 59 minutes and can't be extended.
4. Verified: the agent runs the tests, linters and type checks, reads the errors and edits again until the tests and CI are green. That loop is the Retried exception on the diagram, and it's where the work gets good or goes wrong. Copilot also runs code, secret and dependency scans on its agent's changes, and Cursor's Bugbot, an AI reviewer, caps its own automatic fixes at three attempts per pull request. Breaks: flaky tests send the loop the wrong way, and an agent may edit or delete tests to get to green. In ImpossibleBench, a research benchmark where the only way to pass is to cheat, GPT-5 exploited the tests 76% of the time on one variant and Claude models tended to edit the tests. The researchers point to protected, read-only tests.
5. Merged: a person approves and merges. On GitHub the agent can't mark its own pull request ready, approve it or merge it, the person who asked for the change doesn't count as an approver, and CI won't run on the agent's pull request until someone with write access clicks "Approve and run workflows". The normal deploy follows, with feature flags or a canary, and the agent holds no production credentials. Breaks: the reviewer abandons the pull request, the most common failure in a 2026 study of 33,000 agent pull requests; a large diff gets waved through; two parallel agents open duplicates.

Reverted is the exit after the merge: the change broke something in production and comes back out. Git makes code easy to revert, but data is another matter. Claude Code's own docs say its checkpoints undo file edits only, and that actions on remote systems (databases, APIs, deployments) can't be checkpointed. Production tools now close the loop: Sentry's Seer hands a root cause to Claude Code, Cursor or Copilot, and Bluebox opens a GitHub issue with the trace and the offending lines for a coding agent, though Dynatrace says Bluebox "never pushes a fix on its own".

A local agent in an editor or terminal takes a shorter path. The developer prompts, the agent edits the working copy and runs commands under a permission mode (ask every time, accept edits, or a bypass mode that asks nothing), and the developer commits. The agent acts with the developer's own credentials and files, the riskiest case: most deleted-data incidents in What mistakes cost happened that way. Anthropic says sandboxing Claude Code at the operating-system level cut its internal permission prompts by 84%, so autonomy there was bought with environment controls.

An app builder skips the pull request. Lovable, Replit or Bolt generates the app, its database and its hosting, and "publish" puts it live. The builder is repo host, CI and production at once, so its failures are data failures: in 2025 a researcher found exposed database tables in 170 of 1,645 apps showcased by Lovable, which said securing them was the customer's job.

Cheat sheet: the product shapes

Jules, Devin and Claude Code on the web didn't state their network and secrets defaults in the pages I read, so I'd ask each vendor before letting its agent near private code.

Which coding agent setup? Five questions

1. What counts as done? Agents land work that has an objective check. Merge rates in the 2026 study were highest, at 80% or more, for CI and build changes, GitHub Actions workflows, dependency bumps, docs and typo fixes, and lowest for new functions and code that calls language models. If a task has no test, a person is the test.
2. Can it build and test? If the build needs a private registry, a database and three secrets, write the setup script before buying more seats.
3. What can it reach? A local agent acts as the developer. Cloud agents differ on network and secrets defaults. Production credentials should be out of reach either way, and anything in an issue or a pull request comment is untrusted input.
4. Who reviews? Every pull request an agent opens lands in someone's queue. On card-payment systems covered by PCI DSS the reviewer can't be the author, and whether the person who prompted the agent counts as the author hasn't been tested; I'd ask the assessor before it comes up in an audit.
5. Who pays for the tail? On seats the vendor eats it until it re-prices, and on usage you do. Set budgets per team and user, and decide about overage up front.

My defaults: for the invoicing company I'd give every engineer an editor assistant on a business plan (no training on our code, indemnity on, public-code filter on) and give background agents to a few teams, starting with dependency upgrades, flaky tests and migrations. Every agent pull request goes through the same branch protection as a person's, with a size limit and an approver who didn't prompt the change. Agents get their own identity, no production credentials, the network off or allowlisted, and a setup script that builds the repo from scratch. I'd budget on usage with team caps, track cost per merged change rather than per seat, and before the rollout capture a few weeks of baseline: pull request cycle time split into waiting and reviewing, pull request size, change failure rate and reverts.

What's different here

How the money moves

Underneath, every product now prices the same way: a seat with some usage included, then usage at or near the lab's API rates. List prices in October 2026, rounded:

Under the seats sit token prices. Anthropic lists Opus 5.5 at $4 per million input tokens, $0.20 for cached input and $20 for output, and Sonnet 5.5 at half that. Cursor prices its own Composer 2.5 at $0.50 in and $2.50 out, and Mistral's Devstral 2 costs about the same. OpenAI prices Codex in credits whose dollar value depends on the plan or agreement.

A seat buys little agent time. An agent working on a big task makes 60-150 model requests an hour, each carrying about 150,000 tokens of context, most of it cached. By my arithmetic that's about $5.50-14 an hour on Opus 5.5 and $3-7 on Sonnet 5.5, so a $19 Copilot Business seat covers roughly one and a half to three and a half hours on Opus 5.5, if GitHub's credits track list prices as it says. Anthropic's own figure for Claude Code is an average of about $13 per developer per active day, or $150-250 a month, with nine in ten users under $30 a day. The tail is the problem: companies told one newsletter of engineers spending $500 a day and of a $10,000 week caused by a caching error.

Flat plans kept breaking on heavy agent users in 2025 and 2026: a tail costing 10-100 times the plan price, then caps, then usage pricing, then protests. Cursor moved its Pro plan to usage in June 2025, then refunded surprise charges and its CEO apologized. Anthropic added weekly caps in August 2025 after one account ran tens of thousands of dollars of usage on a $200 plan. GitHub paused self-serve Business signups on April 22, 2026 for lack of capacity, metered everything from June 1 and reopened signups on September 3, and some Copilot users reported projected bills rising from $29 to about $750 a month.

The app layer's margin is set by what it pays the labs. In 2025 Windsurf's margins were reportedly "very negative". Cursor ran negative gross margins until early 2026 and reached a slightly positive one by routing to its own Composer model and to cheaper ones such as Kimi, according to four unnamed sources.

A year of AI coding at the invoicing company

My assumptions: 200 engineers, of whom 40 are heavy agent users, 100 regular and 60 light; about 180 open pull requests; a loaded cost of $176,000-250,000 per engineer, so a payroll of about $35-50 million. Heavy users cost $800-3,000 a month in usage, regular users $150-250, light users $10-50, at list prices.

The seats-only setup looks cheap, and in 2026 it works as a budget cap: the heavy users would burn the pooled credits in about a week, and then agents pause for everyone until the next cycle. The usage-first range runs from the vendor's average ($0.41-0.65 million) to a heavy tail. Across all three, AI coding costs about 0.1-5% of engineering payroll. At the top of the mixed setup it pays for itself if each engineer ships about 2% more; at the top of the usage-first tail it needs about 4-5% more. The bill is dominated by the heaviest fifth of users, and model choice (Opus against Sonnet is about two times), cache misses (about $0.75 each on Opus) and parallel agents move it more than the choice of seat. I found no reliable data on discounts, so I'd budget at list.

What one merged change costs

My rough arithmetic for one background-agent task, from a small fix to a multi-hour job:

Human review is the biggest line in nearly every case, which is the bottleneck written in money. The merge rate multiplies everything: at 40%, the tokens behind each merged change cost two and a half times the per-task figure. If 40 senior engineers each absorb three to eight extra hours of review a week, that's roughly $0.5-1.4 million a year, more than most of the tool bills above. Security tooling adds less: GitHub's Secret Protection and Code Security at $19 and $30 per active committer a month come to about $118,000 a year for 200 committers.

Who holds the power

• Labs hold the scarce input and now the fastest-growing agent products, and they use supply. Anthropic cut Windsurf's direct access to Claude with under a week's notice in June 2025, when OpenAI was buying it; one of its co-founders said it would be odd to sell Claude to OpenAI. OpenAI said on August 29, 2026 that it would stop supplying Cursor on November 12 under a change-of-control clause after SpaceX's purchase and wouldn't give Cursor its next model, according to press reports. Anthropic stopped Claude subscriptions from covering outside agent tools in April 2026, as reported.
• Labs also need the apps. Anthropic reportedly pledged to keep adding compute for Claude in Cursor after OpenAI left, and a leaked draft of its IPO filing reportedly says two unnamed customers made up about a quarter of its 2025 revenue.
• App companies escape the labs in three ways. They build or post-train models: Cursor's Composer, Cognition's SWE-2 (post-trained from Moonshot's open-weight Kimi K3, by Cognition's account) and Microsoft's MAI-Code-1-Flash. They route routine work to cheap open-weight models such as Kimi, GLM and Devstral. Or they sell. In July 2025, within days of each other, Windsurf's leaders went to Google and the rest of Windsurf to Cognition; in 2026 SpaceX bought Cursor, in a deal reported at $60 billion.
• The repo host holds the only hard gate. GitHub uses it to host rival agents through Agent HQ, meter them by credits and keep the pull request, while shipping its own model to cut cost. Cursor's answer is Origin.
• Clouds turn procurement into leverage: Kiro and Claude bill through AWS's marketplace and draw down commitments a company has already signed. Platforms are also the labs' biggest investors and customers.
• Compute owners joined in 2026. SpaceX owns Cursor and reportedly rents its Colossus 1 data center to Anthropic, and Nvidia reportedly agreed a $6 billion licence with Poolside, a coding-model company.
• Platform teams inside the customer own the setup scripts, CI and dev environments that decide whether background agents work. I think they're the buyer that matters most, which may be why labs are buying environment companies such as Ona.
• Customers have little power on price and some on switching (their own keys, several agents in Agent HQ). Their best levers are model routing and caps.

How the rules work

Copyright. Prompts alone don't make you an author, the Copyright Office said in January 2025, though human selection, arrangement or modification can, and the Supreme Court declined Thaler's case for an AI author on March 2, 2026. On September 16, 2026 the Ninth Circuit, in Doe v. GitHub, upheld the dismissal of the claim that Copilot and Codex break the DMCA by producing code without the original's copyright notices: generating new code isn't "removing" anything. The court didn't decide whether training on the code was lawful, and breach-of-licence claims are still pending. I found no lawsuit against a company for using AI-suggested code, so for buyers ownership looks settled enough. Copyright protection for a mostly AI-written codebase is thinner, and most companies rely on trade secrets and contracts anyway.

Data and indemnities. The plan tier decides most of it.

Liability caps are in Liability allocation.

Regulated change control. PCI DSS requires bespoke code to be reviewed before release, and if the review is manual the reviewer can't be the originating author. The PCI Security Standards Council's AI principles (September 11, 2025) add that an AI system gets its own trackable, revocable credentials, can't take responsibility or give management approvals, is logged with a named human responsible, and should be treated as a potential malicious insider; it announced further AI guidance on October 7, 2026. EU financial firms under DORA must separate whoever requests and builds a change from whoever approves it. SOC 2 had no AI-specific criteria as of August 2026. US bank supervisors' April 2026 model-risk guidance leaves generative and agentic AI out, so banks govern them under their existing risk practices. A person who prompts an agent and approves its change fails segregation of duties in all of these.

EU product rules. The AI Act puts few duties on coding tools themselves; general-purpose model duties sit with the labs. The Cyber Resilience Act has required reporting of actively exploited vulnerabilities and severe incidents since September 11, 2026 (24 hours, 72 hours, 14 days), and its main obligations start on December 11, 2027. From December 9, 2026 the revised Product Liability Directive treats software as a product with liability without fault. Both fall on the seller, whoever wrote the code.

What mistakes cost

Let's say an engineer at the invoicing company lets an agent run commands against the cloud provider's API with their own token, and the agent deletes a production volume. If the company spends $150-250 per developer a month with the agent's vendor, its enterprise cap is 12 months of fees, $360,000-600,000. But lost data, lost revenue and consequential damages are excluded, and the terms put the risk of automatically run commands on the user, so the expected recovery is close to zero. An IP claim on an unmodified suggestion is the one case where the vendor pays.

Most of these are old control failures: production credentials in reach, no separation of environments, backups on the same volume, no confirmation before a destructive call. The actor is new. What's new in the attacks is that issue text, comments and rules files are now instructions an agent may follow. The tools have had their own holes too, such as CamoLeak, a hidden-comment attack on Copilot Chat that GitHub fixed in August 2025.

Supply-chain risk has an AI flavour. In a USENIX 2025 study, 19.7% of code samples referred to at least one package that doesn't exist, and 43% of those made-up names came back every time the prompt was repeated, so attackers can register them in advance. GitGuardian, which sells secret scanning, found that commits co-authored by one coding agent leaked secrets at 3.2%, against 1.5% overall. And in SusVibes, an academic benchmark presented at ICML 2026, the best agent setup solved 57% of 186 real-repo tasks correctly but only 11.8% securely.

Do coding agents make teams faster: what's real

My view as of October 2026: agents make writing code faster, and whether a team ships faster depends on its tests, its review capacity and the size of its changes. The best causal evidence is from the autocomplete era; for 2026 agents inside companies there isn't any, and the numbers that circulate are mostly self-reported. What the 2026 research does show is where agent work fails, and it fails at review and CI.

What the trials say

• Autocomplete era: the largest field trial, across 4,867 developers at Microsoft, Accenture and a Fortune 100 company from 2022 to 2024, found about 26% more completed tasks with Copilot, with build success about unchanged.
• Experienced developers, early 2025: METR's July 2025 trial with 16 maintainers of large open-source projects found they were 19% slower with AI tools, while they felt 20% faster.
• 2026: METR changed its design in February 2026 after its follow-up became unreliable (developers wouldn't do tasks without AI, so the sample skewed). Its 2026 outputs are a self-report survey, where 349 technical workers put the value at a median 1.4-2 times, and a methods paper. DORA's 2025 report found AI linked to both higher throughput and higher instability and called AI "an amplifier" of the existing system; it had no 2026 AI report by October 9.

The longer version of this evidence, and how to roll the tools out, is in Becoming AI native.

What agent pull requests show

The best 2026 data comes from public GitHub, so it's open-source-heavy, and I'd expect company repos to differ.

• Merge rates: about 65% for agent pull requests against about 85% for human ones, adjusted for repo and task, flat across four quarters. The adjustment matters: raw and adjusted rates for the same agent differ by 15 points or more, so I wouldn't read the per-agent figures as a league table.
• How they fail: reviewer abandonment is the most common pattern, and failed agent pull requests are bigger, touch more files and fail CI more often. Another study found that more review discussion raises the odds of merging for human pull requests and lowers them for agent ones. A May 2026 preprint found only about a third of rejections reflected a clear agent mistake; another third came from the project's own workflow rules, and the rest had no visible reason.
• What lands: CI and build changes, workflows, dependency bumps, docs and typos merge 80% of the time or more. New functions and code that calls language models merge least.
• After the merge: merged agent pull requests look no more defect-prone than merged human ones, which suggests review is doing its job as a filter. That compares only what survived review.
• Who carries it: in a 2025 study of open-source projects after Copilot, core developers reviewed 6.5% more code and wrote 19% less of their own.

What companies report

The strongest company results come from the strongest pipelines. Stripe's in-house agents open more than 1,300 pull requests a week, every one human-reviewed, and only after CI passes. Spotify reports more than 1,500 merged agent pull requests and 60-90% time saved on migrations, on top of a system that already automated about half its pull requests before AI. Google's 2025 paper on migrations found 80% of the code in landed changes written by AI and about half the total time saved, review included, with review and rollout as the limit.

The labs' claims about their own code are bigger and say less. Google says 75% of its new code is AI-generated, Anthropic says more than 80% of the code merged in May 2026 was written by Claude, OpenAI says nearly all its employees use Codex, and Microsoft said in 2025 that AI wrote 20-30% of the code in its repos. None of them publishes a change failure rate or incident count next to the share. Amazon reportedly went the other way in March 2026, adding a human gate with senior sign-off, and critics quoted in the press said that gives back much of the speed.

What benchmarks can and can't tell you

Model capability is still climbing fast. On SWE-bench Pro's public set, the top score reportedly rose from 23% to 80% in eight months, and METR estimates the length of task an agent can finish doubles about every seven months. The benchmarks themselves are breaking down:

• SWE-bench Verified: OpenAI stopped reporting it in February 2026 after an audit of 138 of the hardest tasks found 59% had flawed tests or specs, and some models reproduced the original fixes from memory.
• SWE-bench Pro: OpenAI withdrew its recommendation in July 2026 after finding about 30% of the 731 public tasks broken, and Artificial Analysis had already dropped it in June.
• What they score: a model and its harness together, graded on whether hidden tests pass, which says nothing about cost, safety or whether a maintainer would merge the change.

So I'd build a replay set from our own closed issues, with tests the agent can't edit, and measure merges after human review and reverts within two to four weeks. Throughput only counts if stability holds, and usage figures (seats, tokens, share of code written by AI) become theatre once they turn into targets.

Questions to ask a vendor

1. Of the agent pull requests your customers opened last quarter, what share merged after human review, and what share was reverted within a month?
2. What can your agent reach while it works: network, secrets, which repos and branches, and production?
3. Where do the session logs live, for how long, and can we export them?
4. Which of your agent's actions can be undone, and how?
5. What do you charge when a task fails or loops, and what caps can an admin set?

What usually goes wrong

Words that mean something else here

What surprised me

Placeholders in your voice, drafted from the research and the earlier guides. Rewrite each with your own moment.

The seat. In spend management a card limit capped what one employee could spend. Here a $19 seat buys a couple of hours of agent time, and everything after that is metered at the lab's price.

The approval. In procure to pay the approver checks an invoice against an order. Here the pull request approval is the only hard gate, and many agent pull requests that fail are never really read.

The supplier. In telco, carriers sold capacity to resellers they competed with. Here a lab can withdraw its models from a rival's coding editor with about 75 days' notice.

Undo. In observability, rolling back a deploy felt like the safety net. An agent's checkpoint undoes file edits, and a deleted database stays deleted.

The issue tracker. In cloud security I thought of permissions as the attack surface. Here a GitHub issue title was enough to get a malicious release published.

Sources

Undated entries were read on October 9, 2026; "search result" means seen only as a search snippet or summary. Company figures are self-reported unless they come from a filing or a regulator, and vendor research measures the vendor's own products or customers.

Agent vendors and labs: documentation, pricing and terms

• GitHub: Copilot cloud agent, cloud agent risks and mitigations, Copilot plans, moving to usage-based billing (Apr 2026), pausing Business signups (Apr 2026, search result), reopening signups (Sep 2026, search result), Claude and Codex on Agent HQ (Feb 2026, search result), Secret Protection and Code Security (Mar 2025), code referencing and indemnity, establishing trust in Copilot, npm classic tokens revoked (Dec 2025)
• Anthropic: how Claude Code works, agent teams, costs, data usage, sandboxing (search result), plans, API pricing, commercial terms (effective Jun 2025)
• OpenAI: Codex cloud, Codex pricing, data usage, harness engineering (Feb 2026, search result), Codex Security (Mar 2026)
• Google: Jules, Gemini Code Assist, privacy notice for individuals, Code Assist FAQ
• AWS: Kiro pricing, Kiro specs, Q Developer IDE end of support (search result), Q Developer service improvement, AWS Transform (search result)
• Cursor: pricing, pricing documentation, cloud agents, Bugbot, semantic search blog (Nov 2025, search result), terms of service (Sep 2026), MSA (Aug 2026), data use (Sep 2026)
• Cognition: Devin pricing, Devin introduction
• Mistral: pricing (search result); Hyscaler, GLM coding plan (2026, search result)
• App builders and review: Replit pricing, Lovable pricing, CodeRabbit pricing
• Standards and tooling: AGENTS.md, pnpm settings, Linux kernel policy on coding assistants (Dec 2025)
• Production feedback: Sentry, Seer (search result); Dynatrace Hub, Bluebox (Oct 2026)

Company disclosures

• Microsoft: FY26 Q4 earnings (Jul 2026)
• Alphabet: Q2 2026 CEO remarks (Jul 2026)
• Anthropic: Series G and Claude Code run-rate (Feb 2026), higher limits and SpaceX compute (May 2026), acquires Bun (Dec 2025, search result)
• SpaceX: S-1/A (2026, search result)
• GitLab: Q4 FY26 results (Mar 2026, search result)

Market, deals and pricing changes (press)

• Cursor and SpaceX: CNBC, SpaceX to acquire Cursor (Jun 2026, search result); Techzine, SpaceX completes Cursor (Aug 2026); Dealroom, Cursor tops $4B (Jun 2026, search result); TechCrunch, Cursor's $50B talks and margins (Apr 2026); eWeek, Cursor Origin (Aug 2026)
• OpenAI and Cursor: TNW, OpenAI ends Cursor contract (Aug 2026); Crypto Briefing, 5% of traffic (search result); The New Stack, Anthropic sticks with Cursor (headline only)
• Windsurf: TechCrunch, Anthropic cuts Windsurf's access (Jun 2025, search result); Bloomberg Law, OpenAI agrees to buy Windsurf (May 2025, search result); Maginative, Cognition acquires Windsurf (Jul 2025, search result)
• Pricing changes: TechCrunch, Cursor apologizes (Jul 2025, search result); Simon Willison, Anthropic's weekly limits (Jul 2025, search result); VentureBeat, Anthropic and third-party harnesses (2026, search result); Visual Studio Magazine, Copilot billing shock (Jun 2026, search result); Pragmatic Engineer, token spend breaks budgets (Apr 2026)
• Margins: TechCrunch, high costs and thin margins (Aug 2025); The Information, coding margins (Aug 2025, search result)
• Labs: Constellation Research, Codex at 5 million weekly users (Jun 2026, search result); TNW, OpenAI acquires Ona (Jun 2026, search result); Fortune, Anthropic's draft S-1 (Sep 2026); TNW, Nvidia and Poolside (Aug 2026, search result)
• App layer: Value Add VC, Cognition at $48B (Sep 2026, search result); Orca Router, SWE-2 (Sep 2026, search result); Dealroom, Lovable at $13.3B (Aug 2026, search result); Trending Topics, Replit at $9B (Mar 2026, search result); Sacra, Bolt (search result)
• Review and security: TechCrunch, CodeRabbit raises $60M (Sep 2025); Hyrax, CodeRabbit's Series C (Aug 2026, search result); SiliconANGLE, Cursor acquires Graphite (Dec 2025); BankInfoSecurity, Snyk cuts 90 jobs (Jun 2026); SiliconANGLE, Semgrep Series D (Feb 2025); BankInfoSecurity, Socket raises $60M (May 2026); SiliconANGLE, Aikido Series B (Jan 2026); Hackmag, Claude Code Security (Feb 2026); Datamation, Google CodeMender (Oct 2025); Help Net Security, Mandiant's vulnerability discovery tool (Aug 2026); TechCrunch, Moderne and OpenRewrite (Feb 2025, search result)

Research, benchmarks and outcomes

• METR: early-2025 study of experienced developers (Jul 2025), uplift update (Feb 2026), research index (Oct 2026), time horizons (search result)
• DORA: AI research (Oct 2026), 2025 report (Sep 2025)
• Cui, Demirer and colleagues, field experiments with Copilot (2024); Xu and colleagues, Copilot in open source (Oct 2025)
• Agent pull requests: Mazloomzadeh, Morovati and Khomh, agentic pull requests (Jul 2026); Ehsani and colleagues, failed agentic pull requests (Jan 2026); Yoshioka and colleagues, review activity and merges (2026, search result); rejected agentic pull requests (May 2026, search result); Raida and Hou, early adoption of agentic tools (Jul 2026)
• Context and environment: Gloaguen and colleagues, evaluating AGENTS.md (Feb 2026)
• Benchmarks: The Decoder, OpenAI retires SWE-bench Verified (Feb 2026) and 30% of SWE-bench Pro broken (Jul 2026); ImpossibleBench (Oct 2025)
• Company reports: Stripe, Minions (2026); Spotify, background coding agent (Nov 2025); Google, migrations with LLMs (Jan 2025); Semafor, Google's 75% (Apr 2026); Fortune, Anthropic's 80% (Jun 2026); OpenAI, the shift to agentic AI (Jun 2026); TechRadar, Microsoft's 20-30% (Apr 2025)
• Amazon's sign-off rule: The Decoder, senior engineers as the filter (Mar 2026); CIO, Amazon links outages to AI efforts (Mar 2026)
• Security of generated code: Veracode, 2026 GenAI code security report (Jul 2026, vendor); SusVibes (2026); Cloud Security Alliance, package hallucinations (Apr 2026); GitGuardian, State of Secrets Sprawl 2026 (Mar 2026, vendor)

Law, regulators and standards

• Copyright: US Copyright Office, Part 2 report (Jan 2025); SCOTUSblog, Thaler v. Perlmutter (Mar 2026); Gibson Dunn, Doe v. GitHub (Sep 2026)
• Training defaults: heise, GitHub to train on user data with opt-out (2026)
• Change control: PCI Security Standards Council, AI principles (Sep 2025); PCI DSS 6.2.3.1 (mirror); Advisera, EU DORA RTS Article 17; Cloud Security Alliance, SOC 2 and AI controls (Aug 2026); Cimcon, SR 26-2 and AI agents (2026); Baker McKenzie, MAS agentic AI safeguards (Jul 2026)
• EU product rules: Jones Day, Cyber Resilience Act reporting (Jul 2026)

Incidents, attacks and supply chain

• Deleted data: The Register, Replit's response (Jul 2025); AI Incident Database, Gemini CLI (2025); TechRadar, Antigravity deletes a drive (Dec 2025); GIGAZINE, Claude Code deletes a home folder (Dec 2025); Computing, an AI coding agent goes rogue (Apr 2026); GeekWire, Amazon pushes back on the FT (Feb 2026)
• Attacks on agents: GitGuardian, inside the Nx credential leak (2025, vendor); CSO Online, AI in CI/CD pipelines (Dec 2025); Simon Willison, Clinejection (Mar 2026); The Register, CamoLeak (Oct 2025)
• Poisoned tooling in CI: CSO Online, CERT-EU on Trivy (2026); ITPro, LiteLLM on PyPI (Mar 2026)