The Platform PM

How it works

Field Guide

AI coding agents: writing, reviewing and shipping code

02

How it works

5 min read · Chapter 2 of 10

How an agent's task gets done, step by step

An agent's task, step by step. An agent is only as good as its definition of done: where tests and CI are strong it can loop until they pass, and where they're weak a person has to catch what it got wrong, sometimes only after it reaches production.

Here's the PDF-library upgrade, run by a background agent from a GitHub issue. An agent is only as good as its definition of done. Where tests and CI are strong, it can loop until they pass, and where they're weak a person has to catch what it got wrong, sometimes only after it reaches production.

  1. Task given: an issue, a prompt, a chat thread, a production alert or a schedule starts the work. On GitHub only someone with write access can start Copilot's cloud agent, and comments from anyone else never reach it. What can go wrong: an underspecified issue leaves the agent to guess what done means (Devin's docs ask for explicit completion criteria); an automation with nobody behind it gets its pull request credited to whoever set it up, who then can't approve it.
  2. Planned: with the context gathered (AGENTS.md or CLAUDE.md, a search of the repo, the issue), the agent drafts a plan of files and steps. Some products stop for approval here: Jules shows its plan before changing any code, Kiro writes requirements, design and task files, and Claude Code has a read-only plan mode. What can go wrong: the agent misses the file that matters; a long session compacts its history and drops an early instruction; in Claude Code's experimental agent teams the lead agent approves its teammates' plans, which is a plan gate with no person at it.
  3. Changed: the agent writes code and produces a diff in a sandbox, either an ephemeral container or virtual machine in the cloud or the developer's own machine, after a setup script installs the dependencies. What it can reach differs: Codex keeps the network off by default while the agent works and gives secrets only to the setup step, Copilot's agent works behind a firewall with an allowlist, and Cursor's cloud agents have the network on, with optional domain limits, and get secrets when they start. What can go wrong: the build fails on a private registry, a missing credential or a database the agent can't reach, which is the usual first failure; Copilot's sessions stop at 59 minutes and can't be extended.
  4. Verified: the agent runs the tests, linters and type checks, reads the errors and edits again until the tests and CI are green. That loop is the Retried exception on the diagram, and it's where the work gets good or goes wrong. Copilot also runs code, secret and dependency scans on its agent's changes, and Cursor's Bugbot, an AI reviewer, caps its own automatic fixes at three attempts per pull request. What can go wrong: flaky tests send the loop the wrong way, and an agent may edit or delete tests to get to green. In ImpossibleBench, a research benchmark where the only way to pass is to cheat, GPT-5 exploited the tests 76% of the time on one variant and Claude models tended to edit the tests. The researchers point to protected, read-only tests.
  5. Merged: a person approves and merges. On GitHub the agent can't mark its own pull request ready, approve it or merge it, the person who asked for the change doesn't count as an approver, and CI won't run on the agent's pull request until someone with write access clicks "Approve and run workflows". The normal deploy follows, with feature flags or a canary, and the agent holds no production credentials. What can go wrong: the reviewer abandons the pull request, the most common failure in a 2026 study of 33,000 agent pull requests; a large diff gets waved through; two parallel agents open duplicates.

Reverted is the exit after the merge: the change broke something in production and comes back out. Git makes code easy to revert, but data is another matter. Claude Code's own docs say its checkpoints undo file edits only, and that actions on remote systems (databases, APIs, deployments) can't be checkpointed. Production tools now close the loop: Sentry's Seer hands a root cause to Claude Code, Cursor or Copilot, and Bluebox opens a GitHub issue with the trace and the offending lines for a coding agent, though Dynatrace says Bluebox "never pushes a fix on its own".

A local agent in an editor or terminal takes a shorter path. The developer prompts, the agent edits the working copy and runs commands under a permission mode (ask every time, accept edits, or a bypass mode that asks nothing), and the developer commits. The agent acts with the developer's own credentials and files, the riskiest case: most deleted-data incidents in What mistakes cost happened that way. Anthropic says sandboxing Claude Code at the operating-system level cut its internal permission prompts by 84%, so autonomy there was bought with environment controls.

An app builder skips the pull request. Lovable, Replit or Bolt generates the app, its database and its hosting, and "publish" puts it live. The builder is repo host, CI and production at once, so its failures are data failures: in 2025 a researcher found exposed database tables in 170 of 1,645 apps showcased by Lovable, which said securing them was the customer's job.

Cheat sheet: the product shapes

Autocomplete and next edit

Where it runs
The editor
Whose credentials
The developer's
The gate
The developer, keystroke by keystroke
Good for
Everyday typing; lowest risk

Agent in the editor or terminal

Where it runs
The developer's machine, optionally sandboxed
Whose credentials
The developer's
The gate
The developer's commit, then the pull request
Good for
Multi-file changes a person watches

Background or cloud agent

Where it runs
A vendor's VM or container
Whose credentials
Scoped to one repo and one branch on GitHub; varies elsewhere
The gate
The pull request and its required checks
Good for
Well-defined work with tests: upgrades, migrations, small fixes

AI code review

Where it runs
On each pull request update
Whose credentials
The review app's token
The gate
A comment or a non-blocking check, never an approval
Good for
Catching bugs before a person reads the diff

Test and migration agent

Where it runs
Cloud or CI
Whose credentials
Scoped
The gate
CI and the pull request
Good for
Large mechanical changes (AWS Transform, OpenRewrite recipes)

App builder

Where it runs
The builder's cloud
Whose credentials
The builder's hosting and database
The gate
"Publish"
Good for
Prototypes and internal tools by people who don't code

Jules, Devin and Claude Code on the web didn't state their network and secrets defaults in the pages I read, so I'd ask each vendor before letting its agent near private code.