The Platform PM

How to choose

Field Guide

AI coding agents: writing, reviewing and shipping code

03

How to choose

2 min read · Chapter 3 of 10

Which coding agent setup? Five questions

  1. What counts as done? Agents land work that has an objective check. Merge rates in the 2026 study were highest, at 80% or more, for CI and build changes, GitHub Actions workflows, dependency bumps, docs and typo fixes, and lowest for new functions and code that calls language models. If a task has no test, a person is the test.
  2. Can it build and test? If the build needs a private registry, a database and three secrets, write the setup script before buying more seats.
  3. What can it reach? A local agent acts as the developer. Cloud agents differ on network and secrets defaults. Production credentials should be out of reach either way, and anything in an issue or a pull request comment is untrusted input.
  4. Who reviews? Every pull request an agent opens lands in someone's queue. On card-payment systems covered by PCI DSS the reviewer can't be the author, and whether the person who prompted the agent counts as the author hasn't been tested; I'd ask the assessor before it comes up in an audit.
  5. Who pays for the tail? On seats the vendor eats it until it re-prices, and on usage you do. Set budgets per team and user, and decide about overage up front.

My defaults: for the invoicing company I'd give every engineer an editor assistant on a business plan (no training on our code, indemnity on, public-code filter on) and give background agents to a few teams, starting with dependency upgrades, flaky tests and migrations. Every agent pull request goes through the same branch protection as a person's, with a size limit and an approver who didn't prompt the change. Agents get their own identity, no production credentials, the network off or allowlisted, and a setup script that builds the repo from scratch. I'd budget on usage with team caps, track cost per merged change rather than per seat, and before the rollout capture a few weeks of baseline: pull request cycle time split into waiting and reviewing, pull request size, change failure rate and reverts.