AI agents: orchestration platforms
The layer that runs AI agents in production: orchestration and control, context and memory, secure tools, evals and governance. What it costs, what goes wrong and who is accountable when an agent acts.
Last updated October 2026
01
Overview
7 min read · Chapter 1 of 10
A customer asks your support agent for a refund. The agent is software that calls a model in a loop: the model reads the request and some context, then answers or asks for a tool (look up the customer, search the policy, issue the refund). The software runs the tool, adds the result and asks again, until the task is done or something stops it. An agent orchestration platform runs that loop for a company. It retrieves context (data, memory) and asks a model from any lab, passing the context in; the model's tool call goes to tools (APIs, MCP servers), which act on the systems where actions land, from a CRM to a payments ledger. I'd watch the tools most closely, because they're where the agent acts on real systems and where an attacker who steers the model gets in.
Adoption is real and hard to measure. KPMG's 2026 survey found about three in five large US companies building or deploying agents; McKinsey found under a quarter scaling one anywhere. Gartner expects more than 40% of agentic projects to be cancelled by the end of 2027, without saying how it got there. The clearest money is at systems of record: Agentforce passed $1.5B in annual recurring revenue in August 2026, after Salesforce widened what the figure counts.
I organize the guide around four pillars, the four jobs the platform does on every run:
| Pillar | What it covers | Where to find it |
|---|---|---|
| 1. Agent orchestration and control | The loop, workflows vs agents, multi-agent, state, approvals, caps, retries | Run steps 1, 3, 5, 6; primitives 2 and 9; How to choose an approach |
| 2. Context, data and memory | Context windows, retrieval with permissions, memory, caching, retention | Run steps 2 and 7; primitives 1, 3 and 8; How the money moves |
| 3. Secure execution and tooling | Tool calling, MCP and A2A, sandboxes, agent identity, prompt injection | Run step 4; primitives 1, 4, 6 and 7; Prompt injection |
| 4. Agentic evals and governance | Evals, tracing, model retirements, guardrails, regulation, liability | Run step 8; primitives 3, 5, 8 and 10; How the rules work |
Below is what I've learned so far. I'm still testing all of it with experts.
- The platform sells control more than intelligence. In a 2025 study, about two-thirds of production agents ran fewer than 10 steps before a human stepped in, and 16 of 20 case studies used fixed workflows rather than open-ended planning. Anthropic, OpenAI, Microsoft and Google Cloud all say to start with the simplest design that works. That holds for back-office and customer-facing agents, but coding and research agents run much longer. Coding agents take a median 41 to 58 turns on SWE-bench Verified, a coding benchmark; METR finds the length of task frontier models can finish doubles about every seven months; and all three API labs sell background execution for long runs.
- Quality is context times model. I mean that as a multiplication; "context beats model" goes too far. With the same model, one terminal-task benchmark moved 20 points or more across harnesses (the code around the model), about a model generation's worth; with the harness fixed, swapping models still moves results 2 to 3 times. The risk sits on the context side. Search indexes copy permissions when they sync, so a user whose access was revoked can keep seeing files until the next sync. If an HR manager loses access to the salary folder at 10 a.m. and the index syncs overnight, the HR agent can still quote salaries to her that afternoon. And in one study, poisoning under 0.1% of an agent's memory steered it more than 80% of the time.
- Every tool is a door, and this is the one I'm most sure of. Nobody claims prompt injection (text that hijacks the agent's instructions) is solved: OWASP, the UK's NCSC, Meta, OpenAI and Anthropic all say some version of "never fully", and Google DeepMind relies on continuous adaptive testing. Adaptive attacks pushed 12 published defenses above 90% success. So security comes from identity, least privilege, egress control (limiting where data can go) and policy outside the model, and the clouds' responsibility splits leave authorizing actions to the customer. Two caveats: model defenses do lower attack rates, and least privilege can't stop a fooled agent misusing a permission it already holds.
- Evals are the unit of change, more than the unit of purchase, though I'm only half sure of this one. Models retire on published clocks, from six months' notice for OpenAI's GA models down to two weeks for Google's previews. Every forced swap needs a regression suite (a fixed set of test tasks you re-run), and public benchmarks can't stand in: OpenAI stopped reporting SWE-bench Verified in February 2026 over flawed tests, and weak graders let agentic benchmarks overstate results by up to about 40%. As something to buy, evals look weaker: clouds meter evaluators at fractions of a cent, OpenAI closes its hosted Evals on November 30, 2026, and in a framework vendor's survey most teams had tracing but only about half ran offline evals. Buyers seem to follow where their data already lives.
So I think of the product as a control loop around a model someone else owns. It decides what the model sees, what it may touch and what changed since the last eval, and it's paid per token, per action or per outcome. Two more ideas sit further down: the retirement clock as a hidden operating cost (Effective dating), and accountability set by courts, contracts and insurers more than by AI statutes (How the rules work).
What makes this hard:
- Same input, different output: a model that solved about 61% of retail support tasks on one try got all eight tries right under 25% of the time, so test results have to be read as statistics.
- Data that gives instructions: a web page, an email, a tool description or a stored memory can tell the agent what to do, and there's no equivalent of a parameterized query (the database trick that keeps data and commands apart).
- A dependency on someone else's calendar: models retire on 45 days' to 6 months' notice, model aliases (names that point at whatever model is current) move without a release on your side, and one lab's newer tokenizer turns the same text into about 30% more tokens.
How card networks treat purchases made by AI agents is in B2B payments: spend management.
The main players
The diagram shows roles; the mental map needs names. These are the companies I'd want to recognize in week one, in no particular order.
Frontier labs
- What they do
- Train the models and sell them by the token, plus their own agent tools
- Main players
- Anthropic, OpenAI, Google DeepMind, Meta, Mistral, xAI
- What they control
- Prices, caching rules, rate limits, retirement dates
Clouds
- What they do
- Host many labs' models and sell metered runtime, memory, identity and policy
- Main players
- AWS, Microsoft, Google Cloud
- What they control
- The company's identity, network and spending commitments
Frameworks and workflow engines
- What they do
- Run the loop in your own code and keep long runs alive through crashes and pauses
- Main players
- LangChain, Temporal, n8n
- What they control
- Developer habits; how state, retries and replays work
Data and search
- What they do
- Hold what agents retrieve, with permissions, and more and more of their memory
- Main players
- Databricks, Snowflake, Elastic, MongoDB
- What they control
- Where governed data lives; how fresh the permissions are
Observability and evals
- What they do
- Trace each run and score it before and after a change
- Main players
- Datadog, Braintrust, Arize, ClickHouse (Langfuse)
- What they control
- Traces and eval datasets, the hardest things to move
Identity and security
- What they do
- Give agents identities and scoped credentials, screen for injection, sandbox code
- Main players
- Okta, Palo Alto Networks, CrowdStrike, Cloudflare, E2B
- What they control
- Which tools an agent may call, and with what credentials
Systems of record
- What they do
- Business apps that sell their own agents on their own data
- Main players
- Salesforce, ServiceNow, SAP, Workday
- What they control
- The data, the workflow and the unit you pay for
Fast-growing agent builders
- What they do
- Startups selling a finished agent for one job, on several labs' models
- Main players
- Sierra, Decagon, Glean
- What they control
- The customer, and what counts as an outcome
How they make money:
Layer: Frontier labs
Charges for: per token
Ballpark (list): From about $0.10 to $50 per million tokens
Layer: Clouds
Charges for: resold tokens, metered calls
Ballpark (list): Agent services at fractions of a cent per call
Layer: Frameworks and workflow engines
Charges for: free code, paid hosting
Ballpark (list): Hosted runs and tracing on top of open source
Layer: Data and search
Charges for: compute and storage
Ballpark (list): Part of the data bill a company already pays
Layer: Observability and evals
Charges for: per trace or seat
Ballpark (list): A free tier, then about $39 per seat a month (LangSmith)
Layer: Identity and security
Charges for: a module
Ballpark (list): Mostly sold inside existing security deals
Layer: Systems of record
Charges for: per action or conversation
Ballpark (list): About 10 cents an action or $2 a conversation (Salesforce)
Layer: Agent builders
Charges for: per resolution
Ballpark (list): About $1 a resolution where published (Intercom)
Tokens grow with every run, so most of the money sits with the labs and the clouds that resell them: OpenAI and Anthropic reportedly each run at $65-70 billion a year, though those are press figures the two count differently, and Google, Meta, Mistral and xAI don't report a comparable one. The control layer around the model is being bought or bundled: ClickHouse bought Langfuse, and Palo Alto Networks and CrowdStrike bought AI-security startups. The independents growing fastest are the plumbing the labs run on, like Temporal, whose run-rate tripled to over $250 million, and finished agents sold per resolution, like Sierra and Decagon.
As of October 2026. Players change through acquisitions and new models, so check anything here before you rely on it.
Take an insurer running a claims agent built with LangGraph on AWS. A claim email starts a run. The agent pulls the policy from Databricks, filtered by the adjuster's permissions, and asks a Claude or GPT model through Bedrock. The model asks to open the claim in ServiceNow; Okta gives the agent a token for that one call, and each step lands as a trace in Datadog, where the team re-runs old claims before any model swap. One claim touches six or seven companies, and the insurer may have no contract with the lab whose model made the call.