Operating an Agent Fleet in 2026: The Practical Guide

23 percent of organizations are scaling agentic AI somewhere in their enterprise (McKinsey, November 2025). Bank of America's Erica has crossed 3 billion interactions; Wells Fargo's Fargo has crossed 1 billion. Salesforce Agentforce 360 has 12,000 customers. And Klarna walked back its agent-only deployment after admitting that cost had been a too-predominant evaluation factor. Here is the practical 2026 guide to operating an agent fleet that actually works.

Should we build agents on Anthropic Claude, OpenAI, or a smaller open-source model?

For most enterprise use cases in 2026, Claude Sonnet/Opus or GPT-5/GPT-5 Codex deliver the best capability-per-dollar. Open-source models (Llama, Qwen, DeepSeek) work well for lower-stakes use cases or where data residency requires on-prem inference. The platform decision matters more than the model decision because the platform is what runs the fleet.

McKinsey's State of AI 2025 (November 2025) reported that 23 percent of organizations are scaling an agentic AI system somewhere in their enterprise, with an additional 39 percent experimenting and customer service leading adoption at 15 percent scaling (McKinsey). Bank of America's Erica has surpassed 3 billion client interactions across nearly 50 million users (Bank of America Newsroom, August 2025). Wells Fargo's Fargo crossed 1 billion interactions in less than three years (eMarketer). Salesforce Agentforce 360 reached GA in October 2025 with 12,000 customers; Reddit deflected 46 percent of support cases and cut resolution time by 84 percent (Salesforce press release).

And Klarna, the original agent ROI poster child (2.3 million conversations in month one, doing the work of 700 full-time agents per OpenAI's Klarna case study), publicly walked back its agent-only customer service deployment in 2025. Klarna's CEO acknowledged that cost had been a too-predominant evaluation factor, resulting in lower quality, and recruited human agents back into the rotation (Customer Experience Dive).

The numbers above tell the actual 2026 story. Agent fleets work at scale. Agent-only deployments fail at scale. The practical guide is the difference between running a fleet (which is the win) and running an agent (which is a feature, not a system).

What "operating an agent fleet" actually means

A single agent is a model with tools and a prompt. An agent fleet is a set of agents that share an identity layer, a tool catalog, an observability plane, an evaluation harness, an incident-response playbook, and a budget. The leading enterprise deployments (Erica at BofA, Fargo at Wells Fargo, Agentforce 360 at 12,000 customers) all operate fleets, not agents.

Three concrete signals that you are operating a fleet rather than an agent: (1) you can name the on-call who pages when an agent fails, (2) you can produce a 30-day evaluation report on agent quality and cost per interaction, and (3) you can show the auth scope and tool catalog that any given agent has access to. If any of those three is "no," you have a deployed agent and not a fleet, and the Klarna walkback is the named risk.

The 2026 reference architecture

The 2026 enterprise agent fleet has six architectural components.

Component 1: Agent runtime. OpenAI Agents SDK, Anthropic Claude with tool use, Microsoft Copilot Studio (the GA of autonomous agents was announced November 2025 at Ignite alongside Agent 365 as the centralized control plane, per Microsoft Copilot Blog), Salesforce Agentforce 360, Databricks Agent Bricks, Google Agentspace. Most enterprises in 2026 run two or three runtimes for different agent classes.

Component 2: AI gateway / control plane. Kong launched Kong Agent Gateway within AI Gateway 3.14 in April 2026, providing centralized authentication, authorization, and rate limiting on agent-to-agent traffic plus LLM and MCP traffic (Kong press release). Cloudflare AI Gateway, Microsoft Agent 365, and Databricks Agent Bricks AI Gateway play the same role inside their respective ecosystems. The AI gateway is the single most important fleet-level component because it is the only place auth, audit, and rate limits can be centrally enforced.

Component 3: Tool catalog and identity layer. Every agent gets an identity (typically an OAuth client or a service principal) and a scoped set of tools (read database X, call API Y, write to file system Z). The Model Context Protocol (MCP) is now the de-facto standard for tool surface description across runtimes. The identity layer is what prevents agent privilege creep.

Component 4: Observability. Langfuse has emerged as the dominant open-source agent observability platform with over 6 million SDK installs per month, with formerly commercial modules (LLM-as-a-judge evaluations, annotation queues, prompt experiments, Playground) open-sourced under MIT in June 2025 (DigitalApplied analysis). LangSmith (LangChain), Arize, and Datadog also play in this space. Pick one and instrument every agent through it.

Component 5: Evaluation harness. Public benchmarks like SWE-bench, GAIA, OSWorld, and WebArena are useful capability proxies but they are not your evaluation. The April 12, 2026 UC Berkeley research showed all eight major agent benchmarks could be reward-hacked to roughly 100 percent (UC Berkeley RDI). Production-grade agent fleets run their own evaluations on their own data with their own success criteria.

Component 6: Incident response playbook. When the agent does something wrong (false answer, wrong action, stuck loop), what is the documented response? Who pages? Who has the kill switch? What is the rollback procedure? This is the component most enterprises in 2025-2026 are still drafting. The Air Canada chatbot precedent from February 2024 made this non-optional.

Two enterprise agent stories worth comparing

The Klarna 2024 number is famous: AI assistant doing the work of 700 full-time agents in its first month live globally, 2.3 million conversations, customers resolving errands in under 2 minutes vs 11 minutes previously (OpenAI customer story). The Klarna 2025 walkback is just as important: cost was a too-predominant evaluation factor, quality dropped, human agents are being recruited back into the rotation (CX Dive). The lesson: agent-only deployment with a single optimization target (cost) produces a quality cliff. Agent fleets that are scoped to augment humans (with humans on the higher-complexity tail) produce a quality plateau.

The Bank of America Erica number is the counter-example. Three billion interactions, nearly 50 million users, more than 58 million interactions per month, more than 98 percent of users finding the information they need (Bank of America Newsroom). Erica is scoped narrowly (banking-specific tasks), evaluated continuously, and operated as a fleet inside a regulated environment. The bank did not try to replace customer service with one agent; it added a fleet that absorbed the volume customer service could not.

The actionable read: scope your agent fleet around tasks where success is measurable, where the failure mode is recoverable, and where human escalation is the default for the long tail.

The talent and partnership signal

The 2026 enterprise agent platform vendors are all signaling that the deployment is not solo. Anthropic's Claude Code crossed a $2.5B run-rate by February 2026 (from full launch in May 2025), and Anthropic launched Claude Cowork to expand beyond developers into the broader enterprise (VentureBeat). Accenture and Anthropic announced the Accenture Anthropic Business Group, training approximately 30,000 professionals to drive Claude adoption at scale, with Deloitte's rollout to approximately 470,000 employees as the largest verifiable enterprise agent deployment to date (Accenture Newsroom).

What that signals for the operating model: the AI labs themselves do not believe the deployment is a vendor-only motion. The pattern across the largest 2025-2026 deployments is platform vendor + systems integrator + internal AI office. For enterprises that are not Deloitte-scale, Thinklytics's role is the AI office function that scopes the fleet, instruments the observability, and runs the evaluation harness.

The 90-day plan to stand up a fleet

Days 1 to 30: pick one runtime (the one that aligns with your existing platform: Microsoft Copilot Studio for Microsoft shops, Salesforce Agentforce for Salesforce shops, Anthropic Claude or OpenAI Agents SDK for build-your-own teams), pick one AI gateway, and pick one observability platform. Inventory all existing agents (sanctioned and shadow).

Days 31 to 60: deploy two agents in production through the gateway with full observability. One should be a high-volume / low-stakes use case (typical: customer service deflection, internal IT helpdesk). The other should be a low-volume / high-leverage use case (typical: financial reconciliation, compliance triage). Stand up the evaluation harness with a labeled test set drawn from the past 30 days of real interactions.

Days 61 to 90: run the incident-response playbook (deliberately page the on-call once to test). Tune the eval harness against the production traces. Publish the first 30-day fleet report (cost per interaction, quality score, escalation rate, top failure modes).

By day 91 you have a functioning fleet, two production agents, and the operating model needed to add a third agent in 30 days rather than 90. That second-agent timeline is the actual ROI of operating a fleet.

Frequently asked questions

Should we build agents on Anthropic Claude, OpenAI, or a smaller open-source model?

For most enterprise use cases in 2026, Claude Sonnet/Opus or GPT-5/GPT-5 Codex deliver the best capability-per-dollar. Open-source models (Llama, Qwen, DeepSeek) work well for lower-stakes use cases or where data residency requires on-prem inference. The platform decision matters more than the model decision because the platform is what runs the fleet.

Do we need an AI gateway?

Yes if you have more than two agents in production. The Kong, Cloudflare, Microsoft Agent 365, and Databricks Agent Bricks gateways all serve the same architectural role: a single point for auth, audit, rate limits, and routing. Without a gateway, every agent is its own attack surface.

How do we evaluate agents reliably?

Build your own evaluation set from real production traces. Public benchmarks (SWE-bench, GAIA, OSWorld, WebArena) are useful capability proxies but the April 2026 UC Berkeley reward-hacking demonstration showed they are not production-quality measures. Use Langfuse, LangSmith, Arize, or your runtime's native eval harness.

What does Salesforce Agentforce or Microsoft Agent 365 cost?

Both are usage-based with platform fees. Agentforce charges per-conversation; Microsoft Agent 365 is in early GA with the centralized control-plane positioning. Plan for $5-25 per agent per month at the platform layer plus inference costs.

What's the right ratio of human:agent for customer service?

The Klarna walkback suggests the all-agent ratio fails at quality scale. The BofA Erica pattern (deflect predictable volume to the agent, escalate the long tail to humans) is the durable model. Most fleets converge to 60-80 percent agent-handled / 20-40 percent human-handled by ticket count, with human-handled biased toward higher-revenue or higher-stakes interactions.

What about responsible AI / safety?

Apply the 2026 AI Governance Operating Model to the fleet. NIST AI RMF + ISO 42001 + OWASP LLM Top 10 + EU AI Act overlay. The agent runtime and AI gateway choices need to support the documentation and audit-trail requirements of those frameworks.

If you want the longer version of this analysis, including the agent fleet inventory template, the AI gateway selection matrix, and the 30-day incident-response runbook, our AI Workflow Automation Consulting, AI Readiness, and Data Governance Consulting practices ship the operating model end-to-end. Background on agents themselves is in our primer on AI agents for ops leaders. The UT System financial-aid disbursement automation is the deepest published Thinklytics case study on production AI agents (180,000 disbursements per year, 14 days to 36 hours, $5.9M additional tuition).

Which observability tools are mature enough for production agent fleets?

Three categories. Action logging (LangSmith, Arize, Helicone) for the per-action trail. Model evaluation (Galileo, Patronus AI, Weights & Biases Weave) for output quality regression. Infrastructure (Datadog, New Relic, Grafana) for the wider system. Most production fleets need at least one from each category.

How does Thinklytics scope fleet operations work?

Typical engagement: 6 to 10 weeks to instrument the shared services, write the operations playbook, and train the on-call team. Read more at AI agent consulting.