How teams move from "one API key per runtime" to "one gateway, any runtime"

The Fragmentation Problem (July 2026)

You've got a squad of coding agents now. Some teams use Claude Managed Agents. Others prefer Cursor. Finance is piloting Bedrock AgentCore. Engineering wants to stay self-hosted.

This sounds flexible. It's actually a disaster.

Here's what happens:

  • Agent A (Claude Managed Agents team) stores API credentials in Anthropic console. No one else on the company can invoke it.
  • Agent B (Cursor team) lives in a separate dashboard. Finance can't call it without direct Cursor access.
  • Agent C (Bedrock, self-hosted) requires AWS IAM roles. Now you need to explain why the Marketing team can't use Bedrock.
  • Agent D (N8N) sits in a workflow tool. No one knows it exists. No one can audit what it does.

The models are powerful. The infrastructure is a mess. Each runtime has its own authentication, its own API surface, its own session model. The agents themselves are stateless—the infrastructure that manages them is fragmented.

This is the multi-runtime agent problem of July 2026. Not a model problem. An infrastructure problem.

The Control Plane Solution

The answer is not "pick one runtime." The answer is: separate the agent control plane from the agent runtime.

The pattern:

  • Runtimes (Claude Managed Agents, Cursor, Bedrock, N8N, custom) are where agents execute.
  • Control Plane (LiteLLM Agent Platform, Microsoft Foundry, or equivalent) is where agents are governed.

The control plane sits between teams and runtimes. It's a unified gateway that:

  1. Normalizes invocation across runtimes — One API to call any agent, regardless of where it runs.
  2. Centralizes credentials — Teams don't need console access to each provider.
  3. Enforces policy uniformly — budgets, rate limits, tool authorization, audit trails — all in one place.
  4. Manages session continuity — If an agent crashes mid-execution, the session survives because Postgres is the backing store, not the runtime's ephemeral memory.
  5. Makes agents shareable — An agent built by one team on one runtime can be discovered and invoked by any other team through the control plane.

How This Works in Practice

Let's build it:

Step 1: Register Your Runtimes

Your control plane starts by connecting to each runtime. One YAML config:

runtimes:
  claude_managed:
    type: anthropic
    api_key: env/ANTHROPIC_API_KEY
  cursor:
    type: cursor
    api_key: env/CURSOR_API_KEY
  bedrock:
    type: aws_bedrock
    region: us-west-2
    credentials: env/AWS_ROLE
  self_hosted:
    type: custom
    base_url: https://internal-agent-api.company.com

Enter fullscreen mode Exit fullscreen mode

The control plane normalizes these. A single POST request to the control plane invokes any runtime:

curl -X POST https://control-plane.company.com/agents/pr-babysitter/invoke \
  -H "Authorization: Bearer $CONTROL_PLANE_KEY" \
  -d '{
    "runtime_hint": "any",  # Fallback: try best available
    "prompt": "Review this PR for security issues",
    "tools": ["git", "github_api"]
  }'

Enter fullscreen mode Exit fullscreen mode

The control plane picks a runtime (round-robin, cost-aware, or explicit), translates the request, executes, and returns a unified response format.

Step 2: Decouple Credentials from Teams

No developer touches provider consoles directly. Instead:

  • Secrets stored in one vault (HashiCorp Vault, AWS Secrets Manager, etc.)
  • Control plane injects credentials at invocation time
  • Every call is logged with who called what, when, and why 
credentials:
  anthropic_shared:
    provider: anthropic
    api_key: vault://secrets/anthropic-prod
    rotated: 2026-07-20
    last_used: 2026-07-24
  cursor_shared:
    provider: cursor
    api_key: vault://secrets/cursor-prod
    last_used: 2026-07-24

Enter fullscreen mode Exit fullscreen mode

Teams request agent invocation. The control plane handles credential binding. No sprawl. Auditable.

Step 3: Enforce Unified Policy

Tools, budgets, rate limits, and approvals live in the control plane—not scattered across four provider consoles.

agents:
  pr_babysitter:
    owner: engineering
    runtime: any  # Will use cheapest/fastest available
    tools:
      - github: read
      - linear: read
    budgets:
      daily_usd: 50
      monthly_usd: 1000
    rate_limits:
      calls_per_minute: 10
      concurrent_sessions: 5
    approval_rules:
      - if_tool: github_push
        requires: human_approval
      - if_cost_exceeds_usd: 100
        requires: human_approval

Enter fullscreen mode Exit fullscreen mode

Policy changes propagate instantly. No redeploy. No framework fighting.

Step 4: Manage Session Continuity

Sessions live in the control plane (Postgres-backed), not in the runtime's memory. An agent running on Cursor can crash, and the control plane resumes it on Bedrock without losing state:

session:
  id: sess-pr-babysitter-82946
  agent: pr_babysitter
  runtime: cursor  # Started here
  created_at: 2026-07-24T14:30:00Z
  steps:
    - turn: 1
      tool_call: github_list_prs
      result: ["PR #1024", "PR #1025"]
      cost_usd: 0.12
    - turn: 2
      tool_call: github_get_diff PR#1024
      result: "[diff content]"
      cost_usd: 0.08
  status: executing
  last_activity: 2026-07-24T14:32:15Z

Enter fullscreen mode Exit fullscreen mode

If the Cursor runtime crashes at turn 3, the control plane can:

  • Resume on Bedrock (same cost model, similar capability)
  • Inject the session history automatically
  • Continue from turn 3 without re-running turns 1-2

Step 5: Discovery and Sharing

Control plane maintains a registry of all agents. Teams discover them:

# List all available agents
curl https://control-plane.company.com/agents \
  -H "Authorization: Bearer $KEY"

# Response:
{
  "agents": [
    {
      "id": "pr_babysitter",
      "owner": "engineering",
      "runtime": "any",
      "description": "Reviews PRs for security issues",
      "calls_this_month": 4829,
      "avg_cost_per_call": 0.23
    },
    {
      "id": "compliance_check",
      "owner": "legal",
      "runtime": "any",
      "description": "Audits contracts for risk clauses",
      "calls_this_month": 312,
      "avg_cost_per_call": 1.42
    }
  ]
}

Enter fullscreen mode Exit fullscreen mode

Finance calls the PR babysitter. Marketing uses the compliance checker. All through the same control plane.

Why This Matters

Before (fragmented):

  • Four teams = four ways to invoke agents = four auth systems = four audit trails
  • New runtime = new integrations everywhere
  • Moving an agent from Cursor to Bedrock = rewrite invocation code in every consumer
  • Cost tracking scattered across provider dashboards

After (control plane):

  • One control plane = one invocation API = one audit trail
  • New runtime = register once, works everywhere
  • Agent portability across runtimes = zero consumer code changes
  • Cost tracked per agent, per team, per runtime in one place

The Production Pattern

This is not theoretical. LiteLLM Agent Platform is a Rust-based AI Gateway and Agent Control Plane whose goal is to let teams register, invoke, observe, and govern agents across multiple runtimes, with LiteLLM treating this as an experimental project to learn quickly.

LiteLLM Agent Platform manages unified API across runtimes—one API to create and run agents, regardless of the runtime underneath, with developers creating and running agents here without requiring Bedrock or Anthropic console access.

The five-question evaluation framework:

  1. Can I invoke any agent through one API? (Not five provider SDKs)
  2. Can I change policies without redeploying agents? (Budgets, approvals, tool auth in control plane)
  3. Can agents move between runtimes without rewriting consumers? (Session portability)
  4. Do I get unified audit trails? (All calls logged in one system)
  5. Can new teams discover and use agents without console access? (Agent registry, centralized secrets)

If the answer is "no" to any of these, you're still in the fragmented phase.

Next: The Maturity Curve

July 2026 moment: Teams are shipping agents on multiple runtimes. The ones that don't solve the fragmentation problem hit walls around month 3:

  • Credential sprawl (who has access to what?)
  • Cost visibility (which runtime is cheaper?)
  • Policy inconsistency (why does one agent have budget approval, the other doesn't?)
  • Session loss (agent crashes mid-execution, work lost)

The ones that build a control plane first move faster. Fewer meetings about credential management. Fewer surprises on the bill. Fewer post-mortems about lost work.

The pattern is converging. Agent infrastructure is separating into three layers: models, harnesses, and runtimes, with a fourth layer emerging as the unified agent control plane to allow calling agents living in different agent runtimes all from 1 place, because companies will not run every agent on one runtime.

This is the infrastructure layer that will define who wins in multi-agent 2026. Not the smartest models. Not the flashiest frameworks. The teams with the clearest control plane.

Paul Twist is an AI infrastructure engineer based in Berlin. This post covers the multi-runtime control plane pattern emerging as table-stakes for production agent deployments in July 2026.