AI Agent Security Audit Checklist: 8 Critical Tests for Production Deployments

AI Agent Security Audit Checklist: 8 Critical Tests for Production Deployments

AI agents are no longer experimental. In 2026, enterprises are deploying LLM-powered agents that read databases, execute code, send emails, and control production infrastructure. The question is no longer "should we use AI agents?" but "how do we secure them in production?"

This article is the fourth in our AI Runtime Security series. We've covered the macro landscape, MCP penetration testing methodology, and why runtime call verification is the missing layer. Here, we distill that experience into a practical, actionable checklist — 8 tests every security team should run before putting AI agents into production.

Why this matters: We've audited 10+ AI agent frameworks using the Correctover CCS scanner, producing over 1,730 verified findings across 12 codebases. Of those, 87 are confirmed production vulnerabilities — real bugs in shipping code, not theoretical attack surfaces. Every item on this checklist is grounded in actual vulnerabilities we've found, reported, and in many cases had patched.

---

Checklist Overview

#	Test Area	Severity	Frameworks Affected
1	Tool Authorization & Read-Only Enforcement	CRITICAL	MCP SDK, AutoGen, Semantic Kernel, FastMCP, Dify, Griptape
2	MCP/Subprocess Command Injection	CRITICAL	CrewAI, LiteLLM, AutoGen, Docker MCP
3	Deserialization & Eval Injection	CRITICAL	AG2, LlamaIndex, Haystack
4	Path Traversal in Configuration Loading	CRITICAL	AutoGen, Dify
5	Environment Variable Leakage	HIGH	MCP Python SDK, FastMCP
6	MCP Transport Security	HIGH	All STDIO-based MCP implementations
7	Runtime Call Verification (The Missing Layer)	MEDIUM	All frameworks (nobody does this)
8	Model/Provider Supply Chain Security	HIGH	All LLM deployments

---

Test 1: Tool Authorization & Read-Only Enforcement

The Problem: Most AI agent frameworks define a readOnlyHint or similar permission flag for tools, but none of them actually enforce it. We discovered this at the protocol level in the official MCP Python SDK — the readOnlyHint field exists in the schema, but the runtime never checks it before executing a tool call. This means every downstream framework inherits the vulnerability.

• MCP Python SDK: 43 instances of AGT-TOOL-NO-READONLY
• AutoGen (Microsoft): 6 instances
• FastMCP: 25 instances
• Dify: 7 instances
• Semantic Kernel (Microsoft): 2 instances
• Griptape: 1 instance

How to Test:

# Scan your MCP server definition for readOnlyHint usage
grep -r "readOnlyHint" --include="*.py" --include="*.ts" --include="*.json" .

# If the property is defined but never checked at the transport layer,
# every tool is writable regardless of what the schema says.

Enter fullscreen mode Exit fullscreen mode

Also check for tool-level permission enforcement:

# Does your framework actually check permissions before execution?
# Look for patterns like:
if not tool.is_readonly and not user_has_write_permission:
    raise PermissionError("Write access required")
# Most frameworks simply skip this check entirely.

Enter fullscreen mode Exit fullscreen mode

What a Good Result Looks Like:

• readOnlyHint is enforced at the transport/runtime layer, not just declared in the schema
• Each tool call is validated against an explicit allowlist
• Write operations require explicit user confirmation or elevated authorization

How Correctover CCS Addresses This:
Our CCS scanner includes rule AGT-TOOL-NO-READONLY that specifically detects frameworks where tool permission flags are declared but unenforced. We reported this to the MCP Python SDK maintainers via HackerOne (Report #3878033, CVSS 7.5) and have parallel submissions for AutoGen and Semantic Kernel through MSRC.

---

Test 2: MCP/Subprocess Command Injection

The Problem: The Model Context Protocol (MCP) uses STDIO transport — the parent process spawns a child process and communicates via stdin/stdout. If the command or arguments passed to stdio_client() come from untrusted input (LLM output, user config, MCP server manifest), you have a command injection vulnerability with CVSS 9.8.

Here's the real-world damage:

• CrewAI MCP RCE (CVE-2026-2287, CVSS 9.8): The StdioTransport.__init__() in crewai/mcp/transports/stdio.py (line 92-97) passes command and args directly to stdio_client() with zero validation. An attacker who controls the MCP server configuration can execute arbitrary OS commands. Zero protection layers. Reported to MSRC, case 126356.

• LiteLLM allowlist bypass (CVE-2026-30623): LitellM has an allowlist, but it can be bypassed using python -c or node -e argument injection.

• Docker MCP: Command injection at the protocol level — the Docker MCP server passes user-controlled parameters directly to subprocess calls.

How to Test:

# Minimal PoC — test if your framework validates commands
import subprocess

# If you can inject an unintended command, your framework is vulnerable
test_cases = [
    "python -c 'import os; os.system(\"calc.exe\")'",
    "bash -c 'echo $FLAG > /tmp/pwned'",
    "node -e 'require(\"child_process\").execSync(\"id\")'"
]

Enter fullscreen mode Exit fullscreen mode

Check your MCP transport layer for any code that looks like:

# UNSAFE — no validation
transport = stdio_client(command, args)
# SAFE — command must be on allowlist
if command not in ALLOWED_COMMANDS:
    raise SecurityError(f"Command {command} not allowed")

Enter fullscreen mode Exit fullscreen mode

What a Good Result Looks Like:

• Commands are validated against a strict allowlist (not just blocklist)
• Arguments are sanitized or constrained
• The framework rejects any command that isn't explicitly authorized

How Correctover CCS Addresses This:
Our rules CW-MCP-001 (CrewAI), MCP-STDIO-001 (cross-framework), and command_injection scanners detect unprotected stdio_client() calls. We've submitted findings through MSRC, HackerOne, and GitHub Security Advisories covering CrewAI, LiteLLM, AutoGen, and Docker MCP.

---

Test 3: Deserialization & Eval Injection

The Problem: AI agent frameworks frequently use eval(), pickle.loads(), or yaml.load() for configuration parsing, workflow serialization, and context expression evaluation. When LLM-controlled data reaches these functions, the result is unauthenticated remote code execution.

Verified vulnerabilities:

• AG2 (Microsoft) eval() str bypass (CVSS 9.8): In autogen/agentchat/group/context_expression.py line 228, eval() escapes string values but does not escape __str__() return values of non-string objects. The code's own comment (lines 218-221) admits: "custom str injection is out of scope." Attackers control the LLM output feeding context_variables, which passes a non-string object whose __str__() returns malicious code. GitHub Issue #3073. PoC verified — file write confirmed.

• LlamaIndex Workflows Pickle RCE (CVSS 9.8): In workflows/context/serializers.py line 243, pickle.loads(base64.b64decode(value)) is the default serializer for workflow state persistence. Classic pickle RCE — fully exploitable. GitHub Issue #22296. PoC verified.

• Haystack Pipeline RCE (2 CRITICAL): Two confirmed RCE paths through pipeline serialization deserialization.

How to Test:

# Scan for dangerous function calls in your agent framework
grep -rn "eval(" --include="*.py" . | grep -v "test" | grep -v "__pycache__"
grep -rn "pickle.loads" --include="*.py" . | grep -v "test" | grep -v "__pycache__"
grep -rn "yaml.load(" --include="*.py" . | grep -v "yaml.safe_load"

Enter fullscreen mode Exit fullscreen mode

What a Good Result Looks Like:

• No use of eval(), pickle.loads(), or unsafe yaml.load() in production code paths
• All serialization uses safe alternatives: json.loads(), yaml.safe_load(), or validated schema-based deserialization
• Input to any serializer is sanitized and type-checked before deserialization

How Correctover CCS Addresses This:
Rules AG2-EVAL-002, LI-PICKLE-001, LI-JSON-001 (all CRITICAL) detect unsafe deserialization patterns. We also scan for JSON deserialization without value validation (which can lead to prototype pollution or schema injection).

---

Test 4: Path Traversal in Configuration Loading

The Problem: AI agents load configuration dynamically — model configs, tool definitions, MCP server manifests. When filenames come from user input or LLM output, path traversal opens the filesystem to attackers.

• AutoGen magentic-one-cli (P1-PATH, CVSS 9.8): In _m1.py line 105, the --config parameter is passed directly to open() with no path validation. An attacker controlling the config path reads any file on the system. Submitted to MSRC.

• Dify Apollo config (P1-PATH, CVSS 9.8): In python_3x.py line 27, user-controlled config_file_path passed directly to open(). Submitted to ZDI.

How to Test:

# Test for path traversal in config loading
test_payloads = [
    "../../../../etc/passwd",
    "....//....//....//etc/passwd",
    "..\\..\\..\\windows\\win.ini",
    "%2e%2e%2f%2e%2e%2fetc%2fpasswd"
]

# If your agent framework accepts config paths from any untrusted source,
# try injecting traversal sequences

Enter fullscreen mode Exit fullscreen mode

What a Good Result Looks Like:

• All file paths are resolved against a sandboxed directory
• Path traversal sequences (../) are rejected or sanitized
• File access uses an allowlist of permitted paths, not blocklists

How Correctover CCS Addresses This:
Our P1-PATH scanner specifically targets path traversal in AI agent frameworks. The pattern is aggressive — we test for encoded traversal sequences, double-dot variants, and OS-specific delimiters.

---

Test 5: Environment Variable Leakage

The Problem: AI agents inherit the parent process's environment, including API keys, database credentials, and service tokens. Several frameworks pass the full os.environ to child processes, effectively broadcasting secrets to any subprocess the agent spawns.

• MCP Python SDK (cli.py line 280): os.environ passed to subprocess without filtering.
• FastMCP (cli.py line 310, apps_dev.py line 1699): Same pattern — unfiltered env inheritance.

How to Test:

# Check if your framework filters environment variables before spawning subprocesses
grep -rn "os.environ" --include="*.py" . | grep -i "subprocess\|Popen\|run\|exec"

Enter fullscreen mode Exit fullscreen mode

What a Good Result Looks Like:

• Subprocesses receive only the minimum required environment variables
• Secrets (API keys, tokens, passwords) are explicitly excluded from child process env
• Environment variables with names matching KEY, TOKEN, SECRET, PASSWORD are filtered

How Correctover CCS Addresses This:
Rule AGT-ENV-LEAK (CVSS 7.0) detects unfiltered os.environ propagation. We reported this to the MCP Python SDK and FastMCP maintainers through ZDI.

---

Test 6: MCP Transport Security

The Problem: MCP STDIO transport is, by design, a thin pipe between a parent process and a child process. But when the parent is an AI agent making tool calls based on LLM reasoning, every tool becomes a potential RCE vector. The core issue: MCP has no built-in authentication, authorization, or encryption for the STDIO transport layer.

The cross-framework impact matrix we documented in MCP-STDIO-001 shows:

Framework	Protection Level	What's Vulnerable
Anthropic MCP SDK (official)	None (by design)	stdio_client() — no command validation
AutoGen + AutoGen Studio	None	Dual attack path: both Python and UI
LiteLLM	Allowlist (bypassable)	python -c / node -e bypass
CrewAI	Zero	Most vulnerable mainstream framework
FastMCP	None	Inherits from MCP SDK base

How to Test:

# Verify your MCP transport layer
from mcp import stdio_client

# Test 1: Can you inject arguments?
try:
    stdio_client("python", ["-c", "import os; os.system('echo VULNERABLE')"])
    print("UNSAFE: No argument validation")
except Exception:
    print("SAFE: Arguments validated")

# Test 2: Does it validate the command itself?
try:
    stdio_client("malicious-binary", ["--exploit"])
    print("UNSAFE: No command allowlist")
except Exception:
    print("SAFE: Command allowlist present")

Enter fullscreen mode Exit fullscreen mode

What a Good Result Looks Like:

• MCP transport uses a command allowlist
• Arguments are validated and sanitized
• STDIO communication is wrapped in at least transport-level integrity checks

How Correctover CCS Addresses This:
Our MCP-STDIO-001 cross-framework scanner checks all known MCP implementations for the same class of vulnerability. We coordinate disclosure through MSRC, HackerOne, and direct maintainer contact.

---

Test 7: Runtime Call Verification

The Problem: This is the missing layer in AI agent security — and arguably the most important gap to close as agents become autonomous.

Every existing security tool works before or after a tool call:

• Guardrails (Lakera Guard, NVIDIA NeMo): Filter input and output content
• Red-teaming tools (Garak, PyRIT, Giskard): Find vulnerabilities pre-deployment
• Agent governance (Zenity, Noma): Control who deploys agents
• Observability (WhyLabs, Arize): Monitor performance and drift

None of these tools inspect what happens during the tool call itself.

When an AI agent calls read_file("/etc/passwd") or exec_sql("DROP TABLE users"), the security decision is made at the call site — not before, not after. If nobody checks "is this specific call authorized?", the agent acts on its own judgment, which is exactly what an attacker exploits through prompt injection.

How to Test:

# Ask: does your deployment have any runtime enforcement for individual tool calls?
# 1. Can you log every tool call with full input/output? (observability)
# 2. Can you BLOCK a tool call mid-flight based on policy? (enforcement)
# 3. Can you intercept and verify the call argument before execution? (validation)

# Most teams answer "no" to questions 2 and 3.

Enter fullscreen mode Exit fullscreen mode

What a Good Result Looks Like:

• Every tool call is intercepted and verified before execution
• Verification includes: argument validation, permission check, anomaly detection
• Blocked calls generate alerts with full context for incident response
• Performance impact is under 100µs to avoid affecting agent latency

How Correctover CCS Addresses This:

Correctover CCS is built specifically for this gap. It operates as an interceptor layer between the agent and its tools, verifying every call in real time:

• Detection rules: 24 CCS rules covering command injection, path traversal, argument tampering, permission bypass, secret leakage, and more
• Performance: P50 of 22µs and P99 of 99µs per call — measured across our test suite of 80,000 API traces spanning 13 providers and 33 models
• Coverage: Detects patterns across MCP, REST, gRPC, and local function calls

Where guardrails say "don't send bad input" and observability says "log what happened," CCS says "intercept the call, verify the arguments, enforce the policy — in real time."

---

Test 8: Model/Provider Supply Chain Security

The Problem: Modern AI agents route through multiple providers and models — sometimes switching between them for cost optimization. Each provider change introduces new attack surface: different API formats, different auth mechanisms, different safety configurations.

Real data from our research: We've analyzed 80,000+ real API traces across 13 providers and 33 models. The distribution reveals that most production deployments use 3-7 providers simultaneously, and the configuration drift between them is significant.

• One provider might respect max_tokens properly; another might silently exceed it
• Safety filters differ across providers — a prompt rejected by one passes through another
• Rate limiting, retry behavior, and error handling vary wildly

How to Test:

# Audit your provider configuration
# 1. List all LLM providers your agent uses
# 2. For each, verify: auth method, rate limits, safety configuration
# 3. Check for any provider that uses HTTP (not HTTPS) or self-signed certs
# 4. Test: can you force the agent to switch to a compromised provider?

Enter fullscreen mode Exit fullscreen mode

What a Good Result Looks Like:

• All providers use TLS with certificate validation
• Each provider has documented safety configuration (content filters, rate limits)
• Runtime provider switching requires explicit authorization
• Provider credentials are stored separately from application code

How Correctover CCS Addresses This:
Our trace analysis pipeline has cataloged provider behaviors across 33 models, building a behavioral baseline for anomaly detection. If a provider starts returning unexpected responses or deviating from its safety configuration, CCS flags the deviation.

---

Test Summary Matrix

#	Test Area	Easy Win	Effort	Impact
1	Tool Authorization	Add readOnlyHint enforcement check	Low	Critical
2	Command Injection	Add command allowlist	Medium	Critical
3	Deserialization	Replace eval()/pickle with JSON	Medium	Critical
4	Path Traversal	Sandbox file access to working dir	Low	Critical
5	Env Leakage	Filter os.environ for subprocesses	Low	High
6	MCP Transport Security	Validate commands and args	Medium	Critical
7	Runtime Call Verification	Add interceptor middleware	Medium	High
8	Provider Supply Chain	Standardize provider config audit	Low	High

---

The Bottom Line

AI agent security in 2026 has a fundamental asymmetry: the capabilities are growing faster than the security controls. Every item on this checklist is grounded in real vulnerabilities we've found in production frameworks — not theoretical exercises.

Here's what we've learned from auditing 10+ frameworks and producing 1,730+ verified findings:

1. The most dangerous vulnerability is the one nobody is looking for. When we found the readOnlyHint bypass at the MCP protocol level, it had existed for over a year — present in the SDK specification, documented but unenforced, inherited by every downstream implementation.

2. Command injection is the new SQL injection. Just as web applications in the 2000s had to learn to sanitize SQL queries, AI agent frameworks in 2026 must learn to sanitize tool call inputs. The patterns are identical — the only difference is the execution context.

3. Runtime call verification is the next essential security layer. Every other security control works around the call. The call itself — the moment when an AI agent decides to execute an action — needs protection. With P50 latency of 22µs, this protection is feasible without compromising user experience.

4. Security is a product differentiator. When we notified vendors about the vulnerabilities we found, the response was consistent: "we didn't know this was a problem." Teams deploying agents today can get ahead of the curve by running these 8 tests — before the attackers do.

---

This article is part of the AI Runtime Security series by Correctover. We build CCS, the runtime call verification layer for AI agents — intercepting and verifying every tool call in real time. Our scanner has analyzed 10+ frameworks, produced 1,730+ findings, and processes 80,000+ API traces across 13 providers and 33 models.

Previous articles in this series:

1. The State of AI Security in 2026 — link
2. MCP Penetration Testing: A Practical Guide — link
3. AI Security Landscape 2026: Why Runtime Call Verification is the Missing Layer — link