MCPSpy / tests
CLAUDE.md519 starsChanged 12 months agoApache-2.0View on GitHub
Requires HF_TOKEN environment variable: Anthropic Claude API (requires CLAUDE_CODE_OAUTH_TOKEN): Google Gemini API (requires GEMINI_API_KEY): Requires GEMINI_API_KEY environment variable: The e2e_test.py is the core test runner. It loads YAML configuration and executes test scenarios. What it tests: What it tests: What it tests: Requirements:
- Reads credentials
In this repository5 other files it gives its agents.
- .claude/CLAUDE.mdCLAUDE.md
- git-commit-creatorSkill
- github-issue-creatorSkill
- go-testingSkill
- security-integration-testsSkill
The file
We split this CLAUDE.md into the rules it gives an agent
2 rules. About 0 are common, 2 are distinctive, and 0 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈2
Distinctive · estimate
This author's own choices. Worth a look.
0
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Distinctive: line 111
All MCP message types (tools, resources, prompts, ping)
- Distinctive: line 119
All MCP message types over HTTP
≈ “Common” is an estimate: at least 1 in 5 of the 27,641 CLAUDE.md files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
MCPSpy Tests Guide
Quick Start
Run All Tests
make test-e2eRun Specific Transport
make test-e2e-stdio # Stdio transport
make test-e2e-https # HTTPS transportRun Scenario Only (no MCPSpy)
make test-scenario-stdio # Stdio transport
make test-scenario-https # HTTPS transport
make test-scenario-security # Security/injection test
make test-scenario-llm-anthropic # Anthropic LLM API test
make test-scenario-llm-gemini # Gemini LLM API test
make test-scenario-claudecode # Claude Code test (MCP + LLM + Tools)
make test-scenario-gemini-cli # Gemini CLI test (MCP + LLM + Tools)Run Security/Prompt Injection E2E Test
Requires HF_TOKEN environment variable:
HF_TOKEN=your_huggingface_token make test-e2e-securityRun LLM API Monitoring E2E Tests
Anthropic Claude API (requires CLAUDE_CODE_OAUTH_TOKEN):
CLAUDE_CODE_OAUTH_TOKEN=your_api_key make test-e2e-llm-anthropicGoogle Gemini API (requires GEMINI_API_KEY):
GEMINI_API_KEY=your_api_key make test-e2e-llm-geminiRun Gemini CLI E2E Test
Requires GEMINI_API_KEY environment variable:
GEMINI_API_KEY=your_api_key make test-e2e-gemini-cliUpdate Expected Outputs
make test-update-all # All scenarios
make test-update-stdio # Specific scenario
make test-update-https
make test-update-llm-anthropic
make test-update-llm-geminie2e_test.py CLI Utility
The e2e_test.py is the core test runner. It loads YAML configuration and executes test scenarios.
Basic Usage
# Run all scenarios from config
python tests/e2e_test.py --config tests/e2e_config.yaml
# Run specific scenario
python tests/e2e_test.py --config tests/e2e_config.yaml --scenario stdio-fastmcp
# Update expected output for a scenario
python tests/e2e_test.py --config tests/e2e_config.yaml --scenario http-fastmcp --update-expected
# Enable verbose output
python tests/e2e_test.py --config tests/e2e_config.yaml --verboseCommand-Line Arguments
| Argument | Type | Required | Description |
--config | Path | Yes | Path to YAML configuration file |
--scenario | String | No | Run specific scenario by name (default: all scenarios) |
--update-expected | Flag | No | Update expected output files instead of validating |
--verbose, -v | Flag | No | Enable verbose logging output |
--skip-mcpspy | Flag | No | Skip MCPSpy monitoring - only run traffic generation and pre/post commands (useful for debugging MCP implementations) |
Test Scenarios
stdio-fastmcp
What it tests:
- Direct subprocess communication via stdio
- All MCP message types (tools, resources, prompts, ping)
- Request/response pairing
http-fastmcp
What it tests:
- HTTPS transport with self-signed certificates
- All MCP message types over HTTP
- StreamableHTTP protocol handling
security-injection
What it tests:
- Prompt injection detection via HuggingFace API
- Security alert event generation for malicious content
- Benign vs malicious content classification
- Integration of security analyzer with MCP event processing
Requirements:
HF_TOKENenvironment variable with valid HuggingFace API token
llm-anthropic
What it tests:
- LLM API monitoring with Anthropic Claude API
- Non-streaming API request/response capture
- Streaming (SSE) API request/response capture
- Model and content extraction from API calls
Requirements:
CLAUDE_CODE_OAUTH_TOKENenvironment variable with valid Claude Code OAuth API key
llm-gemini
What it tests:
- LLM API monitoring with Google Gemini API
- Non-streaming API request/response capture
- Streaming API request/response capture
- Model and content extraction from API calls
Requirements:
GEMINI_API_KEYenvironment variable with valid Gemini API key
claude-code
What it tests:
- MCP server initialization (filesystem and deepwiki HTTP servers)
- Claude Code tool usage (Read, Bash tools)
- LLM API calls (Anthropic API requests/responses)
- Tool invocation events (CALL) and tool result events (RSLT)
- Predictable tool execution with known inputs/outputs
MCPSpy flags: --llm --tools
Test data:
tests/test_tool_data.txt- Test file with known content for Read tool testingtests/claude_config.json- MCP server configuration
gemini-cli
What it tests:
- MCP server initialization (filesystem and deepwiki HTTP servers)
- Gemini CLI tool usage (Read, Bash tools)
- LLM API calls (Gemini API requests/responses)
- Tool invocation events (CALL) and tool result events (RSLT)
- Predictable tool execution with known inputs/outputs
Requirements:
GEMINI_API_KEYenvironment variable with valid Gemini API key- Gemini CLI installed (via
npx @google/gemini-cli)
MCPSpy flags: --llm --tools
Test data:
tests/test_tool_data.txt- Test file with known content for Read tool testingtests/.gemini/settings.json- MCP server configuration
File Reference
| File | Purpose |
e2e_test.py | Main test runner - loads config and executes scenarios |
e2e_config.yaml | Test scenario definitions and validation rules |
e2e_config_schema.py | Pydantic schema for config validation |
mcp_server.py | FastMCP test server (stdio/http/sse transports) |
mcp_client.py | MCP client that generates test traffic |
llm_client.py | LLM API client for Anthropic API testing |
llm_gemini_client.py | LLM API client for Gemini API testing |
test_tool_data.txt | Test file for Claude Code Read tool testing |
claude_config.json | MCP server config for Claude Code tests |
.gemini/settings.json | MCP server configuration for Gemini CLI tests |
expected_output_stdio.jsonl | Expected output for stdio transport |
expected_output_http.jsonl | Expected output for HTTP transport |
expected_output_security.jsonl | Expected output for security/injection test |
expected_output_llm_anthropic.jsonl | Expected output for LLM API monitoring test |
expected_output_llm_gemini.jsonl | Expected output for Gemini LLM API monitoring test |
server.key, server.crt | Self-signed SSL certificates for HTTPS tests |
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

