agentleFS
alex-ilgayevalex-ilgayev/MCPSpy/tests/CLAUDE.md

MCPSpy / tests

CLAUDE.md519 starsChanged 12 months agoApache-2.0View on GitHub

Requires HF_TOKEN environment variable: Anthropic Claude API (requires CLAUDE_CODE_OAUTH_TOKEN): Google Gemini API (requires GEMINI_API_KEY): Requires GEMINI_API_KEY environment variable: The e2e_test.py is the core test runner. It loads YAML configuration and executes test scenarios. What it tests: What it tests: What it tests: Requirements:

  • Reads credentials

The file

We split this CLAUDE.md into the rules it gives an agent

2 rules. About 0 are common, 2 are distinctive, and 0 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈2

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 0

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    All MCP message types (tools, resources, prompts, ping)

    line 111
  2. Distinctive:

    All MCP message types over HTTP

    line 119

≈ “Common” is an estimate: at least 1 in 5 of the 27,641 CLAUDE.md files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
CLAUDE.md

MCPSpy Tests Guide

Quick Start

Run All Tests

make test-e2e

Run Specific Transport

make test-e2e-stdio      # Stdio transport
make test-e2e-https      # HTTPS transport

Run Scenario Only (no MCPSpy)

make test-scenario-stdio          # Stdio transport
make test-scenario-https          # HTTPS transport
make test-scenario-security       # Security/injection test
make test-scenario-llm-anthropic  # Anthropic LLM API test
make test-scenario-llm-gemini     # Gemini LLM API test
make test-scenario-claudecode         # Claude Code test (MCP + LLM + Tools)
make test-scenario-gemini-cli         # Gemini CLI test (MCP + LLM + Tools)

Run Security/Prompt Injection E2E Test

Requires HF_TOKEN environment variable:

HF_TOKEN=your_huggingface_token make test-e2e-security

Run LLM API Monitoring E2E Tests

Anthropic Claude API (requires CLAUDE_CODE_OAUTH_TOKEN):

CLAUDE_CODE_OAUTH_TOKEN=your_api_key make test-e2e-llm-anthropic

Google Gemini API (requires GEMINI_API_KEY):

GEMINI_API_KEY=your_api_key make test-e2e-llm-gemini

Run Gemini CLI E2E Test

Requires GEMINI_API_KEY environment variable:

GEMINI_API_KEY=your_api_key make test-e2e-gemini-cli

Update Expected Outputs

make test-update-all               # All scenarios
make test-update-stdio             # Specific scenario
make test-update-https
make test-update-llm-anthropic
make test-update-llm-gemini

e2e_test.py CLI Utility

The e2e_test.py is the core test runner. It loads YAML configuration and executes test scenarios.

Basic Usage

# Run all scenarios from config
python tests/e2e_test.py --config tests/e2e_config.yaml

# Run specific scenario
python tests/e2e_test.py --config tests/e2e_config.yaml --scenario stdio-fastmcp

# Update expected output for a scenario
python tests/e2e_test.py --config tests/e2e_config.yaml --scenario http-fastmcp --update-expected

# Enable verbose output
python tests/e2e_test.py --config tests/e2e_config.yaml --verbose

Command-Line Arguments

ArgumentTypeRequiredDescription
--configPathYesPath to YAML configuration file
--scenarioStringNoRun specific scenario by name (default: all scenarios)
--update-expectedFlagNoUpdate expected output files instead of validating
--verbose, -vFlagNoEnable verbose logging output
--skip-mcpspyFlagNoSkip MCPSpy monitoring - only run traffic generation and pre/post commands (useful for debugging MCP implementations)

Test Scenarios

stdio-fastmcp

What it tests:

  • Direct subprocess communication via stdio
  • All MCP message types (tools, resources, prompts, ping)
  • Request/response pairing

http-fastmcp

What it tests:

  • HTTPS transport with self-signed certificates
  • All MCP message types over HTTP
  • StreamableHTTP protocol handling

security-injection

What it tests:

  • Prompt injection detection via HuggingFace API
  • Security alert event generation for malicious content
  • Benign vs malicious content classification
  • Integration of security analyzer with MCP event processing

Requirements:

  • HF_TOKEN environment variable with valid HuggingFace API token

llm-anthropic

What it tests:

  • LLM API monitoring with Anthropic Claude API
  • Non-streaming API request/response capture
  • Streaming (SSE) API request/response capture
  • Model and content extraction from API calls

Requirements:

  • CLAUDE_CODE_OAUTH_TOKEN environment variable with valid Claude Code OAuth API key

llm-gemini

What it tests:

  • LLM API monitoring with Google Gemini API
  • Non-streaming API request/response capture
  • Streaming API request/response capture
  • Model and content extraction from API calls

Requirements:

  • GEMINI_API_KEY environment variable with valid Gemini API key

claude-code

What it tests:

  • MCP server initialization (filesystem and deepwiki HTTP servers)
  • Claude Code tool usage (Read, Bash tools)
  • LLM API calls (Anthropic API requests/responses)
  • Tool invocation events (CALL) and tool result events (RSLT)
  • Predictable tool execution with known inputs/outputs

MCPSpy flags: --llm --tools

Test data:

  • tests/test_tool_data.txt - Test file with known content for Read tool testing
  • tests/claude_config.json - MCP server configuration

gemini-cli

What it tests:

  • MCP server initialization (filesystem and deepwiki HTTP servers)
  • Gemini CLI tool usage (Read, Bash tools)
  • LLM API calls (Gemini API requests/responses)
  • Tool invocation events (CALL) and tool result events (RSLT)
  • Predictable tool execution with known inputs/outputs

Requirements:

  • GEMINI_API_KEY environment variable with valid Gemini API key
  • Gemini CLI installed (via npx @google/gemini-cli)

MCPSpy flags: --llm --tools

Test data:

  • tests/test_tool_data.txt - Test file with known content for Read tool testing
  • tests/.gemini/settings.json - MCP server configuration

File Reference

FilePurpose
e2e_test.pyMain test runner - loads config and executes scenarios
e2e_config.yamlTest scenario definitions and validation rules
e2e_config_schema.pyPydantic schema for config validation
mcp_server.pyFastMCP test server (stdio/http/sse transports)
mcp_client.pyMCP client that generates test traffic
llm_client.pyLLM API client for Anthropic API testing
llm_gemini_client.pyLLM API client for Gemini API testing
test_tool_data.txtTest file for Claude Code Read tool testing
claude_config.jsonMCP server config for Claude Code tests
.gemini/settings.jsonMCP server configuration for Gemini CLI tests
expected_output_stdio.jsonlExpected output for stdio transport
expected_output_http.jsonlExpected output for HTTP transport
expected_output_security.jsonlExpected output for security/injection test
expected_output_llm_anthropic.jsonlExpected output for LLM API monitoring test
expected_output_llm_gemini.jsonlExpected output for Gemini LLM API monitoring test
server.key, server.crtSelf-signed SSL certificates for HTTPS tests

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.