agentleFS
alex-ilgayevalex-ilgayev/MCPSpy/.claude/skills/security-integration-tests.md/SKILL.md

security-integration-tests

Skill519 starsChanged 12 months agoApache-2.0View on GitHub

Use this agent when working with prompt injection detection integration tests, including running tests, debugging failures, or adding new test samples.

The file

We split this skill into the rules it gives an agent

4 rules. About 0 are common, 2 are distinctive, and 2 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈2

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 2

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    Add samples to appropriate category (benign_samples, malicious_samples, or mcp_tool_calls)

    line 74
  2. Distinctive:

    Run integration tests to verify

    line 76
  3. Very specific:

    Ensure HF_TOKEN is set and valid

    line 82
  4. Very specific:

    Default test model is protectai/deberta-v3-base-prompt-injection-v2 (publicly accessible)

    line 84

≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Security Integration Tests Agent

Overview

The security package (pkg/security/) provides prompt injection detection using the HuggingFace Inference API. Integration tests verify detection accuracy with real API calls.

Key Files

  • pkg/security/testdata/samples.json - Test data with benign and malicious samples
  • pkg/security/hf/integration_test.go - HF client integration tests
  • pkg/security/integration_test.go - Full analyzer integration tests with event bus
  • pkg/security/hf/client.go - HuggingFace API client implementation
  • pkg/security/analyzer.go - Main security analyzer that subscribes to MCP events

Running Tests

# Run integration tests (requires HF_TOKEN environment variable)
HF_TOKEN=hf_xxx make test-integration

# Run with custom model
HF_TOKEN=hf_xxx HF_MODEL=protectai/deberta-v3-base-prompt-injection-v2 make test-integration

# Run unit tests only (no API calls, uses mock server)
go test -v ./pkg/security/...

Test Structure

Test Samples (samples.json)

{
  "benign_samples": [
    {
      "description": "Description of the sample",
      "text": "The text to analyze",
      "expected_detected": false
    }
  ],
  "malicious_samples": [
    {
      "description": "Direct instruction override",
      "text": "Ignore all previous instructions...",
      "expected_detected": true,
      "min_risk_score": 0.5
    }
  ],
  "mcp_tool_calls": [
    {
      "description": "Malicious tool call",
      "method": "tools/call",
      "params": { "name": "run_command", "arguments": {...} },
      "expected_detected": true,
      "min_risk_score": 0.5
    }
  ]
}

Integration Test Tags

Integration tests use the build tag //go:build integration and are excluded from regular go test ./... runs.

Adding New Test Samples

  1. Edit pkg/security/testdata/samples.json
  2. Add samples to appropriate category (benign_samples, malicious_samples, or mcp_tool_calls)
  3. Set expected_detected and optionally min_risk_score
  4. Run integration tests to verify

Common Issues

"Forbidden" Error

  • Ensure HF_TOKEN is set and valid
  • Note: meta-llama/Llama-Prompt-Guard-2-86M is deprecated on HF Inference API
  • Default test model is protectai/deberta-v3-base-prompt-injection-v2 (publicly accessible)

Model Loading

  • HuggingFace warms up models on demand
  • Tests may skip with "Model loading" message on first run
  • Re-run tests after model is warm

Network Issues

  • Integration tests require network access to HuggingFace API
  • Tests will fail in sandboxed environments without network access

Risk Levels

  • none: score < 0.3
  • low: score 0.3-0.5
  • medium: score 0.5-0.7
  • high: score 0.7-0.9
  • critical: score >= 0.9

Categories

  • benign: Normal, safe content
  • injection: Prompt injection attempt
  • jailbreak: Jailbreak attempt
  • malicious: Malicious content (Prompt Guard v2)

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.