security-integration-tests
Skill519 starsChanged 12 months agoApache-2.0View on GitHub
Use this agent when working with prompt injection detection integration tests, including running tests, debugging failures, or adding new test samples.
In this repository5 other files it gives its agents.
- .claude/CLAUDE.mdCLAUDE.md
- tests/CLAUDE.mdCLAUDE.md
- git-commit-creatorSkill
- github-issue-creatorSkill
- go-testingSkill
The file
We split this skill into the rules it gives an agent
4 rules. About 0 are common, 2 are distinctive, and 2 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈2
Distinctive · estimate
This author's own choices. Worth a look.
2
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Distinctive: line 74
Add samples to appropriate category (benign_samples, malicious_samples, or mcp_tool_calls)
- Distinctive: line 76
Run integration tests to verify
- Very specific: line 82
Ensure HF_TOKEN is set and valid
- Very specific: line 84
Default test model is protectai/deberta-v3-base-prompt-injection-v2 (publicly accessible)
≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Security Integration Tests Agent
Overview
The security package (pkg/security/) provides prompt injection detection using the HuggingFace Inference API. Integration tests verify detection accuracy with real API calls.
Key Files
pkg/security/testdata/samples.json- Test data with benign and malicious samplespkg/security/hf/integration_test.go- HF client integration testspkg/security/integration_test.go- Full analyzer integration tests with event buspkg/security/hf/client.go- HuggingFace API client implementationpkg/security/analyzer.go- Main security analyzer that subscribes to MCP events
Running Tests
# Run integration tests (requires HF_TOKEN environment variable)
HF_TOKEN=hf_xxx make test-integration
# Run with custom model
HF_TOKEN=hf_xxx HF_MODEL=protectai/deberta-v3-base-prompt-injection-v2 make test-integration
# Run unit tests only (no API calls, uses mock server)
go test -v ./pkg/security/...Test Structure
Test Samples (samples.json)
{
"benign_samples": [
{
"description": "Description of the sample",
"text": "The text to analyze",
"expected_detected": false
}
],
"malicious_samples": [
{
"description": "Direct instruction override",
"text": "Ignore all previous instructions...",
"expected_detected": true,
"min_risk_score": 0.5
}
],
"mcp_tool_calls": [
{
"description": "Malicious tool call",
"method": "tools/call",
"params": { "name": "run_command", "arguments": {...} },
"expected_detected": true,
"min_risk_score": 0.5
}
]
}Integration Test Tags
Integration tests use the build tag //go:build integration and are excluded from regular go test ./... runs.
Adding New Test Samples
- Edit
pkg/security/testdata/samples.json - Add samples to appropriate category (benign_samples, malicious_samples, or mcp_tool_calls)
- Set
expected_detectedand optionallymin_risk_score - Run integration tests to verify
Common Issues
"Forbidden" Error
- Ensure HF_TOKEN is set and valid
- Note:
meta-llama/Llama-Prompt-Guard-2-86Mis deprecated on HF Inference API - Default test model is
protectai/deberta-v3-base-prompt-injection-v2(publicly accessible)
Model Loading
- HuggingFace warms up models on demand
- Tests may skip with "Model loading" message on first run
- Re-run tests after model is warm
Network Issues
- Integration tests require network access to HuggingFace API
- Tests will fail in sandboxed environments without network access
Risk Levels
none: score < 0.3low: score 0.3-0.5medium: score 0.5-0.7high: score 0.7-0.9critical: score >= 0.9
Categories
benign: Normal, safe contentinjection: Prompt injection attemptjailbreak: Jailbreak attemptmalicious: Malicious content (Prompt Guard v2)
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

