agentleFS
foversefoverse/seoverse/.agents/skills/verify-local-mcp/SKILL.md

verify-local-mcp

Skill4 starsChanged 13 days agoMITView on GitHub

Verify the OpenSEO MCP server end-to-end on a local dev server — protocol-level correctness against real DataForSEO, then a headless-agent consumer probe that tests tool ergonomics (descriptions, schemas, output size, errors, async flows) without the maintainer manually driving an MCP client. Use after adding or changing MCP tools, or when asked to check that the MCP "works" or "is ergonomic".

  • Reads credentials

1 other project uses a version of this.

Adopted by

Steady: 0 new projects took it in the last 4 weeks, none the 4 weeks before.thin data

Week of Sep 20: 0 new projectsWeek of Sep 27: 0 new projectsWeek of Oct 4: 0 new projectsWeek of Oct 11: 0 new projectsWeek of Oct 18: 0 new projectsWeek of Oct 25: 0 new projectsWeek of Nov 1: 0 new projectsWeek of Nov 8: 0 new projects
Sep 2026Oct 2026Nov 2026
New projects taking a version of this file each week, over the 8 weeks our latest run read: only 0 in all, so this is a hint, not a trend.

Most recently taken up by

The file

We split this skill into the rules it gives an agent

4 rules. About 0 are common, 0 are distinctive, and 4 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈0

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 4

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Very specific:

    Start pnpm dev:agents in the background.

    line 15
  2. Very specific:

    Test the happy path AND at least one edge per changed tool: an empty result (obscure query), an invalid identifier, and for queued tools …

    line 31
  3. Very specific:

    Use --model sonnet as the typical-client proxy — if sonnet navigates it cold, weaker clients likely can too.

    line 58
  4. Very specific:

    Schemas: every constraint the provider enforces silently must be in the field's .describe() (units, whole-number requirements, defaults, …

    line 63

≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Verify local MCP

Two layers, in order. The protocol layer proves the server and provider behave; the consumer layer proves an agent that has never seen the code can use the tools well. They catch different bugs — protocol testing found DataForSEO quirks (zoom-dependent empty SERPs), the consumer probe found ergonomics failures (9KB provider rows overflowing client token budgets, fractional inputs rejected upstream with raw provider errors). Do both.

1. Boot

  • .env.local needs AUTH_MODE=local_noauth and DATAFORSEO_API_KEY (base64 of login:password). Never print the key.
  • Start pnpm dev:agents in the background. The server URL is branch-prefixed: http://<branch-suffix>.open-seo.localhost:1355 (the exact URL is printed on boot; logs tee to .logs/dev-server.log).
  • With local_noauth, /mcp needs no token. Vite hot-reloads server code, so fix → re-call without restarting.
  • After changing a tool's input or output schema, refresh the client's tool discovery (tools/list) or reconnect the MCP client before calling it again. Clients can cache validators from the previous tool list and reject valid results after the provider call has already incurred a charge.

2. Protocol smoke (cheap, deterministic)

Raw JSON-RPC against /mcp — the layer for asserting exact shapes and driving edge cases (resume taskIds, empty results, invalid inputs):

curl -sS http://<url>/mcp \
  -H 'content-type: application/json' -H 'accept: application/json, text/event-stream' \
  -d '{"jsonrpc":"2.0","id":1,"method":"tools/list"}'
# tools/call: {"method":"tools/call","params":{"name":"<tool>","arguments":{...}}}
  • Bootstrap: list_projects, then create_project if empty — most tools need a projectId.
  • Test the happy path AND at least one edge per changed tool: an empty result (obscure query), an invalid identifier, and for queued tools the full lifecycle including resuming with the returned taskId.
  • These are real, billed DataForSEO calls (metering itself short-circuits in local_noauth — billing needs unit tests, not this). Keep depths 10–20.

3. Consumer probe (the ergonomics test)

Spawn a headless Claude subprocess connected as a real MCP client. Write a config:

{
  "mcpServers": {
    "openseo-local": { "type": "http", "url": "http://<url>/mcp" }
  }
}

Then run a NATURAL task — never name the tools; whether the model finds them from descriptions alone is the test:

claude -p "<natural task a customer would ask>. Keep spend minimal: depths 10-20, one 3x3 grid max, ~10 paid calls.
Deliver two sections: 1. FINDINGS — the task result. 2. MCP FEEDBACK — critique the MCP as a first-time consumer:
were descriptions enough to pick tools without trial and error? confusing schemas, surprising output shapes or sizes,
unclear errors, credit-cost surprises? Did async/taskId flows behave as described? List anything that made you hesitate or retry." \
  --mcp-config mcp-local.json --strict-mcp-config \
  --allowedTools "mcp__openseo-local,mcp__openseo-local__*" \
  --model sonnet --max-turns 30

Use --model sonnet as the typical-client proxy — if sonnet navigates it cold, weaker clients likely can too. Read FINDINGS for correctness (did it get real, sensible data?) and MCP FEEDBACK for the rubric below.

4. Ergonomics rubric — what feedback to act on

  • Tool selection: the probe should pick the right tool first try. Retries or wrong-tool detours mean a description needs a sharper "use this when / not this" sentence.
  • Schemas: every constraint the provider enforces silently must be in the field's .describe() (units, whole-number requirements, defaults, what's ignored when). If the probe guessed-and-retried an input, encode the rule server-side (coerce/round) or document it — prefer coercing.
  • Output size: budget roughly a few KB per row. Provider rows carrying popular_times/attribute trees/photo URLs must be trimmed to the fields the tool's job needs; point to the single-entity tool for the full shape.
  • Errors: actionable, never a raw upstream field name without a hint at the fix. Failures after a billed step must keep the recovery handle (e.g. the taskId) in the message.
  • Async copy: descriptions must match typical latency ("usually completes within this call") and the resume path must actually work when driven by the probe, not just by curl.
  • Credit honesty: each description's credit sentence matches reality, including cache-hit and resume paths.

5. Iterate and clean up

Fix findings → hot-reload picks them up → re-verify just the changed behavior via curl (cheap) → rerun the full consumer probe once per iteration round (it re-tests selection and flow, not just the fix). When done: stop the dev server background task, run the repo's tests/ci:check, and fold genuine provider quirks into code comments or tests so the next agent doesn't rediscover them.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.