scrapingdog
Skill3 starsChanged 43 days agoMITView on GitHub
Collect specialized public data through ScrapingDog when its configured route is selected. Use for Maps, social, commerce, jobs and other dedicated endpoints; generic web discovery and public-page acquisition belong to research.
In this repository9 other files it gives its agents.
- AGENTS.mdAGENTS.md
- instructions/AGENTS.mdAGENTS.md
- agent-graphSkill
- computer-useSkill
- frontend-visual-validationSkill
- remove-ai-marksSkill
- researchSkill
- rule-curatorSkill
- stagehand-browserSkill
The file
We split this skill into the rules it gives an agent
5 rules. About 0 are common, 4 are distinctive, and 1 is very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈4
Distinctive · estimate
This author's own choices. Worth a look.
1
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Very specific: line 13
Use research and its configured Scrapinho MCP for generic web discovery and public-page acquisition.
- Distinctive: line 17
Use an existing HTTP helper when that specialized endpoint is not exposed by MCP.
- Distinctive: line 18
Read only the relevant endpoint family below.
- Distinctive: line 23
After a missing key or non-refusal provider failure, disclose the reason before using an available specialized alternative.
- Distinctive: line 41
Use parameter objects or URL encoding for HTTP queries, a bounded timeout and sanitized errors.
≈ “Common” is an estimate: at least 1 in 5 of the 350,495 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
ScrapingDog
Resolve this skill directory before running its scripts/key-env-check.sh.
The checker reports whether the key is available in the current or interactive
shell without printing it. Never expose a key, raw account response or
credential-bearing request URL.
Use research and its configured Scrapinho MCP for generic web discovery and
public-page acquisition. This package owns specialized endpoints without
verified parity, not an automatic fallback from a refused request.
Inspect the available MCP tools and select the dedicated endpoint for the task.
Use an existing HTTP helper when that specialized endpoint is not exposed by MCP.
Read only the relevant endpoint family below; send rendering mode explicitly.
For paid batches, use scripts/account_summary.sh before and after collection.
Respect the user's selected scope and budget; a low price is not authorization.
Bound retries and concurrency, cache useful results, and record failures.
After a missing key or non-refusal provider failure, disclose the reason before
using an available specialized alternative. Stop on CAPTCHA, 403 or 429; do not
change tools, providers or proxies to bypass refusal. Do not submit private or
session-authenticated content to a public scraping provider.
Prompt-based /chatgpt and other external inference modes are not substitutes
for the configured subscription. Invoke them only when the user explicitly
authorizes that service and its costs; a configured key does not grant permission.
- endpoint-index.md: choose an endpoint when the family is unclear; historical costs are not current price evidence.
- core-tools.md: static/dynamic pages, account and proxy.
- google-serp.md: search, news, trends and search answers.
- local-maps.md: places, reviews and local businesses.
- social-video.md: public social data and transcripts.
- commerce-travel.md: products, travel and marketplaces.
- b2b-research.md: jobs, company data and Scholar.
Use parameter objects or URL encoding for HTTP queries, a bounded timeout and sanitized errors. Do not print arbitrary error bodies that may echo credentials. Direct known official sources and local evidence need no proxy when they already provide the requested information.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.