web-fetch
Skill1 starsChanged 12 days agoMITView on GitHub
Fetch the content of a URL (HTML page, text, JSON) when direct access is blocked or unstable, with automatic fallback to a residential proxy. Use when WebFetch fails (403, 429, 503, timeout), or to save a page's text to a file without going through the model.
- Reads credentials
13 other projects use a version of this.
Adopted by
Most recently taken up by
- tonyfettes/skillstheir own version, first seen Nov 8, 2026
- Mathews-Tom/armorytheir own version, first seen Nov 1, 2026
- runxhq/runxtheir own version, first seen Oct 13, 2026
- superbereza/lobster-powerstheir own version, first seen Oct 11, 2026
- jholhewres/devclaw-skillstheir own version, first seen Oct 9, 2026
- Lingyuzhou111/dow-ipad-859their own version, first seen Oct 8, 2026
Used with
Projects with it also use github and skill-creator.
Other files people use alongside this one.
Files found in the same public projects as a version of this one, more often than they turn up anywhere else. Each project counts once, large collections count for less, and this file's own repository is left out.
NVIDIASkillgithub
Interact with GitHub using the gh CLI restricted to REST API only (no GraphQL). Use when the user wants to work with GitHub issues, pull requests, repos, releases, or Actions — especially in sandboxed environments where the GraphQL endpoint may be blocked. Trigger keywords - github, gh, pull request, PR, github issue, github actions, workflow, release, gh api.
Also in 3 projects that use this
SentrySkillskill-creator
Create new agent skills following the Agent Skills specification. Use when asked to "create a skill", "add a new skill", "write a skill", "make a skill", "build a skill", or scaffold a new skill with SKILL.md. Guides through requirements, writing, registration, and verification.
Also in 4 projects that use this
Other versions13 projects use a version of this.
Word for word · 0
None found.
Edited · 0
None found.
Wrote their own version · 13
- Mathews-Tom/armory★ 340same name
- Lingyuzhou111/dow-ipad-859★ 260same name
- runxhq/runx★ 97same name
- archibate/agent-skills★ 16same name
- code-yeongyu/web-fetch★ 4same name
- jholhewres/devclaw-skills★ 4same name
- smart-lty/my-skills★ 4same name
- seven-steven/skills★ 1same name
- Throokie/claude-code-skills★ 1same name
- tonyfettes/skills★ 1same name
3 more not listed
How matches are found. Word for word: the exact same text. Edited: shares passages with this file (“99% same” means 99% of its five-word phrases also appear here; 10% or more counts). Their own version: similar meaning, or, weakest, the same name. We can’t tell which project had it first. Projects are counted among the 200 most-starred candidates, and mentions are links to the repository we found, up to 50 per site, so both counts are a floor.
In this repository12 other files it gives its agents.
- kit/CLAUDE.mdCLAUDE.md
- apple-designSkill
- geo-seoSkill
- light-researchSkill
- mac-controlSkill
- media-downloadSkill
- promo-codesSkill
- repo-showcaseSkill
- site-designSkill
- skill-forgeSkill
- unslopSkill
- video-transcriptSkill
The file
Files
web-fetch
A script fetches the page: direct attempt, then the residential proxy only if direct access is blocked. Your role: run it, read the JSON, decide what follows. The script bypasses neither a login nor a subscription.
Command
python3 ~/.claude/skills/web-fetch/scripts/fetch_url.py "<url>" [--text] [--out file]
--text: readable text of the page (without script or style); without it, raw HTML.--out: writes the body to a file; the JSON then contains only the path (useful for a large page).--no-proxy: never a proxy.--force-proxy: directly through the proxy (rare, see budget).- JSON output:
url,status,via(directorproxy),content_type,bytes,textorfile.
Exit codes
| Code | Meaning | Action |
| 0 | Success | Read text or file. If via is proxy, tell the user. |
| 4 | Blocked, even with the proxy or no proxy configured | Say that the site refuses access. Do not insist; offer the driven browser (skill mac-control) if the user agrees. |
| 6 | HTTP error other than a block (404, 500...) | Check the URL; the proxy would not change anything. |
| 7 | Network unreachable | Check the connection or the domain name. |
| 2 | Invalid URL | Ask for an http(s) URL. |
| 1 | Unexpected error | Repair the script (see below). |
Proxy
Address in local.env at the root of claude-tools (key IPROYAL_RESIDENTIAL_URL, ignored by git). Limited budget
(2 GB): do not download large files through this channel. Never display the proxy value.
Repair
Fix <claude-tools>/skills/web-fetch/scripts/fetch_url.py, then run python3 -m unittest discover -s tests
in the skill's folder. Two attempts at most, then report the exact message to the user.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.