dsh-web-crawler
Skill79 starsChanged 13 days agoMITView on GitHub
网页与后台数据自动化抓取技能。支持公开网页抓取与基于无障碍/登录态的应用数据提取,自动清洗为结构化 JSON/Markdown,规避反爬与繁琐手动复制。
In this repository26 other files it gives its agents.
- doc-tidySkill
- dsh-backupSkill
- dsh-claude-styleSkill
- dsh-code-craftSkill
- dsh-crewSkill
- dsh-data-craftSkill
- dsh-deep-researchSkill
- dsh-diagramSkill
- dsh-doc-publisherSkill
- dsh-doc-searchSkill
- dsh-doc-tidySkill
- dsh-excel-copilotSkill
- dsh-flashgouSkill
- dsh-lifeSkill
- dsh-mail-craftSkill
- dsh-meeting-copilotSkill
- dsh-memory-syncSkill
- dsh-mobileSkill
- dsh-reviewSkill
- dsh-scrum-masterSkill
- dsh-slidesSkill
- dsh-storyboardSkill
- dsh-tierSkill
- dsh-videoSkill
- dsh-voiceSkill
- dsh-write-craftSkill
The file
Files
网页与后台数据自动化抓取技能 (browser-skill)
一句话:告别手动复制粘贴,把复杂的网页与后台表格一键转为整洁的 Markdown / JSON。
⚠️ 2026-11-05 起 3081/3181/8998/8999 本机桥都校验共享密钥(
X-DSH-Token)。 bash 里没有这个环境变量:用sh /sdcard/DeepSeekHarness/tools/curl.sh代替裸curl(自动带密钥, 只对环回地址生效,外网不会泄漏);或者TOKEN=$(sh /sdcard/DeepSeekHarness/tools/bridge-token.sh)再自己加-H "X-DSH-Token: $TOKEN"。dsh-tool-android/dsh-tool-shizuku/dsh-tool-accessibility/dsh-tool-nfc/dsh-native-command等插件已自动带上,插件调用无需关心。
🎯 两种抓取通道(按优先级选择)
- 公开网页通道(免特权、零开销、极速):
- 使用
web_fetch工具直连拉取页面 HTML 并解码; - 配合内置脚本进行 DOM 清洗,去除导航栏、广告和冗余样式,只提取纯净正文与表格。
- 使用
- 带登录态/动态渲染通道(手机端无障碍原生桥):
- 目标需要登录(如个人后台、订单列表、管理控制台):
- 引导用户在手机浏览器中打开对应页面;
- 通过
android_screen/ 无障碍桥 (http://127.0.0.1:3181/dump) 结构化读取页面文本与表格节点; - 自动聚合分页内容,拼装为完整的报表交付。
🛠️ 数据结构化提取原则
- 长列表转 Markdown 表格:自动识别标题行与数据列,生成标准 GFM 表格;
- 文章/文档转清晰段落:保留关键标题分级 (
#,##)、引用块与要点列表; - 严禁乱抓无关噪声:自动剔除 Cookie 声明、页脚版权、无关侧边栏链接。
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

