salesforce-help-site-scraper
Skill27 starsChanged 10 months agoMITView on GitHub
Scrape Salesforce Help articles into clean Markdown with consent handling and content cleanup. Use when you need an internal, readable snapshot of Help content for research or documentation support.
In this repository9 other files it gives its agents.
- AGENTS.mdAGENTS.md
- ai-instructions/AGENTS.mdAGENTS.md
- agent-authoringSkill
- mcp-developmentSkill
- mcp-log-debuggingSkill
- mcp-programmatic-testingSkill
- mcp-yaml-testingSkill
- salesforce-developer-site-scraperSkill
- skill-authoringSkill
The file
We split this skill into the rules it gives an agent
5 rules. About 0 are common, 4 are distinctive, and 1 is very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈4
Distinctive · estimate
This author's own choices. Worth a look.
1
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Distinctive: line 10
Use this skill to extract Salesforce Help article content into clean Markdown when pages render dynamically or are blocked by consent …
- Distinctive: line 17
NOT for: high-volume crawling, bypassing access controls, or republishing Salesforce content.
- Distinctive: line 49
Prefer the Salesforce Knowledge APIs for structured, supported access where possible.
- Very specific: line 50
Check and respect robots.txt before scraping.
- Distinctive: line 51
Do not republish or redistribute Salesforce Help content.
≈ “Common” is an estimate: at least 1 in 5 of the 350,495 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Salesforce Help Site Scraper
Use this skill to extract Salesforce Help article content into clean Markdown when pages render dynamically or are blocked by consent banners.
When to Use This Skill
- You need a readable Markdown snapshot of a Help article for internal research.
- OneTrust cookie banners block access to the main content.
- You want to remove headers, footers, or navigation chrome before extraction.
- NOT for: high-volume crawling, bypassing access controls, or republishing Salesforce content.
Prerequisites
- Node.js 18+
- Scraper script at
skills/salesforce-help-site-scraper/scripts/scrape-help-to-markdown.js
How to Use
Basic Usage
node skills/salesforce-help-site-scraper/scripts/scrape-help-to-markdown.js \
--url "https://help.salesforce.com/s/articleView?id=sf.flow.htm&type=5" \
--out "./artifacts/online-research/help_flow_overview.md" \
--consent-selector "#onetrust-accept-btn-handler" \
--remove-selectors "header,footer,nav,aside" \
--wait 2500Script Options
| Option | Required | Description |
--url | Yes | Target Help article URL. |
--out | Yes | Output Markdown file path. |
--consent-selector | No | Selector for cookie/consent accept button (OneTrust). |
--remove-selectors | No | Comma-separated selectors to remove before extraction. |
--wait | No | Milliseconds to wait after navigation or consent click. |
Compliance Notes
- Prefer the Salesforce Knowledge APIs for structured, supported access where possible.
- Check and respect
robots.txtbefore scraping. - Do not republish or redistribute Salesforce Help content.
- Attribute content to Salesforce when used internally.
Examples
Example: Capture a Flow Help article
node skills/salesforce-help-site-scraper/scripts/scrape-help-to-markdown.js \
--url "https://help.salesforce.com/s/articleView?id=sf.flow_build.htm&type=5" \
--out "./artifacts/online-research/help_flow_build.md" \
--consent-selector "#onetrust-accept-btn-handler" \
--remove-selectors "header,footer,nav,aside" \
--wait 2500Troubleshooting
Issue: Output is empty or too short
Solution: Increase --wait or refine --remove-selectors to avoid removing the main content container.
Issue: Consent banner blocks content
Solution: Provide --consent-selector for the OneTrust accept button.
References
- Salesforce Help: https://help.salesforce.com/
- Salesforce Knowledge Developer Guide: https://developer.salesforce.com/docs/atlas.en-us.knowledge_dev.meta/knowledge_dev/
- Salesforce REST API - Knowledge Resources: https://developer.salesforce.com/docs/atlas.en-us.api_rest.meta/api_rest/resources_knowledge.htm
- Salesforce Master Subscription Agreement: https://www.salesforce.com/company/legal/agreements/
- Salesforce Intellectual Property and Trademarks: https://www.salesforce.com/company/legal/intellectual/
- Robots Exclusion Protocol (RFC 9309): https://www.rfc-editor.org/rfc/rfc9309
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.