agentleFS
Sign inStart free
GabrieleDattileGabrieleDattile/agent-superpowers/skills/scraping/firecrawl-scrape/SKILL.md

firecrawl-build-scrape

Skill1 starsChanged 15 days agoMITView on GitHub

Integrate Firecrawl `/scrape` into product code for single-page extraction.

  • Reads credentials

8 other projects use a version of this, 8 of them edited.

Adopted by

Most recently taken up by

Used with

Projects with it also use firecrawl-interact, firecrawl-agent and 6 more.

Other files people use alongside this one.

Files found in the same public projects as a version of this one, more often than they turn up anywhere else. Each project counts once, large collections count for less, and this file's own repository is left out.
  • RudraDudhat2509Skill

    firecrawl-interact

    Control and interact with a live browser session on any scraped page — click buttons, fill forms, navigate flows, and extract data using natural language prompts or code. Use when the user needs to interact with a webpage beyond simple scraping: logging into a site, submitting forms, clicking through pagination, handling infinite scroll, navigating multi-step checkout or wizard flows, or when a regular scrape failed because content is behind JavaScript interaction. Also useful for authenticated scraping via profiles. Triggers on "interact", "click", "fill out the form", "log in to", "sign in", "submit", "paginated", "next page", "infinite scroll", "interact with the page", "navigate to", "open a session", or "scrape failed".

    Also in 3 projects that use this

  • get-proofpilotSkill

    firecrawl-agent

    AI-powered autonomous data extraction that navigates complex sites and returns structured JSON. Use this skill when the user wants structured data from websites, needs to extract pricing tiers, product listings, directory entries, or any data as JSON with a schema. Triggers on "extract structured data", "get all the products", "pull pricing info", "extract as JSON", or when the user provides a JSON schema for website data. More powerful than simple scraping for multi-page structured extraction.

    Also in 3 projects that use this

  • get-proofpilotSkill

    firecrawl-crawl

    Bulk extract content from an entire website or site section. Use this skill when the user wants to crawl a site, extract all pages from a docs section, bulk-scrape multiple pages following links, or says "crawl", "get all the pages", "extract everything under /docs", "bulk extract", or needs content from many pages on the same site. Handles depth limits, path filtering, and concurrent extraction.

    Also in 3 projects that use this

  • parcadeiSkill

    firecrawl-scrape

    Scrape web pages and extract content via Firecrawl MCP

    Also in 3 projects that use this

  • LuisSambranoSkill

    firecrawl

    Deep web scraping, screenshots, PDF parsing, and website crawling using Firecrawl API

    Also in 3 projects that use this

  • +3 more from get-proofpilot

The file

We split this skill into the rules it gives an agent

9 rules. 6 are common, 0 are distinctive, and 3 are very specific to this project.

  • 6

    Common

    Most of the 8 edited versions we read keep these.

  • 0

    Distinctive

    This author's own choices. Worth a look.

  • 3

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Common:

    Use this when the application already has the URL and needs content from one page.

    line 41
  2. Very specific:

    Use onlyMainContent for article-like pages where nav and chrome add noise.

    line 52
  3. Common:

    Add waits or other rendering options only when the page needs them.

    line 53
  4. Very specific:

    Read metadata.cacheState and metadata.cachedAt to see what you actually got.

    line 58
  5. Common:

    If you do not have the URL yet, start with firecrawl-build-search.

    line 71
  6. Common:

    If content requires clicks, typing, or multi-step navigation, escalate to firecrawl-build-interact.

    line 72
  7. Common:

    Keep the integration narrow: one feature, one URL, one extraction contract.

    line 76
  8. Very specific:

    Treat /scrape as the default primitive for downstream LLM or indexing pipelines.

    line 77
  9. Common:

    Read the source-of-truth page for your project language before writing integration code:

    line 82

“Common” is measured: kept by at least 6 in 10 of the 8 edited versions of this file we could read. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Firecrawl Build Scrape

Use this when the application already has the URL and needs content from one page.

Use This When

  • the feature starts from a known URL
  • you need page content for retrieval, summarization, enrichment, or monitoring
  • you want the default extraction primitive before considering /interact

Default Recommendations

  • Return markdown unless the feature truly needs another format.
  • Use onlyMainContent for article-like pages where nav and chrome add noise.
  • Add waits or other rendering options only when the page needs them.

Freshness and Liveness

  • Firecrawl reuses recently indexed content, which is what makes repeat reads of the same URL fast. Set maxAge (milliseconds) to bound how old a reused copy may be, or maxAge: 0 to skip index reuse for a freshness-critical read.
  • Read metadata.cacheState and metadata.cachedAt to see what you actually got.
  • A successful scrape reports what the page returned. Whether the thing the page describes is still active is a source-specific judgment your code makes.
  • See references/freshness-and-liveness.md for the tradeoff, the metadata, and the decision rule.

Common Product Patterns

  • knowledge ingestion from known URLs
  • enrichment from a company, product, or docs page
  • pricing, changelog, and documentation extraction
  • page-level quality checks or monitoring

Escalation Rules

  • If you do not have the URL yet, start with firecrawl-build-search.
  • If content requires clicks, typing, or multi-step navigation, escalate to firecrawl-build-interact.

Implementation Notes

  • Keep the integration narrow: one feature, one URL, one extraction contract.
  • Treat /scrape as the default primitive for downstream LLM or indexing pipelines.
  • Request richer formats only when the consumer needs them, such as links, screenshots, or branding data.

Docs (Source of Truth)

Read the source-of-truth page for your project language before writing integration code:

See Also

  • firecrawl-build
  • firecrawl-build-search
  • firecrawl-build-interact

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

Check your setup, free