agentleFS
Sign inStart free
OpenMinisOpenMinis/MinisSkills/web-content-extractor/SKILL.md

web-content-extractor

Skill584 starsChanged 59 days agoMITView on GitHub

Clean and extract the main body content from a webpage URL. Use this skill whenever the user asks to extract, scrape, or read the main article, text, or content from a webpage, especially if they mention wanting clean Markdown or text without ads and navigation.

We split this skill into the rules it gives an agent

8 rules. About 0 are common, 4 are distinctive, and 4 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈4

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 4

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    Use this as the primary method.

    line 19
  2. Distinctive:

    Use this if Defuddle fails or returns an error.

    line 24
  3. Very specific:

    Fetch the content: Use the shell_execute tool with curl -sL "FETCH_URL" to download the content.

    line 30
  4. Very specific:

    Handle Fallbacks: If the curl command fails, returns empty, or returns an error message indicating failure, try the Jina AI service …

    line 32
  5. Distinctive:

    If the user asked you to read it to answer a question, use the content to answer.

    line 34
  6. Distinctive:

    If the user asked you to extract or save it, present the Markdown to them or save it to a file as requested.

    line 35
  7. Very specific:

    Always enclose the URL in quotes in the curl command to prevent shell interpretation of special characters like & or ?

    line 39
  8. Very specific:

    If the target URL is missing http:// or https://, prepend https:// before appending it to the service URL.

    line 40

≈ “Common” is an estimate: at least 1 in 5 of the 308,232 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Web Content Extractor

This skill helps you extract the clean main content (body text/Markdown) from a webpage URL by using Defuddle or Jina AI's reader API.

How it works

To extract the content of a target URL, you will prepend a specific service URL to the target URL and fetch it. This converts the messy webpage into clean Markdown containing only the main content.

Available Services

  1. Defuddle (Default)
    • Format: https://defuddle.md/<target-url>
    • Example: https://defuddle.md/https://example.com/article
    • Use this as the primary method.
  2. Jina AI Reader (Fallback)
    • Format: https://r.jina.ai/<target-url>
    • Example: https://r.jina.ai/https://example.com/article
    • Use this if Defuddle fails or returns an error.

Execution Steps

  1. Identify the target URL: Extract the full URL the user wants to read from their request. Ensure it includes the protocol (e.g., https://).
  2. Construct the fetch URL: Prepend https://defuddle.md/ to the target URL.
  3. Fetch the content: Use the shell_execute tool with curl -sL "FETCH_URL" to download the content.
    • Example command: curl -sL "https://defuddle.md/https://example.com/article"
  4. Handle Fallbacks: If the curl command fails, returns empty, or returns an error message indicating failure, try the Jina AI service instead: curl -sL "https://r.jina.ai/https://example.com/article"
  5. Process the output: The output will be in Markdown format.
    • If the user asked you to read it to answer a question, use the content to answer.
    • If the user asked you to extract or save it, present the Markdown to them or save it to a file as requested.

Notes

  • Always enclose the URL in quotes in the curl command to prevent shell interpretation of special characters like & or ?.
  • If the target URL is missing http:// or https://, prepend https:// before appending it to the service URL.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

Reviews can't be read right now.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

Check your setup, free