agentleFS
Sign inStart free
yohei1126yohei1126/agent-skills/download-website-to-markdown/SKILL.md

download-website-to-markdown

Skill2 starsChanged 22 days agoMITView on GitHub

Downloads a website URL and saves its content as a local markdown file. Uses a headless browser (Playwright) to handle JS-rendered pages. Also includes a PDF-to-Markdown converter using Docling. Use when the user wants to archive a webpage or PDF, save documentation locally, or convert web/PDF content to markdown for offline reading.

The file

We split this skill into the rules it gives an agent

5 rules. About 0 are common, 4 are distinctive, and 1 is very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈4

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 1

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Very specific:

    Run the download script from the agent_skills/ directory.

    line 13
  2. Distinctive:

    If the URL is unreachable or times out (30 s), the script exits with an error and does not create a file.

    line 53
  3. Distinctive:

    If the derived filename already exists, a numeric suffix is appended rather than overwriting silently.

    line 55
  4. Distinctive:

    If a URL is given, the PDF is downloaded to a temporary file.

    line 83
  5. Distinctive:

    If the output path already exists, a numeric suffix is appended rather than overwriting silently.

    line 111

≈ “Common” is an estimate: at least 1 in 5 of the 350,495 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Instructions

Run the download script from the agent_skills/ directory.

Usage

cd agent_skills
npx tsx download-website-to-markdown/scripts/download.ts <url> [output]
ArgumentRequiredDescription
urlYesThe URL to download
outputNoOutput file path. Derived from the URL if omitted.

Steps

  1. Run the script with the URL and optional output path.
  2. The script fetches the page using a headless Chromium browser (waits for network idle to handle JS-rendered content).
  3. HTML is converted to Markdown using Turndown. Navigation, footers, scripts, and styles are stripped automatically.
  4. A YAML front matter block is prepended:
    ---
source: <original URL>
fetched: <YYYY-MM-DD>
---
  5. The file is saved. If the output path already exists, a numeric suffix is appended (e.g. page-2.md).

Examples

# Derive filename from URL → saves to: ai-observability.md
npx tsx download-website-to-markdown/scripts/download.ts https://docs.snowflake.com/en/user-guide/snowflake-cortex/ai-observability

# Save to a specific path
npx tsx download-website-to-markdown/scripts/download.ts https://example.com outputs/reference-docs/example.md

Edge cases

  • If the URL is unreachable or times out (30 s), the script exits with an error and does not create a file.
  • Query strings and fragments are ignored when deriving the output filename.
  • If the derived filename already exists, a numeric suffix is appended rather than overwriting silently.

PDF to Markdown (pdf-to-markdown.py)

Converts a remote or local PDF to Markdown using Docling, which preserves document structure (headings, tables, lists, code blocks).

Requirements

  • Python 3.10+
  • uv — dependencies (docling, requests) are declared inline and installed automatically on first run.

Usage

cd agent_skills
uv run download-website-to-markdown/scripts/pdf-to-markdown.py <pdf-url-or-path> [output]
ArgumentRequiredDescription
pdf-url-or-pathYesRemote URL (https://…) or local file path to a PDF
outputNoOutput .md path or directory. Derived from the PDF filename if omitted.

Steps

  1. Run the script with the PDF URL or local path and an optional output path.
  2. If a URL is given, the PDF is downloaded to a temporary file.
  3. Docling converts the PDF to Markdown, preserving structure (headings, tables, code blocks).
  4. A YAML front matter block is prepended:
    ---
source: <original URL or path>
fetched: <YYYY-MM-DD>
---
  5. The file is saved. If the output path already exists, a numeric suffix is appended (e.g. paper-2.md).

Examples

# Remote PDF — filename derived from URL → saves to: 2310.06825.md
uv run download-website-to-markdown/scripts/pdf-to-markdown.py https://arxiv.org/pdf/2310.06825

# Save to a specific path
uv run download-website-to-markdown/scripts/pdf-to-markdown.py https://example.com/spec.pdf outputs/spec.md

# Save into a directory (filename derived automatically)
uv run download-website-to-markdown/scripts/pdf-to-markdown.py report.pdf outputs/

Edge cases

  • Remote PDFs are downloaded to a temp file and deleted after conversion.
  • .pdf is stripped from the derived filename so the output is paper.md, not paper.pdf.md.
  • If the output path already exists, a numeric suffix is appended rather than overwriting silently.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

Check your setup, free