agentleFS
duotifyduotify/GitHubClawToolkit/skills/felo-web-fetch/SKILL.md

felo-web-fetch

Skill254 starsChanged 7 months agoMITView on GitHub

Extract webpage content with Felo Web Extract API. Use for turning URLs into html/markdown/text, selecting specific page areas with CSS selectors, and controlling extraction options like crawl mode, cookies, user-agent, and timeout.

  • Reads credentials
  • Installs packages
  • Sends data out

1 other project uses a version of this, 1 of them edited.

Adopted by

Most recently taken up by

The file

We split this skill into the rules it gives an agent

9 rules. About 0 are common, 6 are distinctive, and 3 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈6

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 3

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    Do NOT use this skill for:

    line 25
  2. Distinctive:

    Create and copy your API key

    line 37
  3. Distinctive:

    When the user wants a specific part of the page or a specific output format, phrase the command like this:

    line 96
  4. Distinctive:

    Default output is extracted content only (for direct use or piping).

    line 178
  5. Distinctive:

    If response content is not a string, script prints JSON.

    line 179
  6. Very specific:

    Use --json when user needs metadata and full response object.

    line 180
  7. Distinctive:

    Always require URL before running.

    line 203
  8. Very specific:

    Use --target-selector when users only want a specific part of the page.

    line 207
  9. Very specific:

    Use --request-timeout-ms for page rendering/extraction wait, and --timeout for local HTTP timeout.

    line 208

≈ “Common” is an estimate: at least 1 in 5 of the 350,495 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Felo Web Fetch Skill

When to Use

Trigger this skill when users want to extract or convert webpage content from a URL:

  • Fetch or scrape content from a webpage URL
  • Convert webpage content to html, markdown, or text
  • Extract specific blocks using CSS selector
  • Get article/main text from a link with readability mode
  • Tune extraction behavior with crawl mode (fast/fine)
  • Pass request details such as cookies, user-agent, timeout

Trigger keywords (examples):

  • fetch webpage, scrape URL, fetch page content, web fetch, url to markdown
  • Explicit: /felo-web-fetch, "use felo web fetch", "extract this URL with felo"
  • Same intent in other languages (e.g. 网页抓取, 提取网页内容) also triggers this skill

Do NOT use this skill for:

  • Real-time Q&A search summaries (use felo-search)
  • Slide generation tasks (use felo-slides)
  • Local file parsing in current workspace

Setup

1. Get API key

  1. Visit felo.ai
  2. Open Settings -> API Keys
  3. Create and copy your API key

2. Configure environment variable

Linux/macOS:

export FELO_API_KEY="your-api-key-here"

Windows PowerShell:

$env:FELO_API_KEY="your-api-key-here"

How to Execute

Option A: Use the bundled script or packaged CLI

Script (from repo):

node .agents/skills/felo-web-fetch/scripts/run_web_fetch.mjs --url "https://example.com/article" [options]

Packaged CLI (after npm install -g felo-ai): same options, with short forms allowed:

felo web-fetch -u "https://example.com" [options]
# Short forms: -u (url), -f (format), -t (timeout, seconds), -j (json)

Required parameter:

  • --url

Core optional parameters:

  • --output-format html|markdown|text
  • --crawl-mode fast|fine
  • --target-selector "article.main-content"
  • --wait-for-selector ".content-ready"

Other key optional parameters:

  • --cookie "session_id=xxx" (repeatable)
  • --set-cookies-json '[{"name":"sid","value":"xxx","domain":"example.com"}]'
  • --user-agent "Mozilla/5.0 ..."
  • --timeout 60 (HTTP request timeout in seconds)
  • --request-timeout-ms 15000 (API payload timeout in ms)
  • --with-readability true
  • --with-links-summary true
  • --with-images-summary true
  • --with-images-readability true
  • --with-images true
  • --with-links true
  • --ignore-empty-text-image true
  • --with-cache false
  • --with-stypes true
  • --json (print full JSON response)

How to write instructions (target_selector + output_format)

When the user wants a specific part of the page or a specific output format, phrase the command like this:

  • Output format: "Fetch as text" / "Get markdown" / "Return html" → use --output-format text, --output-format markdown, or --output-format html.
  • Target one element: "Only the main article" / "Just the content inside #main" / "Fetch only article.main-content" → use --target-selector "article.main" or the selector they give.

Examples:

# Basic: fetch as Markdown
node .agents/skills/felo-web-fetch/scripts/run_web_fetch.mjs --url "https://example.com" --output-format markdown

# Article-style with readability
node .agents/skills/felo-web-fetch/scripts/run_web_fetch.mjs --url "https://example.com/article" --with-readability true --output-format markdown

# Only the element matching a CSS selector
node .agents/skills/felo-web-fetch/scripts/run_web_fetch.mjs --url "https://example.com" --target-selector "article.main" --output-format markdown

# With cookies and custom user-agent
node .agents/skills/felo-web-fetch/scripts/run_web_fetch.mjs --url "https://example.com/private" --cookie "session_id=abc123" --with-readability true --json

# Full JSON response
node .agents/skills/felo-web-fetch/scripts/run_web_fetch.mjs --url "https://example.com" --output-format text --json

Option B: Call API with curl

curl -X POST "https://openapi.felo.ai/v2/web/extract" \
  -H "Authorization: Bearer $FELO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com", "output_format": "markdown", "with_readability": true}'

API Reference (summary)

  • Endpoint: POST /v2/web/extract
  • Base URL: https://openapi.felo.ai. Override with FELO_API_BASE env if needed.
  • Auth: Authorization: Bearer YOUR_API_KEY

Request body (JSON)

ParameterTypeRequiredDefaultDescription
urlstringYes-Webpage URL to fetch
crawl_modestringNofastfast or fine
output_formatstringNohtmlhtml, text, markdown
with_readabilitybooleanNo-Use readability (main content)
with_links_summarybooleanNo-Include links summary
with_images_summarybooleanNo-Include images summary
target_selectorstringNo-CSS selector for target element
wait_for_selectorstringNo-Wait for selector before fetch
timeoutintegerNo-Timeout in milliseconds
with_cachebooleanNotrueUse cache
set_cookiesarrayNo-Cookie entries
user_agentstringNo-Custom user-agent

Response

Success (200):

{
  "code": 0,
  "message": "success",
  "data": {
    "content": { ... }
  }
}

Fetched content is in data.content; structure depends on output_format.

Error codes

HTTPCodeDescription
400-Parameter validation failed
401INVALID_API_KEYAPI key invalid or revoked
500/502WEB_EXTRACT_FAILEDFetch failed (server or page error)

Output Format

  • Default output is extracted content only (for direct use or piping).
  • If response content is not a string, script prints JSON.
  • Use --json when user needs metadata and full response object.

Language

  • 預設輸出語言為繁體中文(zh-TW)。除非使用者明確指定其他語言,否則所有摘要、整理、說明等 AI 產出的文字,一律使用繁體中文輸出。
  • 網頁原始內容(如簡體中文頁面)維持原文擷取,但 AI 的整理、摘要與說明部分仍須使用繁體中文。

Error response format:

## Web Fetch Failed
- Message: <error message>
- Suggested Action: verify URL/parameters and retry

Important Notes

  • ⚠️ 路徑安全:skill 腳本位於 repo root 的 .agents/skills/ 下。若 cwd 不在 repo root,先執行 git rev-parse --show-toplevel 取得路徑再 cd。禁止用 $(...) 語法。
  • ⚠️ 指令安全:Copilot CLI 的安全過濾器會封鎖含有特定 shell 語法的指令,執行時禁止加入以下任何寫法,否則指令會被直接擋下:
    • set -e、set -u、set -o、set -euo pipefail 等 set 指令
    • ${var@P}、${!var} 等參數展開語法
    • 巢狀的 $(...) 指令替換
    • 正確寫法:直接執行 cd 與 node 指令,不要包在 shell 腳本控制流程內
  • Always require URL before running.
  • Validate enum values:
    • output_format: html, markdown, text
    • crawl_mode: fast, fine
  • Use --target-selector when users only want a specific part of the page.
  • Use --request-timeout-ms for page rendering/extraction wait, and --timeout for local HTTP timeout.
  • For long articles or slow sites, consider increasing --timeout.
  • API may cache results; use --with-cache false only when fresh content is required.

References

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.