wiki-research-social
Skill0 starsChanged 4 months agoMITView on GitHub
Fetch Twitter/X threads, Hacker News discussions, or Reddit posts into raw/social/. Invoked by wiki-research for social/community content.
- Installs packages
In this repository36 other files it gives its agents.
- AGENTS.mdAGENTS.md
- CLAUDE.mdCLAUDE.md
- wiki-extract-annasSkill
- wiki-extract-crunchbaseSkill
- wiki-extract-ebookSkill
- wiki-extract-ecommerceSkill
- wiki-extract-githubSkill
- wiki-extract-linkedinSkill
- wiki-extract-newsletterSkill
- wiki-extract-patentsSkill
- wiki-extract-paywallSkill
- wiki-extract-podcastSkill
- wiki-extractSkill
- wiki-extract-wikipediaSkill
- wiki-extract-youtubeSkill
- wiki-fetchSkill
- wiki-ingestSkill
- wiki-learnSkill
- wiki-lintSkill
- wiki-maintainerSkill
- wiki-onboardSkill
- wiki-pipelineSkill
- wiki-querySkill
- wiki-raw-prepareSkill
- wiki-research-academicSkill
- wiki-research-deepSkill
- wiki-research-feedsSkill
- wiki-research-loopSkill
- wiki-research-newsSkill
- wiki-researchSkill
- wiki-research-webSkill
- wiki-retroSkill
- wiki-session-memorySkill
- wiki-setupSkill
- wiki-statusSkill
- wiki-upgradeSkill
The file
We split this skill into the rules it gives an agent
5 rules. About 0 are common, 2 are distinctive, and 3 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈2
Distinctive · estimate
This author's own choices. Worth a look.
3
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Distinctive: line 137
Before writing each file, check for existing ingests of the same post:
- Very specific: line 143
If found and the existing file is recent, skip and log [SKIP] duplicate: raw/social/....
- Very specific: line 149
Run each file through skills/wiki-research/references/source-eval.md.
- Distinctive: line 158
source-eval: run; orchestrator post-process handles wiki-ingest.
- Very specific: line 182
CLI: Run llm-wiki integrations status and any llm-wiki line from Step 1 of this skill (from the vault root).
≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Wiki research — social
Fetches social and community content (Twitter/X, Hacker News, Reddit) into raw/social/. Use this sub-skill when the input is a social URL or the user wants community discussion around a topic.
Step 1 — Identify the platform
| Input signal | Platform | Command |
x.com, twitter.com URL or "tweet"/"thread" | Twitter/X | llm-wiki ingest twitter |
news.ycombinator.com URL, HN item ID, "HN thread", "Show HN" | Hacker News | llm-wiki ingest hackernews |
reddit.com URL, "subreddit", "Reddit post" | Firecrawl on old.reddit.com | |
| Keyword search ("what are people saying about X") | Multiple | Run HN search + Twitter search |
Step 2 — Twitter/X
Single tweet (zero-config, public):
llm-wiki ingest twitter https://x.com/<user>/status/<id> \
--out social/twitter/<YYYY-MM-DD>/<user>-<id>.mdFull thread (requires bird CLI):
llm-wiki ingest twitter https://x.com/<user>/status/<id> --thread \
--out social/twitter/<YYYY-MM-DD>/<user>-thread-<id>.mdKeyword search (requires bird CLI + TWITTER_AUTH_TOKEN):
llm-wiki ingest twitter --search "<query>" --limit 20 \
--out social/twitter/<YYYY-MM-DD>/search-<slug>.md
# Good query patterns:
# "AI agents" lang:en -is:retweet since:2026-01-01
# from:<handle> filter:linksUser timeline (requires bird CLI + TWITTER_AUTH_TOKEN):
llm-wiki ingest twitter --user @<handle> --limit 50 \
--out social/twitter/<YYYY-MM-DD>/<handle>-timeline.mdRequired frontmatter:
source_type: social
platform: twitter
post_id: "<tweet-id>"
author: "@handle"
posted_date: YYYY-MM-DD
thread: true | falseStep 3 — Hacker News
Thread by URL:
llm-wiki ingest hackernews https://news.ycombinator.com/item?id=<ID> \
--out social/hn/<YYYY-MM-DD>/<id>-<slug>.mdThread by item ID:
llm-wiki ingest hackernews <ID> \
--out social/hn/<YYYY-MM-DD>/<id>.mdSearch for HN threads on a topic (Algolia API):
curl "https://hn.algolia.com/api/v1/search?query=<QUERY>&tags=story&hitsPerPage=5" \
| python3 -c "import json,sys; [print(h['url'],h['objectID']) for h in json.load(sys.stdin)['hits']]"
# Then fetch each thread ID individuallyRequired frontmatter:
source_type: social
platform: hackernews
post_id: "<hn-item-id>"
hn_url: https://news.ycombinator.com/item?id=<id>
posted_date: YYYY-MM-DD
score: <points> # if availableSee skills/wiki-research-loop/references/hn-example.md for HN API ethics and rate limits.
Step 4 — Reddit
Reddit does not require an API key for public posts via old.reddit.com.
Single post with comments:
# Plain-text JSON (no auth needed)
curl "https://old.reddit.com/r/<subreddit>/comments/<id>/<slug>.json?limit=100" \
-H "User-Agent: wiki-llm-research/1.0" \
-o /tmp/reddit-<id>.json
# Or fetch readable HTML via Firecrawl:
llm-wiki ingest firecrawl "https://old.reddit.com/r/<subreddit>/comments/<id>/<slug>/" \
--out social/reddit/<YYYY-MM-DD>/r-<subreddit>-<id>.mdTop posts from a subreddit:
curl "https://old.reddit.com/r/<subreddit>/top.json?t=week&limit=10" \
-H "User-Agent: wiki-llm-research/1.0" | python3 -c \
"import json,sys; [print(p['data']['url']) for p in json.load(sys.stdin)['data']['children']]"Required frontmatter:
source_type: social
platform: reddit
subreddit: "<name>"
post_id: "<reddit-id>"
reddit_url: https://old.reddit.com/r/<subreddit>/comments/<id>/
posted_date: YYYY-MM-DDStep 5 — Deduplication
Before writing each file, check for existing ingests of the same post:
grep -r "post_id: \"<id>\"" raw/social/If found and the existing file is recent, skip and log [SKIP] duplicate: raw/social/....
Step 6 — Evaluate and return to orchestrator
Run each file through skills/wiki-research/references/source-eval.md. Social content often scores lower on authority (tier 3) — this is expected; adjust expectations accordingly.
Then return to wiki-research Step 3 (post-process).
Done looks like
- Platform identified; content under
raw/social/(or agreed path); dedup applied. - source-eval run; orchestrator post-process handles wiki-ingest.
Troubleshooting
| Symptom | Fix |
| fxTwitter returns empty | Tweet may be deleted; try Wayback Machine |
| bird CLI not found | npm install -g bird-cli or use llm-wiki integrations wizard |
TWITTER_AUTH_TOKEN missing | Sign in to X, copy auth token from browser devtools (Application → Cookies) |
| HN item 404 | Check item ID; item may have been flagged/deleted |
| Reddit 429 | Add User-Agent header; wait 10s between requests |
| Reddit post is [removed] | Note as [REMOVED] in frontmatter; do not ingest placeholder text |
Related skills
- wiki-research — orchestrator that invoked this skill
- wiki-research-web — for general URLs discovered in social posts
- wiki-research-academic — for papers linked in social threads
- wiki-research-loop — batch HN/social monitoring via research-tasks.json
Smoke check
- CLI: Run
llm-wiki integrations statusand anyllm-wikiline from Step 1 of this skill (from the vault root). - Prompt: Invoke this skill by name in Claude Code; complete Step 1 only and confirm expected CLI or file output.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

