scrape
Skill0 starsChanged 3 months agoMITView on GitHub
Finds new job postings matching your profile via installed portal-search CLIs (LinkedIn, local job boards, and any skills added with /add-portal). Deduplicates across runs. Triggers on: job scrape, find jobs, search jobs, new jobs, job search, scrape jobs, /scrape
36 other projects use a version of this, 5 of them edited.
Adopted by
Most recently taken up by
- Ivan-propio/claude-skillstheir own version, first seen Nov 16, 2026
- Mateus-Nogueira-GT/ai-job-search-brasiledited, first seen Nov 16, 2026
- alexcarney460-hue/walkietheir own version, first seen Nov 15, 2026
- 5uck1ess/devkittheir own version, first seen Nov 15, 2026
- Fan-Pier-Labs/openrecordtheir own version, first seen Nov 15, 2026
- MadsLorentzen/ai-job-searchedited, first seen Nov 9, 2026
Used with
Projects with it also use ios-sync, setup-gbrain and 8 more.
Other files people use alongside this one.
Files found in the same public projects as a version of this one, more often than they turn up anywhere else. Each project counts once, large collections count for less, and this file's own repository is left out.
Saqlain-811Skillios-sync
Regenerate the iOS debug bridge against the latest upstream gstack templates. (gstack)
Also in 6 projects that use this
ShreyRaviSkillsetup-gbrain
Set up gbrain for this coding agent: install the CLI, initialize a local PGLite or Supabase brain, register MCP, capture per-remote trust policy. One command from zero to "gbrain is running, and this agent can call it." Use when: "setup gbrain", "connect gbrain", "start gbrain", "install gbrain", "configure gbrain for this machine". (gstack)
Also in 8 projects that use this
ShreyRaviSkillplan-tune
Self-tuning question sensitivity + developer psychographic for gstack (v1: observational). Review which AskUserQuestion prompts fire across gstack skills, set per-question preferences (never-ask / always-ask / ask-only-for-one-way), inspect the dual-track profile (what you declared vs what your behavior suggests), and enable/disable question tuning. Conversational interface — no CLI syntax required.
Also in 10 projects that use this
Saqlain-811Skillios-clean
Remove the DebugBridge SPM package and all #if DEBUG wiring from an iOS app. (gstack)
Also in 5 projects that use this
timurgaleevSkilldocument-generate
Generate missing docs for a feature, module or project as tutorials, how-tos, reference and explanation.
Also in 7 projects that use this
bastosSkillios-design-review
Review iOS app UI/UX design quality using Apple's design principles and the Liquid Glass-era design language from WWDC25/26. Use when asked for a design review, UX critique, visual design assessment, or evaluation of an app's interface quality, design intent, alignment with the new Apple design system, or usability. Triggers: "design review", "UX review", "visual design critique", "review the UI", "does this design work", "evaluate this interface", "design feedback", "Liquid Glass review".
Also in 5 projects that use this
- +3 more from ShreyRavi
- +1 more from Saqlain-811
Other versions36 projects use a version of this.
Word for word · 0
None found.
Edited · 5
- ai-job-search-dev/ai-job-search★ 72k34% same
- MadsLorentzen/ai-job-search★ 72k34% same
- JiemsLBJ/offer-harvester★ 466% same
- Mateus-Nogueira-GT/ai-job-search-brasil★ 247% same
- arafat1023/job-application-assistant★ 077% same
Wrote their own version · 31
- garrytan/gstack★ 142ksame name
- davekilleen/Dex★ 509same name
- brightdata/skills★ 270same name
- Fan-Pier-Labs/openrecord★ 75same name
- zytedata/claude-skills★ 33same name
- timurgaleev/vibestack★ 8same name
- aranculleton/last-job-search★ 6same name
- 5uck1ess/devkit★ 4same name
- scrapesmith/scrape-skill★ 4same name
- GTMify/claude-skills-starter★ 2same name
21 more not listed
How matches are found. Word for word: the exact same text. Edited: shares passages with this file (“99% same” means 99% of its five-word phrases also appear here; 10% or more counts). Their own version: similar meaning, or, weakest, the same name. We can’t tell which project had it first. Projects are counted among the 200 most-starred candidates, and mentions are links to the repository we found, up to 50 per site, so both counts are a floor.
In this repository9 other files it gives its agents.
- AGENTS.mdAGENTS.md
- CLAUDE.mdCLAUDE.md
- czyjesteldorado-searchSkill
- freehire-searchSkill
- justjoinit-searchSkill
- linkedin-searchSkill
- nofluffjobs-searchSkill
- job-application-assistantSkill
- upskillSkill
The file
Show its rules and text
We split this skill into the rules it gives an agent
23 rules. 0 are common, 7 are distinctive, and 16 are very specific to this project.
0
Common
Most of the 29 edited versions we read keep these.
7
Distinctive
This author's own choices. Worth a look.
16
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Very specific: line 41
Read data/jobs.json (create if missing - start with {"seen": {}})
- Very specific: line 43
Read search-queries.md (this directory) for the search strategy
- Very specific: line 47
Read search-queries.md (this directory) for the search strategy.
- Distinctive: line 57
If this fails (bun not installed), skip to 1c (WebSearch fallback) for all portals and note the fallback in the Step 5 output.
- Very specific: line 67
Read its SKILL.md to find the correct bun run … invocation and supported flags.
- Very specific: line 71
Use --format json for machine-readable output.
- Very specific: line 73
Run all portal CLI calls in parallel where possible using the Agent tool.
- Distinctive: line 75
If a CLI tool exits with a non-zero code, log the error message and continue — do not abort the whole search.
- Distinctive: line 82
When bun is unavailable (Step 1a failed)
- Very specific: line 84
Use the site-specific query strings from search-queries.md directly as WebSearch queries for these portals.
- Very specific: line 90
From CLI results: Search output already includes title, company, location, date, and URL.
- Very specific: line 95
From WebSearch results: Use WebFetch on the posting URL and extract the same fields manually.
- Very specific: line 104
Every posting a detail or WebFetch call returned in full gets written to documents/postings/<company-slug>-<role-slug>.md, and the entry …
- Very specific: line 109
Write the text the fetch returned, never a summary and never a reconstruction.
- Very specific: line 143
Add ALL fetched jobs (new and skipped) to data/jobs.json with structure:
- Distinctive: line 183
Only present jobs NOT already in the seen list or tracker.
- Very specific: line 204
Use a short keyword drawn from the posting's title for <role keyword> - e.g.
- Distinctive: line 270
If the user picks a number, invoke the job-application-assistant skill workflow (fit evaluation first, then the tailored CV if approved).
- Very specific: line 276
If the user decides to apply to any job, set that entry's status to applied and its applied_date in data/jobs.json.
- Distinctive: line 285
Never fabricate job postings: Only present jobs from actual CLI search/detail output or WebSearch/WebFetch results.
- Very specific: line 286
Respect deduplication: Always check data/jobs.json before presenting - it holds every posting ever seen and every application ever made.
- Very specific: line 289
Be efficient with detail fetches: Don't run detail or WebFetch on every search hit — pre-filter by title/snippet, then fetch only promising …
- Distinctive: line 290
Parallel searches: Run portal CLI searches in parallel.
“Common” is measured: kept by at least 6 in 10 of the 29 edited versions of this file we could read. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Job Scraper
How It Works
This skill searches job portals using the installed portal-search CLIs in
.agents/skills/ (plus WebSearch as a fallback), using queries from your profile.
It deduplicates against previously seen jobs and the application tracker, and
presents new matches with a quick fit assessment.
Invocation
The user triggers this skill by saying things like:
- "Find new jobs"
- "Scrape for jobs"
- "Any new positions?"
- "/scrape"
Optional arguments:
- A focus area, e.g. "/scrape data science" or "/scrape geophysics"
- "broad" to run all search categories, e.g. "/scrape broad"
- "health" to run the portal health check only (Step 4.75), without searching, deduplicating, or presenting jobs - e.g. "/scrape health", or "/scrape health linkedin" to probe one portal even if disabled
Execution Steps
Step 0: Load State
- Read
data/jobs.json(create if missing - start with{"seen": {}}) - From the same file, extract the already-applied companies+roles: every entry whose status is
applied,screening,tech_interview,offerorrejected - Read
search-queries.md(this directory) for the search strategy
Step 1: Search
Read search-queries.md (this directory) for the search strategy. By default, run the top 3 priority query categories. If the user said "broad", run all categories. If the user specified a focus area (e.g. "data science"), prioritize queries from that category.
Use the installed CLI tools as the primary search mechanism. Fall back to WebSearch only for portals that do not have a CLI skill, or if bun is unavailable on the system.
1a. Check bun availability
bun --versionIf this fails (bun not installed), skip to 1c (WebSearch fallback) for all portals and note the fallback in the Step 5 output.
1b. Run CLI tools (primary — run these in parallel where possible)
Discover all installed portal CLI skills by reading every SKILL.md found under .agents/skills/*/SKILL.md. Each file documents that portal's exact CLI flags and usage examples. Use each portal's own documented interface — do not guess flags. This approach automatically includes any new portals added via /add-portal without requiring changes to this file.
Honor the enabled toggle. A portal is enabled unless its SKILL.md frontmatter sets enabled: false (a missing key means enabled — the default). Skip each disabled portal and record it for the Step 5 summary. A fork can thus keep a portal installed but sit out a run without deleting its directory.
For each enabled portal skill:
- Read its
SKILL.mdto find the correctbun run …invocation and supported flags. - Translate the query terms from
search-queries.mdinto that portal's flag format (e.g.--key,--search-string,--query, filter codes — whatever the portal's SKILL.md specifies). - Scope to the last 14 days using the portal's supported recency flag (
--jobage,--since <YYYY-MM-DD>,--order PublicationDate, etc. — as documented per portal). - Cap results to ~20 per call using the portal's limit flag.
- Use
--format jsonfor machine-readable output.
Run all portal CLI calls in parallel where possible using the Agent tool. Collect all results arrays into a single pool for Step 2, keeping each result tagged with its source portal skill (for Step 2 detail lookups).
If a CLI tool exits with a non-zero code, log the error message and continue — do not abort the whole search.
1c. WebSearch fallback
Use WebSearch for:
- Portals listed in
search-queries.mdthat do not have a corresponding directory under.agents/skills/ - Any portal whose CLI fails at runtime
- When bun is unavailable (Step 1a failed)
Use the site-specific query strings from search-queries.md directly as WebSearch queries for these portals.
Step 2: Fetch & Parse
For each promising result from Step 1:
From CLI results: Search output already includes title, company, location, date,
and URL. For jobs worth a deeper look, fetch full detail with that portal's detail
command (see its SKILL.md — do not guess flags) to extract key requirements,
application deadline, and a brief description snippet.
From WebSearch results: Use WebFetch on the posting URL and extract the same
fields manually.
For every candidate:
- Skip if the URL or company+title combo already exists in
data/jobs.json - Skip if the company+role is already applied to (see Step 0.2)
Step 2.25: Keep the Posting Text
Every posting a detail or WebFetch call returned in full gets written to
documents/postings/<company-slug>-<role-slug>.md, and the entry records that path in
posting_file (Step 4). Listings expire; this is the copy that survives, and the board
reads it there. If the filename is taken by a different posting, append -2, -3.
Write the text the fetch returned, never a summary and never a reconstruction. A posting
that could not be fetched in full gets no file and no posting_file.
# <Title> - <Company>
<location> · <work mode> · <salary or "no salary stated">
Source: <url>
Saved: YYYY-MM-DD from <portal>
---
<the description, verbatim>Save nothing for a posting fetched only as a search-result snippet: half a posting under a filename that promises the whole one is worse than the link alone.
Step 2.5: Mass-Posting Detection (within this run)
A distribution pattern worth flagging to the user as a caution signal, not as an accusation against the employer - it describes how a listing is being distributed, not a verdict on whether the company is legitimate. It alone proves nothing is wrong (companies do legitimately hire the same role across several cities); flag it so the user can factor it in when deciding whether to invest time, don't downgrade fit or silently exclude the result because of it.
If two or more results in this run's pool (from the same company, or sharing the same req/job ID visible in the URL or title) have substantially the same description and differ only in city/location/title, don't present them as separate rows. Consolidate into a single row and note the spread, e.g. "posted identically across 6 cities (BR, MX, GT)".
Step 3: Quick Fit Assessment
For each new job, do a rapid fit check (NOT the full evaluation from 04-job-evaluation.md - just a quick signal):
- High match: Role directly involves your core skills
- Medium match: Role is adjacent to your experience
- Low match: Role requires significant skills you lack
Step 4: Deduplicate & Store
- Add ALL fetched jobs (new and skipped) to
data/jobs.jsonwith structure:
{
"seen": {
"<url_or_company_title_key>": {
"title": "...",
"company": "...",
"url": "...",
"first_seen": "YYYY-MM-DD",
"fit": "high/medium/low",
"status": "new/skipped/evaluated/ranked/expired",
"portal": "<source portal skill, e.g. justjoinit-search>",
"posting_file": "documents/postings/<company-slug>-<role-slug>.md"
}
}
}posting_file is the Step 2.25 copy, repo-relative. Absent means nothing was saved -
the board's only existence check, so never write the field without writing the file.
The portal field records which CLI skill produced the job (results are already tagged per portal in Step 1b - persist that tag here). Entries written before this field existed lack it; the health check (Step 4.75) attributes those by matching the URL's domain against each portal's base URL, so do not backfill.
/rank extends this schema additively: ranked entries also carry rank_score (0–100 overall score), rank_verdict (fit band, e.g. "strong fit"), and rank_date (ISO date of ranking). The status field is set to "ranked". Do not drop any of these fields when re-writing entries.
The kanban board (studio/apps/web/) writes two fields of its own on entries the user moves by hand: status, and notes for anything they typed about the posting. One status is the board's alone - dismissed, meaning the user read the posting and ruled it out themselves, with the reason in notes. Treat those entries as closed: they stay in the dedup set like any other seen entry, /rank never re-scores them, and nothing here overwrites their notes. status in the schema above is therefore one of new/skipped/evaluated/ranked/expired/dismissed, plus the application stages the board sets (applied/screening/tech_interview/offer/rejected).
ghosted and withdrawn are no longer statuses. Every way an application ends without an offer is now status: "rejected", and which way is in the outcome array. Writing either old value still works - the board translates it on read and on write - but write rejected plus the tag instead. There are only ten statuses now, not twelve.
Six more fields belong to the board alone. Carry them through untouched and never write them here:
| Field | Where it lives | What it means |
next_id | file root, not an entry | The next id to hand out. Leave it alone; if it goes missing the board recomputes it from the highest id present. |
id | integer, first key of an entry | The role's stable number, the handle the user refers to a posting by (#153). Assigned once by the board, never reused, never renumbered. Do not invent one for a new entry - write the entry without an id and the board backfills it on next load. |
status_date | ISO date | The day status last changed, which is how the board ages a live application. |
applied_date | ISO date | The day the application actually went out, set by hand on the board. Separate from status_date, which is only ever the day of the change: an application logged after the fact needs to say when it happened. |
duplicate_of | key, or null | The entry this one is a copy of, so the board can hide it. Explicit null means the user checked and ruled it standalone - absent and null are not the same, so never replace one with the other. |
outcome | array of tags, next to status | Why a rejected application ended, and combinable: ghosted, withdrawn, on_hold, failed_screening, failed_tech, failed_behavioural. ["ghosted", "failed_tech"] is a real entry - they went quiet after the technical. Absent or empty means they simply said no; there is deliberately no tag for that, so never add one. |
- Only present jobs NOT already in the seen list or tracker.
- Compare URLs in their canonical form when deduplicating. Portals hand the same posting back under tracking parameters and locale prefixes, and treating those as distinct is what produced ~7 twin entries in this file:
nofluffjobs.com/pl/job/qa-test-engineer-merit-remoteandnofluffjobs.com/job/qa-test-engineer-merit-remote?utm_source=freehire.meare one Merit listing. Before comparing a fetched URL againstdata/jobs.json, lowercase the host, drop a leadingwww., drop a leading language segment (/pl,/en, …), strip a trailing slash, and removeutm_*and other referrer parameters (ref,source,src,fbclid,gclid,trk*).packages/jobs-data/dupes.tsholds the same rule in code if you need the exact list. A match means the posting is already seen - do not add a second entry.
Step 4.5: Generate Referral Contact Links (High & Medium Fit Only)
For every job from this run with fit of high or medium (skip low-fit jobs),
build two LinkedIn people-search URLs so the user can find a recruiter or team member to
reach out to for a referral or a warm intro. This is deliberately a link-generation step,
not an automated lookup: no scraping, no third-party API, zero runtime dependencies or
credentials required.
A. Recruiters / Talent Acquisition (the referral path)
https://www.linkedin.com/search/results/people/?keywords=<url-encoded "<Company Name> recruiter">&origin=GLOBAL_SEARCH_HEADERB. Role/team peers (informational-outreach / warm-intro path)
https://www.linkedin.com/search/results/people/?keywords=<url-encoded "<Company Name> <role keyword>">&origin=GLOBAL_SEARCH_HEADERUse a short keyword drawn from the posting's title for <role keyword> - e.g. a posting
titled "AI Program Manager" becomes "<Company Name> AI Program Manager".
Both links are for the user to open and browse themselves - never fetch or scrape the LinkedIn people-search result pages programmatically. Never fabricate contacts or claim a specific person was found; these are search links, not results.
Step 4.75: Portal Health Check
Scraper-based portal CLIs rot silently: when a portal changes its markup, the parser usually exits 0 with zero results or with null/garbled fields, and the Step 1c fallback never fires because it only triggers on hard failure. This step catches that from evidence the run already holds.
Free pass (no extra requests). For each enabled portal that ran in Step 1b:
- Degraded scan: inspect the results it returned this run. Flags:
companynull or empty on every result, empty titles, undecoded entities (&) or HTML fragments in titles, URLs that do not point at the portal. Any of these means the parser is half-working and/scrapeis silently collecting junk. - Yield history: if the portal returned zero results across all of this run's queries, check whether
data/jobs.jsonholds prior entries from it (via theportalfield, or by matching URL domains for entries predating the field). A portal that produced jobs on earlier runs and produces nothing now is suspect - the same queries worked before.
Escalation (bounded, on suspicion only). A suspect portal gets one sentinel probe: run its documented search with the example query from its own SKILL.md (that query provably worked when the skill was registered), the portal's limit flag capped at 3, --format json. If that returns nothing, retry once with a single common word. Only then is the verdict broken. A 429 or block page is never evidence of breakage - record the portal as inconclusive (rate-limited), back off, and do not retry.
Verdicts. Healthy portals get silence - no table, no line. Anything else surfaces in the Step 5 summary as a health line.
Probe-only mode (/scrape health). Skip Steps 1-4 and this step's free pass (there is no fresh run to scan); instead probe every installed portal directly - enabled ones by default, a disabled one only when named explicitly (e.g. /scrape health linkedin). Each portal gets the sentinel probe above, the degraded criteria applied to whatever it returns, and - since the user explicitly asked for diagnosis - one detail fetch on the first result of each healthy portal (description must be readable decoded text; a failure downgrades to degraded). Report all statuses in this mode, including healthy. Volume stays bounded: one search, at most one retry, at most one detail per portal.
Step 5: Present Results
Present new jobs in a table sorted by fit (high first). When Step 1b skipped
portals (enabled: false), report them with the skipped (disabled): line below
so opting one out stays visible rather than silent; omit the line when nothing
was skipped. When Step 4.75 found a portal degraded, broken, or inconclusive,
add one health: line per suspect portal (healthy portals get no line); after
the report, offer to set that portal's enabled: false so /scrape stops
running it (and covers it via the Step 1c fallback) until it is fixed - only
edit the toggle with the user's confirmation, and never edit anything else in
the skill.
## New Job Matches - YYYY-MM-DD
Found X new positions (Y high, Z medium, W low match).
skipped (disabled): <portal-name>, <portal-name>
health: <portal-name> - degraded (company null on all 12 results); parsing anchors in .agents/skills/<portal-name>/url-reference.md
health: <portal-name> - broken (0 results for the SKILL.md test query and a broader retry); parsing anchors in .agents/skills/<portal-name>/url-reference.md
| # | Fit | Title | Company | Location | Deadline | URL |
|---|-----|-------|---------|----------|----------|-----|
| 1 | High | ... | ... | ... | ... | [Link](...) |
If Step 2.5 flagged a mass-posting pattern, note it in the Title cell (e.g. "Frontend Developer (posted in 6 cities)") rather than burying it - it's a signal the user should see at a glance, not just in the detail highlights below.
### High-Match Highlights
For each high-match job, add 2-3 bullet points:
- Why it matches your profile
- Key requirements to check
- Any red flags (including mass-posting signals from Step 2.5)
### Contacts
For each high/medium-fit job from Step 4.5, add a short contacts block with the two
LinkedIn search links:
- Recruiters/TA search link, for the referral path
- Role/team-peer search link, for the warm-intro / informational-outreach pathAfter presenting, ask:
"Want me to evaluate any of these in detail? Just give me the number(s)."
If the user picks a number, invoke the job-application-assistant skill workflow (fit evaluation first, then the tailored CV if approved).
If the run found many new jobs (roughly 8+), also suggest /rank - it batch-scores all new postings against the full fit framework and returns a ranked shortlist, which beats eyeballing a long table. (/rank sets the ranked and expired status values in data/jobs.json; treat both as already-seen for dedup purposes.)
Step 6: Record an Application (Optional)
If the user decides to apply to any job, set that entry's status to applied and its
applied_date in data/jobs.json. A posting they applied to that this run did not find
gets a new entry keyed manual:<company-slug>:<role-slug> with portal: "manual (user)",
the same shape the board's Add posting dialog writes.
Important Rules
- Never fabricate job postings. Only present jobs from actual CLI search/detail output or WebSearch/WebFetch results.
- Respect deduplication. Always check
data/jobs.jsonbefore presenting - it holds every posting ever seen and every application ever made. - Focus on configured geographic area. Skip jobs that require relocation or are clearly outside commute range.
- Only open positions. Skip postings with expired deadlines or those marked as closed.
- Be efficient with detail fetches. Don't run
detailor WebFetch on every search hit — pre-filter by title/snippet, then fetch only promising matches. - Parallel searches. Run portal CLI searches in parallel; use WebSearch only for gaps the CLIs don't cover.
- No automated people lookups. Referral contacts (Step 4.5) are LinkedIn search links only - never fetch or scrape LinkedIn people-search result pages programmatically.
- Health checks are bounded and honest. Step 4.75 spends at most one probe, one retry, and (in
healthmode) one detail fetch per portal - a diagnosis, not a crawl. A rate-limit is never evidence of breakage. Health verdicts come only from observed CLI output; a portal that could not be tested is reported as inconclusive, never guessed. Theenabledtoggle is the only thing the health check may edit, and only with confirmation. - Flag distribution patterns, never accuse. The mass-posting signal (Step 2.5) describes how a listing is being distributed, not a claim that the employer is a scam. Never name a company as fraudulent or untrustworthy - present the observation and let the user decide.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

