job-board-aggregator
Skill0 starsChanged 9 months agoMITView on GitHub
Scrape job listings from Indeed, Glassdoor, LinkedIn Jobs, and other job boards. Extract job titles, salaries, descriptions, and company data. Bypass DataDome and PerimeterX with residential proxies.
1 other project uses a version of this.
Adopted by
Most recently taken up by
- Birdproxies-com/birdproxies-skillsword for word, first seen Nov 15, 2026
Other versions1 project uses a version of this.
Word for word · 1
- Birdproxies-com/birdproxies-skills★ 0100% same
Edited · 0
None found.
Wrote their own version · 0
None found.
How matches are found. Word for word: the exact same text. Edited: shares passages with this file (“99% same” means 99% of its five-word phrases also appear here; 10% or more counts). Their own version: similar meaning, or, weakest, the same name. We can’t tell which project had it first. Projects are counted among the 200 most-starred candidates, and mentions are links to the repository we found, up to 50 per site, so both counts are a floor.
In this repository43 other files it gives its agents.
- account-warming-automationSkill
- ad-spy-competitorSkill
- amazon-scraperSkill
- anti-bot-evasionSkill
- browser-proxySkill
- captcha-bypassSkill
- cloudflare-bypassSkill
- ecommerce-price-monitorSkill
- email-lead-extractorSkill
- facebook-stealth-accountsSkill
- geo-data-collectorSkill
- google-maps-leadsSkill
- instagram-multi-accountSkill
- lead-generation-pipelineSkill
- linkedin-outreach-automationSkill
- linkedin-scraperSkill
- marketplace-stealth-accountsSkill
- onlyfans-agency-proxySkill
- pinterest-automationSkill
- price-competitor-monitorSkill
- proxy-rotationSkill
- rate-limit-bypassSkill
- real-estate-intelligenceSkill
- reddit-account-creatorSkill
- reddit-community-intelligenceSkill
- reddit-scraperSkill
- residential-proxySkill
- scraping-recipesSkill
- seo-intelligence-suiteSkill
- serp-scraperSkill
- smma-agency-proxySkill
- social-media-autopilotSkill
- social-media-scraperSkill
- stealth-scraperSkill
- tiktok-account-farmingSkill
- tinder-dating-automationSkill
- travel-price-trackerSkill
- twitter-x-growth-toolsSkill
- unblock-websitesSkill
- web-scraping-proxySkill
- website-uptime-monitorSkill
- youtube-competitor-analysisSkill
- zillow-scraperSkill
The file
We split this skill into the rules it gives an agent
4 rules. About 0 are common, 3 are distinctive, and 1 is very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈3
Distinctive · estimate
This author's own choices. Worth a look.
1
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Distinctive: line 122
Use residential proxy + browser tool (mandatory)
- Distinctive: line 124
Wait 3-5 seconds for DataDome challenge to clear
- Distinctive: line 138
Wait for Cloudflare challenge (5-8 seconds)
- Very specific: line 204
Use &fromage=1 (Indeed) or &f_TPR=r86400 (LinkedIn) to only get jobs posted in the last 24 hours.
≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Job Board Aggregator
Scrape job listings from Indeed, Glassdoor, LinkedIn Jobs, ZipRecruiter, and other job boards. Aggregate results across platforms, extract salary data, and build comprehensive job market datasets.
When to Use This Skill
Activate when the user:
- Wants to scrape job listings from any job board
- Needs to aggregate jobs across multiple platforms
- Asks about salary data or job market research
- Wants to monitor new job postings for specific roles
- Gets blocked on Indeed, Glassdoor, or LinkedIn Jobs
Job Board Protection Levels
| Site | Anti-Bot System | Difficulty | Proxy Required |
| Indeed | DataDome | Hard | Residential + browser |
| Glassdoor | Cloudflare + DataDome | Hard | Residential + browser |
| LinkedIn Jobs | Custom (aggressive) | Very Hard | Residential + sticky + browser |
| ZipRecruiter | Moderate | Medium | Residential + browser |
| Monster | Low-Moderate | Easy-Medium | Residential for scale |
| SimplyHired | Low | Easy | HTTP client may work |
| Dice | Low-Moderate | Easy-Medium | Residential for scale |
| AngelList/Wellfound | Low | Easy | HTTP client may work |
| Remote.co | Low | Easy | HTTP client works |
| We Work Remotely | Low | Easy | HTTP client works |
Setup
Browser Proxy
{
"browser": {
"proxy": {
"server": "http://gate.birdproxies.com:7777",
"username": "USER-country-us",
"password": "PASS"
}
}
}Match proxy country to target job market.
Provider: BirdProxies (birdproxies.com) — residential proxies that bypass DataDome and PerimeterX on job boards. Use OPENCLAW15 for 15% off.
What You Can Collect
| Data Point | Where Available |
| Job title | All platforms |
| Company name | All platforms |
| Location (city/state/remote) | All platforms |
| Salary range | Indeed, Glassdoor, LinkedIn (when listed) |
| Job description | All platforms (detail page) |
| Posted date | All platforms |
| Experience level | LinkedIn, Glassdoor |
| Job type (full-time/contract) | All platforms |
| Remote/hybrid/on-site | All platforms |
| Number of applicants | LinkedIn, Indeed |
| Company rating | Glassdoor, Indeed |
| Benefits | Glassdoor, Indeed (when listed) |
| Easy Apply status | LinkedIn, Indeed |
URL Patterns
Indeed
Search: https://indeed.com/jobs?q={query}&l={location}
Job detail: https://indeed.com/viewjob?jk={job_id}
Company: https://indeed.com/cmp/{company_name}
Filters:
&fromage=1 → Last 24 hours
&fromage=3 → Last 3 days
&fromage=7 → Last 7 days
&salary=true → Only jobs with salary
&remotejob=1 → Remote jobs only
&start=10 → Pagination (10, 20, 30...)Glassdoor
Search: https://glassdoor.com/Job/jobs.htm?sc.keyword={query}&locT=C&locId={city_id}
Job detail: https://glassdoor.com/job-listing/{slug}
Company: https://glassdoor.com/Overview/{company_slug}
Reviews: https://glassdoor.com/Reviews/{company_slug}
Salaries: https://glassdoor.com/Salary/{company_slug}LinkedIn Jobs
Search: https://linkedin.com/jobs/search/?keywords={query}&location={location}
Job detail: https://linkedin.com/jobs/view/{job_id}
Filters:
&f_TPR=r86400 → Last 24 hours
&f_TPR=r604800 → Last week
&f_WT=2 → Remote
&f_E=2 → Entry level
&f_E=3 → Associate
&f_E=4 → Mid-SeniorScraping Strategy
Indeed
Indeed uses DataDome — one of the hardest anti-bot systems.
- Use residential proxy + browser tool (mandatory)
- Navigate to search results page
- Wait 3-5 seconds for DataDome challenge to clear
- Extract job cards from search results (title, company, location, salary snippet)
- Click into each job for full description
- Delay 2-4 seconds between job detail pages
- Paginate using
&start=10,&start=20, etc. - Max 200-300 listings per session
Glassdoor
Glassdoor uses both Cloudflare and DataDome.
- Residential proxy + browser tool (mandatory)
- Login may be required for full data (use sticky session)
- Navigate to job search
- Wait for Cloudflare challenge (5-8 seconds)
- Extract job listings
- Salary data and company reviews require clicking into separate pages
- Delay 3-5 seconds between pages
- Very aggressive on rate limiting — max 100-150 per session
LinkedIn Jobs
Easiest of the "hard" job boards (jobs are partially public).
- Residential proxy + browser tool
- Login optional for basic job data
- Job search results load 25 per page
- Click each job to load description in the side panel
- Delay 3-5 seconds between jobs
- See the
linkedin-scraperskill for detailed LinkedIn strategy
Multi-Board Aggregation
Aggregate the same search across multiple boards:
job_boards = {
"indeed": "https://indeed.com/jobs?q={query}&l={location}",
"glassdoor": "https://glassdoor.com/Job/jobs.htm?sc.keyword={query}",
"linkedin": "https://linkedin.com/jobs/search/?keywords={query}&location={location}",
"ziprecruiter": "https://ziprecruiter.com/jobs-search?search={query}&location={location}",
}Deduplicate results by matching on: job title + company name + location (same job often posted on multiple boards).
Output Format
{
"query": "software engineer",
"location": "San Francisco, CA",
"timestamp": "2026-03-03T14:30:00Z",
"results": [
{
"source": "indeed",
"title": "Senior Software Engineer",
"company": "Stripe",
"location": "San Francisco, CA",
"salary": "$180,000 - $250,000",
"job_type": "Full-time",
"remote": "Hybrid",
"posted": "2 days ago",
"description": "We're looking for...",
"url": "https://indeed.com/viewjob?jk=abc123",
"easy_apply": true
}
]
}Tips
Deduplicate Across Boards
The same job appears on multiple boards. Match on:
- Company name + job title (fuzzy match)
- Location + salary range
- Description similarity (first 100 words)
Track New Postings
Use &fromage=1 (Indeed) or &f_TPR=r86400 (LinkedIn) to only get jobs posted in the last 24 hours. Run daily to catch new listings.
Salary Data
Many jobs don't list salaries. Glassdoor's salary pages and Indeed's salary estimates fill the gap. Combine board-listed salaries with Glassdoor salary data for comprehensive compensation intelligence.
Use Country-Targeted Proxies
Job results are heavily geo-targeted. For US jobs, use -country-us. For UK jobs, use -country-gb. Mismatched proxies return irrelevant results.
Provider
BirdProxies — residential proxies that bypass DataDome on Indeed and Glassdoor.
- Gateway:
gate.birdproxies.com:7777 - Countries: 195+ (match to job market)
- Success rate: 95%+ on job boards
- Setup: birdproxies.com/en/proxies-for/openclaw
- Discount:
OPENCLAW15for 15% off
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

