enrichment-waterfall
Skill32 starsChanged 7 months agoMITView on GitHub
Build multi-vendor data enrichment waterfalls in n8n — cascading API calls across SerpAPI, Hunter.io, Apollo, Clearbit, LLM extractors, and scrapers with cost-aware fallbacks. Use this skill whenever the user wants to enrich leads, contacts, companies, or any entity with external data in n8n — phrases like "lead enrichment", "email finder", "data waterfall", "Clay alternative", "find LinkedIn profile", "get company info", "enrich this list of leads". Also use when designing any flow where multiple vendors are tried in sequence until one succeeds. Use this skill before designing such workflows because naive sequential API calls produce $10/lead costs — the waterfall pattern drops that to $0.10 by ordering vendors correctly.
1 other project uses a version of this.
Adopted by
Most recently taken up by
- guerrilla2799/ops-and-scale-ostheir own version, first seen Oct 13, 2026
Other versions1 project uses a version of this.
Word for word · 0
None found.
Edited · 0
None found.
Wrote their own version · 1
- guerrilla2799/ops-and-scale-os★ 1same name
How matches are found. Word for word: the exact same text. Edited: shares passages with this file (“99% same” means 99% of its five-word phrases also appear here; 10% or more counts). Their own version: similar meaning, or, weakest, the same name. We can’t tell which project had it first. Projects are counted among the 200 most-starred candidates, and mentions are links to the repository we found, up to 50 per site, so both counts are a floor.
In this repository4 other files it gives its agents.
- chain-llm-patternSkill
- debug-workflowSkill
- mysql-checkpointingSkill
- workflow-architectSkill
The file
We split this skill into the rules it gives an agent
9 rules. About 0 are common, 4 are distinctive, and 5 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈4
Distinctive · estimate
This author's own choices. Worth a look.
5
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Distinctive: line 75
After each vendor, check BOTH response status AND payload content:
- Very specific: line 86
Don't just check .error — vendors often return 200 with empty results on a miss.
- Very specific: line 102
Before calling ANY vendor, SELECT on input_key WHERE enriched_at > NOW() - INTERVAL 30 DAY.
- Distinctive: line 106
When paid vendors miss, SerpAPI + LLM extract works 80%+ of the time:
- Very specific: line 118
IF confidence < 0.7 → treat as miss
- Very specific: line 120
Use Groq llama-3.3-70b-versatile for extract — it's fast enough that even at 90% hit rate, per-lead cost stays under 2 cents.
- Distinctive: line 124
Every vendor has limits. Hitting them burns waterfalls silently.
- Distinctive: line 139
No vendor health monitoring: Add a dashboard query: hit rate per vendor per week.
- Very specific: line 158
Add a final Set node to normalize each vendor's response into this schema before returning.
≈ “Common” is an estimate: at least 1 in 5 of the 350,495 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Enrichment Waterfall for n8n
A waterfall = try cheapest/fastest vendor first, fall through to more expensive/accurate vendors only when the previous fails. This is how Clay, Clearbit, and every production enrichment pipeline actually works.
The core pattern
Input (name, email, or domain)
↓
Vendor 1 (cheap, fast, ~60% hit rate) — e.g., Hunter.io
↓ IF no match
Vendor 2 (medium cost, ~80% cumulative) — e.g., Apollo
↓ IF no match
Vendor 3 (expensive / LLM extract, ~95% cumulative) — e.g., SerpAPI + LLM
↓ IF no match
Dead letter: log as "unenrichable"At each step, a hit short-circuits the rest. You pay only for what the cheap vendors miss.
Ordering: cost × accuracy × rate limit
Order vendors by expected cost per successful enrichment, not sticker price. Calculate:
effective_cost = price_per_call / hit_rateExample for email-from-name+company:
| Vendor | Price/call | Hit rate | Effective cost |
| Hunter.io | $0.004 | 55% | $0.007 |
| Apollo bulk | $0.01 | 75% | $0.013 |
| SerpAPI + LLM extract | $0.02 | 90% | $0.022 |
| Manual LinkedIn scrape | $0.05 | 60% | $0.083 |
Order: Hunter → Apollo → SerpAPI+LLM → dead letter. Effective cost per enriched lead ≈ $0.012 vs $0.083 if you'd started with the scraper.
n8n implementation
Structure
1. Trigger (Webhook / Schedule / Manual)
2. Set — normalize input (lowercase email, strip whitespace, extract domain)
3. MySQL / Google Sheets — check cache (was this already enriched in last 30 days?)
4. IF cache hit → return cached → END
5. HTTP Request: Hunter.io
6. IF match found → Set enriched data → merge back → END
7. HTTP Request: Apollo (on Hunter miss)
8. IF match → merge → END
9. HTTP Request: SerpAPI
10. Information Extractor (LangChain) — extract contact from SERP results
11. IF match → merge → END
12. MySQL insert — dead letter tableCritical configuration
Each HTTP Request node needs:
continueOnFail: true— so one vendor's 500 doesn't kill the pipelineretry.maxTries: 2withretry.waitBetweenTries: 3000- Timeout: 10s. Waterfalls with 6 vendors at 30s timeouts = 3-minute-per-lead worst case
- Auth via n8n credentials, never inline
The IF check pattern
After each vendor, check BOTH response status AND payload content:
// In an IF node expression:
={{
$('Hunter Request').item.json.error
? false
: $('Hunter Request').item.json.data?.email != null
}}Don't just check .error — vendors often return 200 with empty results on a miss.
Cache layer (mandatory)
Enrichment data goes stale in ~30 days but doesn't change daily. Cache aggressively:
CREATE TABLE enrichment_cache (
input_key VARCHAR(255) PRIMARY KEY, -- normalized email/domain
enriched_data JSON,
source VARCHAR(50), -- which vendor hit
enriched_at TIMESTAMP,
INDEX idx_enriched_at (enriched_at)
);Before calling ANY vendor, SELECT on input_key WHERE enriched_at > NOW() - INTERVAL 30 DAY. Cache hit rate of 40% is normal after a few weeks — that's 40% cost reduction for free.
LLM extract stage (the Stage 3 secret weapon)
When paid vendors miss, SerpAPI + LLM extract works 80%+ of the time:
HTTP Request→ SerpAPI search:"{{ $json.first_name }} {{ $json.last_name }}" "{{ $json.company }}" site:linkedin.comInformation Extractorwith schema:{ "linkedin_url": "string", "title": "string", "location": "string", "confidence": "number (0-1)" }- IF
confidence < 0.7→ treat as miss
Use Groq llama-3.3-70b-versatile for extract — it's fast enough that even at 90% hit rate, per-lead cost stays under 2 cents.
Rate limits (the silent killer)
Every vendor has limits. Hitting them burns waterfalls silently.
| Vendor | Typical limit | n8n handling |
| Hunter.io | 50/min (free tier 25/day) | Split In Batches size=1, Wait 1200ms between |
| Apollo | 600/min enterprise | Usually fine at batch size 10 |
| SerpAPI | Plan-dependent | Check headers, backoff if X-RateLimit-Remaining < 5 |
For high-volume pipelines, run the waterfall as a sub-workflow called from a Split In Batches parent with batchSize: 10, waitBetweenBatches: 60000.
Anti-patterns
- Parallelizing vendors. Running all 3 in parallel and picking the best defeats the point — you pay for all 3 on every lead. Waterfall = sequential with early exit.
- No dead letter. Unenrichable leads must go somewhere queryable so you can spot-check WHY they're missing. Otherwise vendor regressions go unnoticed.
- Same timeout across vendors. Cheap vendors are usually fast (set 5s). Scrapers need 30s. Tune per vendor.
- No vendor health monitoring. Add a dashboard query: hit rate per vendor per week. When Hunter's hit rate drops from 55% to 20%, you want to know immediately.
Output contract
Whatever the source, the waterfall should output a unified schema downstream consumers can rely on:
{
"email": "string",
"full_name": "string",
"company": "string",
"linkedin_url": "string|null",
"title": "string|null",
"enrichment_source": "hunter|apollo|serpapi_llm",
"confidence": "number (0-1)",
"enriched_at": "ISO timestamp"
}Add a final Set node to normalize each vendor's response into this schema before returning.
Reference
references/waterfall-template.json— importable 18-node starter waterfall
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

