covidence-full-text-retrieval
Skill3 starsChanged 10 days agoMITView on GitHub
Autonomously retrieve full-text PDFs for references in the full-text review stage of a Covidence systematic review, in the user's Chrome session via CDP. Looks up open-access copies via Unpaywall, Semantic Scholar, and arXiv, with an optional NotebookLM-assisted last-resort web search, then uploads any PDF it finds through Covidence's "Upload full text" action. References with no locatable full text get a note logged for manual follow-up. Never casts an Include/Exclude decision -- that stays manual.
In this repository20 other files it gives its agents.
- AGENTS.mdAGENTS.md
- covidence-full-text-reviewSkill
- covidence-screeningSkill
- impeccableSkill
- ilearning-autoadvanceSkill
- oh-my-piSkill
- auditing-worktreesSkill
- camofox-browserSkill
- demo-runbookSkill
- hermes-deepseek-cacheSkill
- i-have-adhdSkill
- memory-uiSkill
- multi-repo-pr-review-merge-roundSkill
- oh-my-ghosttySkill
- tldrSkill
- writing-reportsSkill
- sci-hub-accessSkill
- slack-mention-watchSkill
- slack-scanSkill
- superpowers-using-superpowersSkill
The file
Show its rules and text
We split this skill into the rules it gives an agent
47 rules. About 0 are common, 15 are distinctive, and 32 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈15
Distinctive · estimate
This author's own choices. Worth a look.
32
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Distinctive: line 33
No manual confirmation for any action — the agent decides and acts
- Distinctive: line 39
Run this procedure whenever any of the following occur:
- Very specific: line 46
Check if Brave is running but windowless: computer_use(action='list_apps') -- look for com.brave.Browser with running=true but windows=[].
- Very specific: line 49
If no visible window, create one via AppleScript: osascript -e 'tell application "Brave Browser" to make new window'
- Distinctive: line 63
Always run this proactively when the user says they cannot see any tabs or the browser looks empty -- before concluding a Covidence session …
- Distinctive: line 73
No PICO criteria file is needed -- this skill makes no inclusion/exclusion judgment.
- Very specific: line 105
If hostname is not app.covidence.org, OR the active tab is anything other than Screen references (e.g.
- Distinctive: line 117
Look across the whole visible list, not just the top of the page.
- Very specific: line 118
Do not click Sort, Filter, Show criteria, More options, or Tag in the top toolbar, and do not click All (select-all checkbox) or Bulk …
- Very specific: line 177
If doi is set, query DOI:${doi} as Q for a precise match.
- Very specific: line 184
Only relevant when notebooklm_topic is set.
- Very specific: line 188
Every Layer 4 step below explicitly names which tab it acts on.
- Very specific: line 194
If Tab B lands on a Google sign-in page instead: log "NotebookLM not authenticated, Layer 4 disabled for this session" once, set an …
- Distinctive: line 196
If step 3 finds MORE than one notebook with an exact title match: log a warning naming both, use the first match, do NOT create a third.
- Very specific: line 204
Ensure notebook_ready (run NotebookLM Notebook Reuse above if this is the first Layer 4 call this session).
- Very specific: line 207
Wait for the recommendation cards to render (each has a title and an AI-generated summary), up to 20s.
- Very specific: line 208
Read up to notebooklm_max_candidates cards, best-ranked first.
- Distinctive: line 224
If it came from Layer 1, retry Layer 2, then Layer 3, then Layer 4 (if enabled).
- Distinctive: line 225
If it came from Layer 2, retry Layer 3, then Layer 4 (if enabled).
- Very specific: line 226
If it came from Layer 3 (arXiv direct), retry Layer 4 (if enabled) -- otherwise NOT_FOUND overall.
- Very specific: line 227
If it came from Layer 4, try Layer 4's next candidate per its own internal retry rule (see the dedicated section).
- Very specific: line 228
If valid: discovery result is FOUND with this candidate_url and its source.
- Very specific: line 230
Never construct a candidate_url on a domain containing sci-hub, libgen, annas-archive, or z-lib (defensive check that applies to every …
- Distinctive: line 234
When validating candidate URLs, these publisher-specific behaviors are known:
- Very specific: line 251
Truncated arXiv ID resolution: when the citation text has arXiv preprint arXiv … // (truncated ID), search for the full ID via either:
- Very specific: line 257
CDP session loss: if browser_console returns "Session with given id not found.", the CDP tab session expired.
- Distinctive: line 261
Instead of the sequential Observe→Classify→Discover→Act loop for each individual ref, use this batch workflow for better throughput:
- Very specific: line 297
Find the first unresolved block: If all visible blocks are already resolved, scroll down (tab.evaluate(() => window.scrollBy(0, …
- Very specific: line 298
Idempotency guard: if current_ref_id is already in processed_ref_ids, skip this block (do NOT process it again).
- Very specific: line 299
Run the Full-Text Discovery Step for the target block (including Layer 4 if enabled and Layers 1-3 missed).
- Very specific: line 300
Approve-first-N onboarding: if actions_approved_this_session < approve_first_n AND auto_mode is false:
- Distinctive: line 302
Wait for user input. Accept three responses only:
- Very specific: line 306
When auto_mode is true from session start: skip onboarding entirely and act immediately.
- Distinctive: line 316
If not confirmed after 3 ticks: screenshot+vision fallback once to check for a stuck modal or error toast.
- Very specific: line 318
If no_notes is true: skip the note dialog entirely.
- Very specific: line 323
Add current_ref_id to not_found_ref_ids.
- Distinctive: line 333
When both hold, stop the loop, log "queue empty", print summary.
- Distinctive: line 357
Stop the loop when any of:
- Very specific: line 369
Never click Include or Exclude on any reference -- the full-text include/exclude decision is entirely manual, by explicit user instruction.
- Very specific: line 370
Never type into any field of type="password" (Hermes hard-blocks this anyway).
- Distinctive: line 371
Never click any element whose accessible name contains: logout, sign out, sign-out, signout, log off.
- Very specific: line 372
Never click Move to screening, Duplicate, Bulk upload missing full texts, Sort, Filter, Show criteria, More options, Tag, or the All …
- Very specific: line 373
Never navigate to the Resolve conflicts, Awaiting other reviewer, or Excluded references tabs, nor into Data Extraction, Risk of Bias, or …
- Very specific: line 374
If Tab A navigates outside the covidence.org domain (check via tab.evaluate(() => location.hostname)), stop the loop immediately and log.
- Very specific: line 375
Never fetch or construct a candidate_url on a domain containing sci-hub, libgen, annas-archive, or z-lib -- this applies to every discovery …
- Very specific: line 376
Never modify the review's inclusion/exclusion criteria, team members, or settings -- the agent is read-only on everything except the Upload …
- Very specific: line 382
Every tick, write a JSON line to ~/.hermes/logs/covidence-full-text-retrieval-<session-id>.jsonl (create the logs dir if missing).
≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Covidence Full-Text Retrieval
Parameters
max_refs(int, default 100): hard cap on the number of references to process before stopping.max_time(int, default 60): hard cap on session wall-clock in minutes. Whichever ofmax_refsormax_timefires first stops the session.daily_cap(int, default 300): hard cap on total references processed across sessions in a single UTC day.0disables the daily cap. Persisted inSTATE.md.approve_first_n(int, default 5): number of consequential actions (an upload or a not-found note) at the start of the session for which the agent pauses and waits for user confirmation before acting. After N approvals, the agent setsauto_mode=trueand acts unattended for the rest of the session.no_notes(bool, default false): when true, skip the per-reference notes dialog entirely for NOT_FOUND refs instead of writing a manual-follow-up note. Just silently skip. Setapprove_first_n=0from session start andno_notes=truefor a fully unattended run.contact_email(string, REQUIRED, no default): a real email address sent as theemailquery parameter on every Unpaywall API call, per Unpaywall's usage policy (https://unpaywall.org/products/api). If missing, or does not contain both@and., REFUSE TO RUN and tell the user to supply it (e.g.contact_email=you@example.org).download_dir(string, default~/.hermes/downloads/covidence-full-text-retrieval): local scratch directory for PDFs fetched before upload. Created on first use if missing.notebooklm_topic(string, optional, no default): when set, enables Layer 4 of the Full-Text Discovery Step -- an AI-assisted last-resort web search via NotebookLM's "Discover sources" feature, run only for references Layers 1-3 (Unpaywall / Semantic Scholar / arXiv) could not resolve. Also names the single shared NotebookLM notebook used for this research topic (found by exact title match, or created if none exists). When omitted, Layer 4 is skipped entirely -- no NotebookLM tab is opened, and a Layer 1-3 miss goes straight toNOT_FOUND.notebooklm_max_candidates(int, default 3): only relevant whennotebooklm_topicis set. How many of NotebookLM Discover's recommendation cards to evaluate per reference, best-ranked first, before declaring a Layer 4 miss for that reference.dry_run(bool, default false): when true, run full-text discovery for real (the lookups, including Layer 4's Discover search and source-add, are read-only-ish informational calls) but describe the upload or note action WITHOUT downloading past the validation check, clicking "Upload full text", callingtab.uploadFile, or opening the Covidence notes dialog. Use for first-pass validation against a real review or the Covidence Demo review.tick_seconds(int, default 5): idle polling interval between ticks when no action was taken. After an action, poll again immediately (the page needs time to react).
User Preferences (set these at session start for auto-mode)
This user runs fully unattended with these overrides:
approve_first_n=0— skip onboarding, enter auto_mode immediatelyno_notes=true— never write follow-up notes for NOT_FOUND refs; just skip silently- No manual confirmation for any action — the agent decides and acts
Set these on every session invocation unless the user explicitly overrides them.
Browser Window Management
Run this procedure whenever any of the following occur:
- The user says "open Brave", "re-open Brave", or "the browser isn't showing anything".
- The user reports that the tabs they see on screen don't match what CDP tools report (e.g.
browser_snapshotreturns empty,browser_navigatelands on/sign_inbut the user says they are already logged in, the page title here differs from what is visible on screen). browser_snapshotreturns(empty page)with 0 elements even though CDP reports the correct URL.
Steps (in order):
- Check if Brave is running but windowless:
computer_use(action='list_apps')-- look forcom.brave.Browserwithrunning=truebutwindows=[]. - If no visible window, create one via AppleScript:
osascript -e 'tell application "Brave Browser" to make new window' - Bring Brave to front (this is required even in background-first mode, because the user needs to see the window):
computer_use(action='focus_app', app='com.brave.Browser', raise_window=true) - Confirm the window is now visible:
computer_use(action='capture', app='com.brave.Browser', mode='vision') - Navigate the CDP session to the correct Covidence URL -- making the window visible does NOT sync the CDP tab; an explicit browser_navigate is always required after restoring a window:
browser_navigate(url='https://app.covidence.org/reviews/773228/full_text_reviews/screen_references') - Click the Covidence tab in the visible tab strip (via computer_use coordinate/element click on the tab) so the on-screen active tab matches the CDP tab, then re-capture to verify the page loaded.
Always run this proactively when the user says they cannot see any tabs or the browser looks empty -- before concluding a Covidence session has expired.
Prerequisites
- A Chrome instance is running with
--remote-debugging-port=9222. - The user has logged into Covidence at
app.covidence.orgin that Chrome and is on the Full text review -> Screen references tab of the target review (NOT Resolve conflicts, Awaiting other reviewer, or Excluded references -- those are out of scope; see Safety Rules). - Hermes's
terminaltoolset is enabled alongsidebrowser(this skill shells out tocurlandpython3for open-access lookups -- see Full-Text Discovery Step). Verify withhermes tools list; bothbrowserandterminalmust show as enabled. curlandpython3are on$PATH(macOS ships both by default).- Hermes is attached to that Chrome via
/browser connect(run this once per Hermes session before invoking the skill). - Unlike
covidence-screening, this skill does NOT require single-reviewer mode -- it never casts an Include/Exclude vote, so the reviewer-mode setting is irrelevant here. - No PICO criteria file is needed -- this skill makes no inclusion/exclusion judgment. Full-text review decisions remain entirely manual.
- If
notebooklm_topicis set: the user is already logged intonotebooklm.google.comin the SAME Chrome instance (the one Hermes is attached to via CDP). No separate NotebookLM credential setup, no cookie export. If the NotebookLM tab redirects to a Google sign-in page when Layer 4 first tries to use it, the skill disables Layer 4 for the rest of the session (logs once, falls back to Layers 1-3 only) rather than blocking the whole run. - Tool-name mapping: this skill's steps are written against a generic
tab.observe()/tab.evaluate()/tab.uploadFile()/tab.fill()vocabulary. On Hermes, map these to the actual toolset:tab.observe()->browser_snapshot;tab.evaluate(fn)->browser_console(pass the JS as an expression string, e.g.location.hrefor an IIFE);tab.screenshot()->browser_vision;tab.fill(el, text)->browser_typewhen a stable@eNref exists, otherwisebrowser_consolewith a native-setter +dispatchEvent(new Event('input', {bubbles:true}))pattern (React-controlled inputs ignore a raw.value =assignment). There is notab.uploadFileequivalent in this Hermes CDP setup -- see the NotebookLM Preflight and Action Policy sections below for the actual upload mechanism. - NotebookLM preflight (do this BEFORE starting the Screen Loop, not lazily on first Layer 4 miss): if
notebooklm_topicis set, navigate a spare tab tonotebooklm.google.comright away, before touching Covidence. If it redirects to a Google sign-in page, STOP and tell the user: "NotebookLM isn't authenticated in this CDP-attached browser profile -- please log in at notebooklm.google.com in this same browser/profile, then tell me to continue." Wait for their go-ahead, then re-check. Only proceed to the main Screen Loop once NotebookLM is confirmed authenticated (or the user explicitly says to proceed without Layer 4). This avoids burning through several real references before discovering deep into a run that Layer 4 was never going to work.
Screen Loop
Repeat until a stop condition (see Loop Control) fires:
Observe -- call
tab.observe()to get the accessibility tree of the current tab. Also calltab.evaluatewith the snippet below to get the current URL, hostname, and scroll position. Keep both results in context.(() => ({ hostname: location.hostname, pathname: location.pathname, href: location.href, scrollY: window.scrollY, innerHeight: window.innerHeight, docHeight: document.documentElement.scrollHeight }))()- Classify the screen into one of:
FULL_TEXT_REVIEW,UNKNOWN. - Act per the action policy below.
- Sleep
tick_secondsif no action was taken this tick; otherwise immediately repeat from step 1 (so the page has time to react to an upload/note before re-reading).
State Classification
Two signals, in order of preference:
- URL + page landmark (primary, deterministic) -- the
tab.evaluatesnippet above returnshostnameandpathname. Classify:FULL_TEXT_REVIEW: hostname isapp.covidence.orgAND the page heading readsFull text reviewwith aScreen references Ntab active (bold/underlined), alongsideResolve conflicts,Awaiting other reviewer, andExcluded referencestabs, and aBulk upload missing full textsbutton top-right. The accessibility tree contains at least one reference block matching the structure below.- If hostname is not
app.covidence.org, OR the active tab is anything other thanScreen references(e.g. the user navigated toResolve conflicts), classify asUNKNOWNand apply the off-domain/off-tab safety rule -- do NOT act.
- Accessibility tree scan (
tab.observe()) -- the Full text review "Screen references" tab is a single scrollable list of reference blocks (NOT one reference per screen), same layout family ascovidence-screening's T&A list. Each reference block contains, in order:- A header line like
#82 - Agarwal 2026(Covidence's own ref number + first author + year). Use the numeric part after#ascurrent_ref_idif aRef ID:line is not present; otherwise prefer theRef ID:value, same waycovidence-screeningdoes. - Right-aligned
Include/Excludebuttons -- present on this page too. This skill MUST NEVER click either (see Safety Rules); their presence is not a signal to act on, only to avoid. - The article title (large bold text).
- The authors line.
- The journal/venue + year line -- for preprints this may be two lines, e.g.
arXiv preprint arXiv:2602.08254 // 2026;(Query date: 2026-05-23 15:57:58): arxiv.org 2026 //. Keep the full citation text; the Full-Text Discovery Step regexes it for an arXiv ID. - A
DOI:line with the DOI text and an external-link icon, e.g.DOI: 10.1609/aaai.v40i36.40246 ↗. Not every reference has one (preprint-only entries often omit it). - A primary action row:
Upload full textbutton (cloud-upload icon) and anAbstractdisclosure toggle. - A per-reference footer with
Note,History,Duplicate,Move to screeninglinks.Noteopens a notes dialog -- it is NOT an inline textarea (same ascovidence-screening). - Already-resolved detection: a reference block already has full text attached when its primary action button's accessible name is anything other than exactly
Upload full text(Covidence swaps the button/label once a file exists). Treat any block whose primary button does not read exactlyUpload full textas already resolved and skip it. - Look across the whole visible list, not just the top of the page. Multiple ref blocks are on screen at once; the agent must walk them top-to-bottom.
- Do not click
Sort,Filter,Show criteria,More options, orTagin the top toolbar, and do not clickAll(select-all checkbox) orBulk upload missing full texts.
- A header line like
- Screenshot + vision (fallback only) -- invoked when the tree is ambiguous:
- The primary action button is icon-only (no accessible name) so the already-resolved heuristic can't be applied from the tree alone.
- The notes dialog or file-chooser input is not exposed in the tree (shadow DOM).
- An unexpected modal or overlay. The vision pass describes what is on screen and returns a ref or coordinate. Never use vision for the discovery decision itself -- discovery is the deterministic Unpaywall / Semantic Scholar / arXiv / NotebookLM-Discover lookup in the Full-Text Discovery Step, not visual reasoning over the page.
UNKNOWN handling
Increment an unknown_streak counter. If unknown_streak > 2: take a screenshot via tab.screenshot(), describe what you see, log the observation, and keep polling (NO clicks) until FULL_TEXT_REVIEW reappears or a stop condition fires. Do not click blindly, and do NOT navigate the tab yourself -- if the user is on the wrong tab, log and wait rather than clicking Screen references for them.
Full-Text Discovery Step
Pure terminal lookups, not UI actions, unless noted otherwise. Run once per unresolved reference block, top-to-bottom. Layers are tried in order; the first one that resolves to a validated PDF wins.
- Extract identifiers from the block's text (already in context from the Observe step):
doi: the text afterDOI:up to (not including) the trailing↗glyph or whitespace, e.g.10.1609/aaai.v40i36.40246. Empty string if noDOI:line.arxiv_id: regex the full citation text (header + journal/venue lines) forarXiv:(\d{4}\.\d{4,5}). Empty string if no match.title: the block's title text, used as a Semantic Scholar search fallback query and (if Layer 4 is enabled) as the NotebookLM Discover search query.
- Layer 1 -- Unpaywall (only if
doiis non-empty):curl -s --max-time 20 "https://api.unpaywall.org/v2/${DOI}?email=${CONTACT_EMAIL}" \ | python3 -c "
import json, sys try: d = json.load(sys.stdin) except Exception: print('') sys.exit() loc = d.get('best_oa_location') or {} print(loc.get('url_for_pdf') or loc.get('url') or '') "
Non-empty stdout -> `candidate_url`, `source=unpaywall`. Empty, or a JSON `{"error":true,...}` body -> fall through to Layer 2.
**Important**: Unpaywall's `best_oa_location.url` often returns the DOI landing page (text/html), not a PDF. `url_for_pdf` is preferred. For Springer OA articles, try `https://link.springer.com/content/pdf/{doi}.pdf` directly. For MDPI, the unpaywall `url_for_pdf` may return a 403 — use `mdpi-res.com` pattern instead (see Known Discovery Patterns).
3. **Layer 2 -- Semantic Scholar** (if Layer 1 produced nothing):
```bash
Q="${DOI:-$TITLE}"
ENC_Q=$(python3 -c "import urllib.parse, sys; print(urllib.parse.quote(sys.argv[1]))" "$Q")
curl -s --max-time 20 "https://api.semanticscholar.org/graph/v1/paper/search?query=${ENC_Q}&fields=title,openAccessPdf&limit=1" \
| python3 -c "
import json, sys
try:
d = json.load(sys.stdin)
except Exception:
print('')
sys.exit()
data = d.get('data') or []
if not data:
print('')
sys.exit()
oa = data[0].get('openAccessPdf') or {}
print(oa.get('url') or '')
"If doi is set, query DOI:${doi} as Q for a precise match; otherwise query the title text directly. Non-empty stdout -> candidate_url, source=semantic_scholar. Empty -> fall through to Layer 3.
- Layer 3 -- arXiv direct (only if
arxiv_idis non-empty and Layers 1-2 produced nothing):candidate_url="https://arxiv.org/pdf/${arxiv_id}",source=arxiv.
Tab Management (Layer 4 only)
Only relevant when notebooklm_topic is set. The skill coordinates two tabs on the same CDP-attached Chrome:
- Tab A (Covidence) -- the tab used by every other section of this skill. Default target for every action unless a step explicitly says otherwise.
- Tab B (NotebookLM) -- opened once, lazily, the first time Layer 4 actually fires this session (the first reference where Layers 1-3 all miss). Stays open and is reused for every subsequent Layer 4 attempt in the session.
Every Layer 4 step below explicitly names which tab it acts on. Never issue a click/fill intended for one tab while the other is active -- address the tab explicitly via Hermes' tab list from /browser connect, never implicitly by "whichever tab was last used."
NotebookLM Notebook Reuse (Layer 4 only)
The first time Tab B is opened in a session:
- Navigate Tab B to the NotebookLM home page (the notebook list) at
notebooklm.google.com. - If Tab B lands on a Google sign-in page instead: log "NotebookLM not authenticated, Layer 4 disabled for this session" once, set an in-memory
notebooklm_disabled_this_session = true, and skip Layer 4 for every reference for the rest of the session (Layers 1-3 still run normally). - Otherwise, scan the notebook list for a notebook whose title exactly matches
notebooklm_topic. Found -> open it, setnotebook_ready = true. Not found -> create a new notebook and set its title tonotebooklm_topicexactly, then setnotebook_ready = true. - If step 3 finds MORE than one notebook with an exact title match: log a warning naming both, use the first match, do NOT create a third. Surface this in the end-of-session summary so the user can clean up manually.
This lookup runs once per session (cached as notebook_ready in memory) and is repeated fresh next session -- nothing about the notebook's identity is written to STATE.md or any other file.
Layer 4 -- NotebookLM Discover (only if notebooklm_topic is set and notebooklm_disabled_this_session is not true)
Runs only when Layers 1-3 (per the Full-Text Discovery Step above) produced no candidate for the current reference.
- Ensure
notebook_ready(run NotebookLM Notebook Reuse above if this is the first Layer 4 call this session). If that setsnotebooklm_disabled_this_session = true, stop here -- this reference's Layer 4 is a miss, proceed to the Full-Text Discovery Step's step 5 conclusion withsource=none. - On Tab B, open Discover sources (the button next to "Add source" in the Sources panel of the resolved notebook).
- Type a search query into the "Describe something you'd like to learn about" box: the reference's exact
title, plus the first author's surname for disambiguation (e.g.MAMA-Memeia! Multi-Aspect Multi-Agent Collaboration for Depressive Symptoms Identification in Memes Agarwal). Submit. - Wait for the recommendation cards to render (each has a title and an AI-generated summary), up to 20s. No cards render -> Layer 4 miss for this reference (
notebook_sources_added=0), do not retry the same query -- go to step 7. - Read up to
notebooklm_max_candidatescards, best-ranked first. For each, in order: a. Title-match check: normalize both the candidate's title and the target reference'stitle(lowercase, strip punctuation, collapse whitespace) and require high token overlap. Not a confident match -> do not add it, move to the next candidate. b. Confident match -> click the card's Add action to import it as a source into the notebook. Increment a per-referencenotebook_sources_addedcounter. c. On Tab B, open the newly added source's "view original" affordance (external-link icon, same pattern as Covidence's own DOI links). This opens the source's real URL in a new tab, Tab C (ephemeral). Readlocation.hreffrom Tab C viatab.evaluate(() => location.href). Close Tab C. Switch back to Tab B. d. That URL iscandidate_url,source=notebooklm. Validate it per the Full-Text Discovery Step's step 6 (curl -sILContent-Type check, run overterminal, not either browser tab). Valid PDF -> discovery result isFOUND, stop trying further candidates for this reference, go to step 6 below. Invalid -> continue to the next candidate in this loop (the invalid source stays added to the notebook regardless -- it's still a relevant byproduct source for later human reading). e. Ifnotebooklm_max_candidatescandidates have all been tried (added-but-invalid, or skipped on title-match) with no valid PDF: Layer 4 is a miss for this reference. Go to step 7. - Layer 4 hit: switch back to Tab A. Discovery result is
FOUNDwith thiscandidate_url,source=notebooklm. Resume the Full-Text Discovery Step's step 6 conclusion / Action Policy'sFOUNDbranch exactly as Layers 1-3 would. - Layer 4 miss: switch back to Tab A. Discovery result is
NOT_FOUND,source=notebooklmif any candidates were evaluated (even if none added), elsesource=none. Lognotebook_sources_added(may be 0). Resume the Full-Text Discovery Step's step 5 conclusion / Action Policy'sNOT_FOUNDbranch exactly as a Layers-1-3-only miss would, except the composed note additionally credits however many sources landed in the notebook (the Action Policy's Not-Found branch below already composes this note text). - No candidate from Layers 1-3: if
notebooklm_topicis set, run Layer 4 -- NotebookLM Discover (see the dedicated section above) before concluding. Ifnotebooklm_topicis not set, or Layer 4 also produces nothing, discovery result isNOT_FOUND(source=none, orsource=notebooklmif Layer 4 ran but missed). Go to the Action Policy's Not-Found branch. - Validate whichever
candidate_urlwas produced (Layer 1, 2, 3, or 4) actually serves a PDF, not an HTML landing/paywall page:
Expected: a line containingcurl -sIL --max-time 20 "$candidate_url" | grep -i '^content-type:' | tail -1application/pdf. If the header is missing, times out, or reads anything else (e.g.text/html), this candidate is invalid:- If it came from Layer 1, retry Layer 2, then Layer 3, then Layer 4 (if enabled).
- If it came from Layer 2, retry Layer 3, then Layer 4 (if enabled).
- If it came from Layer 3 (arXiv direct), retry Layer 4 (if enabled) -- otherwise
NOT_FOUNDoverall. - If it came from Layer 4, try Layer 4's next candidate per its own internal retry rule (see the dedicated section); if all of Layer 4's candidates are exhausted,
NOT_FOUNDoverall. If valid: discovery result isFOUNDwith thiscandidate_urland itssource.
Never construct a candidate_url on a domain containing sci-hub, libgen, annas-archive, or z-lib (defensive check that applies to every layer, including Layer 4, even though none of the four legitimate sources used here can produce one) -- see Safety Rules.
Known Discovery Patterns (pitfalls from production sessions)
When validating candidate URLs, these publisher-specific behaviors are known:
| Publisher | Unpaywall URL | PDF accessibility |
| arXiv | https://arxiv.org/pdf/{id} | ✅ Always application/pdf |
| MDPI | Unpaywall url_for_pdf e.g. https://www.mdpi.com/.../pdf?version=... returns 403; use https://res.mdpi.com/d_attachment/{journal}/{article-id}/article_deploy/{file.pdf} which redirects 301 to https://mdpi-res.com/... and serves application/pdf | ✅ via mdpi-res redirect |
| Springer OA | https://link.springer.com/content/pdf/{doi}.pdf | ✅ When article is OA |
| Springer LNCS / CCIS (978-3-031-, 978-3-032-) | Landing page | ❌ Paywalled — skip when is_oa: False |
| IEEE (10.1109/*) | Landing page | ❌ Paywalled — all IEEE conference DOIs fail |
| Elsevier (10.1016/*) | Landing page | ❌ Paywalled unless is_oa: True + has PMC link |
| ACM (10.1145/*) | dl.acm.org/doi/pdf/... returns text/html | ❌ Paywalled for most conferences |
| PMC / PubMed Central | PMC landing page | ❌ curl returns text/html; usable as Covidence link (not a PDF upload) |
| SCITEPRESS (10.5220/*) | Landing page | ❌ Paywalled |
| SPIE (10.1117/*) | Landing page | ❌ Paywalled |
| JCO / ASCO (10.1200/*) | Abstract only | ❌ Abstract-only conference paper |
| researchgate.net | No DOI | ❌ Lacks stable PDF URL |
Truncated arXiv ID resolution: when the citation text has arXiv preprint arXiv … // (truncated ID), search for the full ID via either:
curl -s "https://arxiv.org/search/?searchtype=all&query=...title...&start=0" | python3 -c "import sys,re;html=sys.stdin.read();print(chr(10).join(re.findall(r'arxiv\.org/abs/(\d{4}\.\d{4,5})',html))[:3])"- Semantic Scholar paper search with
externalIdsfield:curl -s "https://api.semanticscholar.org/graph/v1/paper/search?query=...&fields=externalIds&limit=3"
Covidence filter accidentally activating: if the tab count (Screen references N) stays high but the page shows "N studies found" with a "Reset filters" button, a filter was accidentally clicked. Click the Reset button before continuing. The filter button and the Upload full text modal button are close in the DOM.
CDP session loss: if browser_console returns "Session with given id not found.", the CDP tab session expired. Reconnect with browser_profile(name='brave-live') and re-verify you're on the correct page with browser_console(() => location.href) before continuing.
Batch Workflow (preferred; faster than per-ref sequential)
Instead of the sequential Observe→Classify→Discover→Act loop for each individual ref, use this batch workflow for better throughput:
- Read all visible refs at once from the initial
browser_navigatesnapshot or a freshbrowser_snapshot(full=true). Parse all headers, DOIs, arXiv IDs, and title text. - Scroll down to reveal more refs (
window.scrollBy(0, window.innerHeight * 0.8)), capture another snapshot, parse those too. Continue until no new refs appear. - Run discovery in parallel batches — group DOIs and query Unpaywall for dozens at once across multiple terminal calls. Then batch Semantic Scholar queries, then arXiv lookups.
- Compile a single upload queue of
{ref_id, url}for all FOUND refs. - Upload via the JS helper — inject
uploadQueueintobrowser_consolefromscripts/upload_link.js(see below), then call it with the compiled queue. - Log the batch to the JSONL file with a single timestamp for all uploads.
- Skip NOT_FOUND refs silently — no notes, no dialogs, just add to
not_found_ref_idsand update STATE.md count.
Batch Upload Helper
A reusable async JS helper is stored at scripts/upload_link.js in this skill directory.
Usage:
// 1. Load into browser_console via:
// skill_view(name='covidence-full-text-retrieval', file_path='scripts/upload_link.js')
// then copy the function definition into browser_console
// 2. Call:
uploadQueue([
{ref_id: '443', url: 'https://arxiv.org/pdf/2506.06574'},
{ref_id: '382', url: 'https://arxiv.org/pdf/2507.22504'},
]).then(r => JSON.stringify(r));The function handles: finding the ref block by header paragraph, clicking Upload, setting the link input via React-compatible native HTMLInputElement setter, clicking Add link + Done, and verifying resolution. Returns per-ref status.
Action Policy
FULL_TEXT_REVIEW -- walk the list top-to-bottom
The Screen references list is a scrollable list of reference blocks, many visible at once. The agent walks them top-to-bottom, resolving the first unresolved block it finds, then re-observing and continuing.
- Scan the visible reference blocks top-to-bottom. For each block, determine
current_ref_idand whether it is already resolved (primary button does not read exactlyUpload full text; see State Classification). - Find the first unresolved block. If all visible blocks are already resolved, scroll down (
tab.evaluate(() => window.scrollBy(0, window.innerHeight * 0.8))) to reveal more, and re-observe. If scrolling produces no new unresolved blocks after 2 attempts, the queue is empty -- see "Queue empty" below. - Idempotency guard: if
current_ref_idis already inprocessed_ref_ids, skip this block (do NOT process it again). Continue scanning down the list. - Run the Full-Text Discovery Step for the target block (including Layer 4 if enabled and Layers 1-3 missed). Result is
FOUND(withcandidate_url,source) orNOT_FOUND. - Approve-first-N onboarding: if
actions_approved_this_session < approve_first_nANDauto_modeis false:- Print to the terminal: the
current_ref_id, the ref header, the title (short), the discovery result (FOUND+candidate_url+source, orNOT_FOUND), and the action about to be taken (uploadornote). - Wait for user input. Accept three responses only:
approve-- take the action as decided. Incrementactions_approved_this_session. If it now equalsapprove_first_n, setauto_mode = trueand announce "Onboarding complete, switching to unattended mode."skip-- do not act on this reference. Do NOT incrementactions_approved_this_session. Addcurrent_ref_idtoskipped_ref_idsand continue scanning down. Log the skip.stop-- end the session immediately. Log summary including which references were skipped. Whenauto_modeis true from session start: skip onboarding entirely and act immediately.
- Print to the terminal: the
- If
FOUND(or describing it underdry_run):- Download:
mkdir -p "${DOWNLOAD_DIR}"thencurl -sL --max-time 60 -o "${DOWNLOAD_DIR}/${current_ref_id}.pdf" "$candidate_url". - Verify the download: the file must exist and be larger than 1024 bytes (
stat -f%zon macOS). A file at or under that size is a truncated response or an error page that slipped past the Content-Type check -- treat asNOT_FOUNDinstead and fall through to step 7's Not-Found handling for this reference (log the discrepancy). - In
dry_run: skip the actualcurldownload and print "would download$candidate_urland upload it" instead; do not touch the browser. - Otherwise, in the browser (Covidence tab): click the target block's
Upload full textbutton. Because@eNrefs frequently expire between an observe and a click on this list (Covidence re-renders often), prefer locating the button viabrowser_consoleby climbing from the ref's header paragraph (e.g.Array.from(document.querySelectorAll('p')).find(p => p.textContent.trim().startsWith('#'+ref_id+' -'))) up to its containing block, then.click()the button whose text is exactlyUpload full textwithin that block -- this guarantees you hit the button belonging to THIS block, not the one above or below. - Known DOM-depth quirk: the
Upload full text/Abstractbuttons sit twoparentElementhops above the header paragraph, butNote/History/Duplicate/Move to screeningsit ONE hop further up (threeparentElements total). Don't assume a fixed depth -- when a button search comes back empty at one level, climb one moreparentElementand retry before concluding the button is missing. - File upload mechanism (Hermes CDP has no
tab.uploadFile): this environment's CDP browser tools cannot drive a native OS file picker, sotab.uploadFileas an action does not exist here. Use Covidence's own "Link to full text" fallback in the same modal instead: locate theinput[name="document_url"]field viabrowser_console, set its value with the nativeHTMLInputElementvalue setter + dispatch aninputevent (React-controlled input; a plain.value =assignment is silently ignored), click theAdd linkbutton, confirm the link chip appears in the modal's list, then clickDone. This is the primary upload path on Hermes, not a last resort -- do not attempt a real PDF upload flow here; download-and-verify (viacurl) still matters for validating the candidate URL is a real PDF, but the actual "upload" step in Covidence is done by attaching that same URL as a link, not re-uploading the downloaded bytes. - For batch uploads, prefer the
uploadQueuehelper inscripts/upload_link.jsover manual per-ref click/fill/done cycles. - Re-observe within 3 ticks to confirm: the block's primary button no longer reads
Upload full text(per the already-resolved heuristic; Covidence swaps it toFull text/Manage full textonce a link or file is attached). If confirmed: logok:true, addcurrent_ref_idtouploaded_ref_ids. - If not confirmed after 3 ticks: screenshot+vision fallback once to check for a stuck modal or error toast. If still unresolved, log "upload did not confirm, needs manual check", add
current_ref_idtoupload_failed_ref_ids-- do NOT retry the click a second time (avoid double-uploading).
- Download:
- If
NOT_FOUND(or describing it underdry_run):- If
no_notesis true: skip the note dialog entirely. Do not open the Note link. Do not write anything. Log the skip and move on. - Otherwise: compose a note accordingly and write it.
- Compose a note:
"[covidence-full-text-retrieval] No full text found via automated lookup on <ISO date>. Tried: Unpaywall<if doi present>, Semantic Scholar, arXiv<if arxiv_id present><, NotebookLM Discover (N sources added to notebook '<notebooklm_topic>') if Layer 4 ran>. Needs manual retrieval."(include only the sources actually attempted per the identifiers extracted and whether Layer 4 was enabled). - In
dry_run: print the note text instead of opening the dialog. - Otherwise: click the target block's
Notelink (samebrowser_consoleblock-locator technique as the Upload step above -- remember the DOM-depth quirk:Notesits oneparentElementhop further up thanUpload full text/Abstract). Find the textarea inside the opened dialog viabrowser_snapshotorbrowser_console, fill it withbrowser_type(or the native-setter +input-event pattern viabrowser_consoleif it's a React-controlled textarea andbrowser_typedoesn't stick), then click the dialog'sAdd note/ save button. Confirm by re-checking the block's footer now shows1 Note(or an incremented count) instead of bareNote. If the dialog or textarea is not exposed, fall back to screenshot+vision to locate it. If still not found after 1 retry, log "notes dialog unavailable, skipped note" and move on without a note -- do NOT clickUpload full textor either vote button as a substitute action. - Add
current_ref_idtonot_found_ref_ids.
- If
- In all cases (uploaded, upload-failed, note written, note-skipped, or no_notes skip): add
current_ref_idtoprocessed_ref_ids, incrementrefs_processed. Also updateSTATE.md: read itsdatefield; if it is not today's UTC date, resetprocessed_todayto 0 and setdateto today's UTC date before incrementing; then incrementprocessed_todayby 1 and write both fields back toSTATE.md. - Poll the next tick immediately (no sleep) so the page has time to register the action before re-reading.
FULL_TEXT_REVIEW -- queue empty
The Screen references list does NOT show a "No more references" banner. The queue is empty when:
- Scrolling reveals no new unresolved reference blocks after 2 scroll attempts, AND
- The top tab count confirms (e.g.
Screen references 0remaining, or the count matchesrefs_processedthis session).
When both hold, stop the loop, log "queue empty", print summary. If the tab count shows unresolved refs remain but none render, log and STOP -- do not click blindly.
UNKNOWN
Increment unknown_streak. Screenshot + vision to classify. If genuinely a transient page transition (Covidence loading), wait one tick and re-observe. If a modal/error overlay is blocking, or the user has navigated to a different tab (Resolve conflicts, Awaiting other reviewer, Excluded references) or a different Covidence section, log and STOP -- do not click through modals or navigate the tab back yourself.
Loop Control
Track these counters across the session:
refs_processed(int, starts 0): references resolved (uploaded, note-written, or note-skipped) this session.current_ref_id(string or null): the reference currently being processed.processed_ref_ids(set): ref IDs resolved this session (idempotency guard).uploaded_ref_ids(set): ref IDs whose full text was successfully uploaded.not_found_ref_ids(set): ref IDs with a note logged for manual follow-up, or silently skipped whenno_notes=true.upload_failed_ref_ids(set): ref IDs where a PDF was found and downloaded but the Covidence upload did not confirm.skipped_ref_ids(set): ref IDs skipped during onboarding (so they aren't re-targeted next tick).last_3_ref_ids(list of last 3current_ref_idvalues, infinite-loop guard).actions_approved_this_session(int, starts 0): approvals received in onboarding.auto_mode(bool, starts false): whether onboarding is complete.session_start_ts(ISO timestamp): formax_timeenforcement.unknown_streak(int, starts 0): consecutive UNKNOWN classifications; screenshot threshold for the UNKNOWN handling rule.notebook_ready(bool, starts false): whether Layer 4's notebook find-or-create has run this session (Layer 4 only).notebooklm_disabled_this_session(bool, starts false): set true if Tab B ever hits a Google sign-in page; disables Layer 4 for the rest of the session without stopping the whole run (Layer 4 only).
Stop the loop when any of:
refs_processed >= max_refs.now - session_start_ts >= max_time * 60seconds.- Daily cap: read
STATE.md. If itsdatefield is not today's UTC date, treatprocessed_todayas 0 for this comparison (the on-disk counter has not been reset yet -- see Action Policy step 8 for the reset-and-increment mechanics). If today's counter >=daily_cap(anddaily_cap != 0), stop with "daily cap reached (N/N)". - Queue empty (per the queue-empty rule above).
last_3_ref_idsare all identical AND the last action was an upload or note attempt (infinite-loop guard).- Unrecoverable error: CDP connection dropped, page navigates outside
covidence.org(Tab A) or an unrecoverable state on Tab B that isn't the sign-in redirect covered bynotebooklm_disabled_this_session, primary action button missing for > 30 s,curl/API failure after 3 backoffs. - Approve-first-N: user responded
stop.
Safety Rules
Hard rules the agent MUST follow:
- Never click
IncludeorExcludeon any reference -- the full-text include/exclude decision is entirely manual, by explicit user instruction. This skill only retrieves and uploads full text, or notes that it could not. - Never type into any field of
type="password"(Hermes hard-blocks this anyway). - Never click any element whose accessible name contains: logout, sign out, sign-out, signout, log off.
- Never click
Move to screening,Duplicate,Bulk upload missing full texts,Sort,Filter,Show criteria,More options,Tag, or theAllselect-all checkbox. - Never navigate to the
Resolve conflicts,Awaiting other reviewer, orExcluded referencestabs, nor into Data Extraction, Risk of Bias, or Settings -- strict allowlist of the Full text review -> Screen references tab. If Tab A navigates elsewhere insidecovidence.org, stop and log. - If Tab A navigates outside the
covidence.orgdomain (check viatab.evaluate(() => location.hostname)), stop the loop immediately and log. - Never fetch or construct a
candidate_urlon a domain containingsci-hub,libgen,annas-archive, orz-lib-- this applies to every discovery layer, including Layer 4, even though none of the four legitimate sources used here can produce one, but the check stays as a hard backstop. - Never modify the review's inclusion/exclusion criteria, team members, or settings -- the agent is read-only on everything except the
Upload full textfile input and the per-reference notes dialog. - Layer 4 / NotebookLM only: strict allowlist on Tab B of the Discover-sources flow, the Add-source action, and opening a source's "view original" link, all confined to the one notebook resolved for
notebooklm_topic. Never delete, rename, or share that notebook. Never touch any OTHER notebook in the account. Never remove a source once added, even one that didn't validate as a PDF. If Tab B ever navigates outsidenotebooklm.google.com(other than the deliberate, immediately-closed Tab C for reading a source's original URL), disable Layer 4 for the rest of the session (log, setnotebooklm_disabled_this_session = true) and continue with Layers 1-3 only -- do not stop the whole run over a Layer 4-only problem. - Hermes's built-in destructive-action blocklists (recursive force-delete, piped-shell installers, fork bombs, lock-screen combos) remain active and are NOT overridden by this skill.
Logging
Every tick, write a JSON line to ~/.hermes/logs/covidence-full-text-retrieval-<session-id>.jsonl (create the logs dir if missing). The <session-id> is the ISO timestamp at session start.
{"ts":"2026-07-25T12:34:56Z","ref_id":"82","ref_header":"#82 - Agarwal 2026","title":"<short title>","state":"FULL_TEXT_REVIEW","discovery":"found","source":"notebooklm","candidate_url":"https://.../paper.pdf","action":"upload","ok":true,"notebook_sources_added":1}Fields: ts (ISO), ref_id, ref_header (the #82 - ... line), title (truncated), state (FULL_TEXT_REVIEW/UNKNOWN), discovery (found/not_found), source (unpaywall/semantic_scholar/arxiv/notebooklm/none), candidate_url (empty string if not_found), action (upload/note/skip/dry-run describe), ok (bool), notebook_sources_added (int, default 0 -- only nonzero when Layer 4 ran and added at least one source to the notebook for this reference, regardless of whether one validated as a PDF).
At loop end, print a summary to the terminal: refs processed, uploaded/not-found/upload-failed counts, the not-found reference list (for manual follow-up), time elapsed, any stuck points, daily-cap remaining, the local path to download_dir, and (if notebooklm_topic was set) the notebook's title, how many references triggered Layer 4, and the total notebook_sources_added across the session.
Error Recovery
| Failure | Detection | Recovery |
| CDP connection dropped | tab.observe() returns connection error | Stop, log, surface to user. Cannot auto-recover (user must restart Chrome). |
| CDP session lost mid-action | browser_console returns "Session with given id not found." | Reconnect with browser_profile(name='brave-live'), verify page with location.href, resume. |
| Filters accidentally activated | Page shows "N studies found" count < tab count, "Reset filters" button visible | Click the Reset filters button before continuing. |
| Session/login expired (Covidence) | Page navigates to institutional SSO login URL, or no reference blocks render for > 30 s | Stop loop, log URL, surface to user. No auto-relogin. |
| Unpaywall/Semantic Scholar rate limit or timeout | curl returns non-2xx, empty body, or times out | Fall through to the next discovery layer immediately (do not retry the same layer) rather than failing the whole session. Only stop if ALL layers fail for 3 consecutive references in a row -- log and surface to user (likely a network-wide outage). |
| Download produced a truncated/error file | Downloaded file <= 1024 bytes despite a application/pdf Content-Type header | Treat as NOT_FOUND for this reference; log the discrepancy; continue with the Not-Found note action. |
| Upload did not confirm | Target block still reads Upload full text 3 ticks after tab.uploadFile | Screenshot+vision once to check for a stuck modal/toast. If still unresolved, log, add to upload_failed_ref_ids, do NOT retry the click. Continue scanning down. |
| Notes dialog write failed | Note link @eN not found, dialog didn't open, or tab.fill returns error | Screenshot+vision to locate the Note link / dialog. If still not found after 1 retry, log "notes dialog unavailable, skipped note" and move on without writing it -- do NOT vote or upload as a substitute. |
| Queue-empty detection false negative | Tab count shows unresolved refs remain but scrolling yields no unresolved blocks after 2 attempts | Screenshot + vision to classify. If genuinely empty, stop with summary. If a modal/error overlay is blocking, log and STOP rather than clicking through it. |
| Page JS error / blank render | Screenshot returns empty canvas or error overlay | Wait 2 ticks; if persists, log and STOP. |
Infinite loop (same current_ref_id targeted 3x without landing) | last_3_ref_ids identical after action attempts | Stop, log, surface to user. |
browser_navigate false-timeout | browser_navigate returns "Operation timed out" even though the navigation actually completed | This is a known false-negative on this Hermes CDP setup, not a real failure. Immediately check location.href via browser_console before treating it as an error -- if the URL matches the intended destination, proceed normally; only escalate to a real error if the URL is still stale after this check. |
| Daily cap hit | STATE.md counter >= daily_cap (and daily_cap != 0) | Stop, log "daily cap reached (N/N)", surface to user. |
Approve-first-N: user says skip | User response during onboarding | Leave reference unresolved, add to skipped_ref_ids, continue scanning down. Do NOT auto-act. Onboarding counter does NOT advance. |
Approve-first-N: user says stop | User response during onboarding | End session immediately. Log summary including which references were skipped. |
| NotebookLM tab (Tab B) not authenticated | Tab B navigates to a Google sign-in URL | Log once, set notebooklm_disabled_this_session = true, continue with Layers 1-3 only for the rest of the session. Do not stop the whole run. |
| Discover sources produces no cards / times out | No recommendation cards render within 20s of submitting the query | Treat as a Layer 4 miss for this reference (no candidates to evaluate); do not retry the same query. |
| "View original" doesn't open a new tab / URL unreadable | tab.evaluate on the expected Tab C errors or times out | Skip this candidate, try the next one (up to notebooklm_max_candidates); if all fail, Layer 4 miss for this reference. |
Notebook-list scan finds two notebooks matching notebooklm_topic | More than one exact-title match on the NotebookLM home page | Log a warning, use the first match, do not create a third. Surface this in the end-of-session summary so the user can clean up manually. |
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

