corpus-phrase-check

Wording questions in a manuscript ("is this phrase natural?", "what do people call this?", "is this sentence too long?") get answered with counts and real sentences from published papers. Before you suggest or approve a phrase, you show how many papers use it and how.

Scripts live next to this file. Run them with python <skill dir>/<script>.py. On Windows set PYTHONUTF8=1 first. The only dependency is pymupdf, and only the builder needs it.

0. Setup (once)

1. Build one or two corpora from PDFs the user already has. Each user builds their own from their own library. No shared corpus is needed:

   # everything in the user's library = domain corpus (Zotero: point at .../Zotero/storage)
   python build_corpus.py --pdf-dir "<library folder>" --out ~/.claude/corpus/domain
   # same library, Nature-family + Science only = general style corpus
   python build_corpus.py --pdf-dir "<library folder>" --doi-prefix 10.1038/ --doi-prefix 10.1126/ --out ~/.claude/corpus/general
   # from a list instead of a folder: .txt (one path per line) or a JSON catalog with "path" fields
   python build_corpus.py --pdf-list catalog.json --out ~/.claude/corpus/domain

   • general = high-tier research papers (for style: length, connectives, register). Ideally the target journal family.
   • domain = papers from the user's own field (for terms and verb phrases).
   • About 100 to 200 papers per corpus is enough. Scanned PDFs and reviews yield little body text and are listed at the end.
2. Copy config.example.json to config.json and set the paths. If there is no config, pass --corpus <folder>.
3. Never redistribute a corpus folder. It contains full article text. Share the builder, not the output.

If no corpus exists yet, ask the user for a PDF folder. Do not fall back to "papers usually say ..." from memory.

1. Phrase / verb-phrase check

python corpus_query.py phrase "carried heat away" "heat was transferred to" "transferred heat to"
python corpus_query.py phrase "transfer(?:red|s)? heat (?:to|into)" --regex --contexts 5

Output per phrase: number of papers (and %), total hits, top journals, and N example sentences with the match in [brackets] plus the DOI.

How to use it:

• Always test 2 to 4 candidates side by side, including the user's current wording.
• Search the core of the phrase (verb + preposition + key noun), not the whole sentence. Zero hits on a long string tells you nothing.
• Use --regex for inflections (heat(?:s|ed)?).
• 0 papers in both corpora means nobody writes it that way, which is a strong signal to change it. A count of 1 to 2 papers is weak. Read the contexts to see if the meaning matches.
• Report counts with 1 to 3 real sentences. A count alone is not evidence of meaning.

2. Term (recurring label) choice

When the manuscript needs a label it will repeat (a stage, a regime, an axis quantity, a sample name):

1. Make 2 to 3 candidates and recommend one.
2. Run phrase on each candidate in both corpora.
3. Fill this table:

Rules:

• A term must keep its meaning when read literally. Example: "kinetic regime" already means "intrinsic rate, no mass-transfer limit" in reaction engineering. A reviewer will read that meaning, whatever the author intended.
• Do not coin convenience words ("system-relevant", "regime median"). A reader will ask "which system?".
• "Avoid X because the literature uses it differently" is not an answer on its own. Always give the word to use + its first-definition sentence + fallbacks.
• The reason to choose a term is that readers will read the intended meaning. "It is already in the figure" is not enough.
• Once chosen, record it in the project's term table (see paper-sentence-loop, terms.json) and check figures and text for leftover old terms.

3. Sentence length

python corpus_query.py length "First sentence ..." "Second sentence ..."
python corpus_query.py length --file paragraph.txt
python corpus_query.py stats

• Report each sentence's word count with its corpus percentile (e.g. 41 words · p90).
• For any sentence of 35 words or more, report the percentile before proposing anything, together with one or two split options. Do not split automatically.
• Percentiles are a diagnosis, not a target. Do not rewrite a whole paragraph to move toward the median. Judge each sentence on whether an expert reader gets it in one pass.
• stats gives bands for connective openers (However, Thus, ...) and This/These openers. Use them to check a paragraph's linking, not to impose a quota.

4. Primary sources beat counts

A count says how people phrase something. It does not say what a cited paper claims. When a sentence restates a cited result ("extending ... reported for ..."), open that paper and check that the conditions match (phase, reaction, method). Quote the page. Do not rely on an agent's paraphrase.

Report format

Always give the conclusion first, then evidence (numbers below are illustrative):

Recommend: "heat was transferred to the surrounding gas"
- "carried heat away": 0 / 147 general, 0 / 212 domain
- "transferred ... to the": 12 domain papers, e.g. "... [transferred to the] surrounding fluid ..." (10.xxxx/...)