PDF → Markdown OCR (operator runbook)

Audience: you, an AI agent operating this tool for a human user. Goal: OCR every PDF under a folder the user picks and write matching .md files into an output folder they pick, mirroring the directory structure.

This file is your runbook. Work through the sections in order. After every command, look at the result before continuing. If a step fails, fix it or tell the user exactly what's needed — never push ahead blindly onto their real files.

Throughout, $SKILL means the directory containing this SKILL.md. Set it once:

SKILL="/path/to/this/skill"   # the folder this SKILL.md lives in

If the skill directory is read-only in your environment, copy it to a writable working directory first and set $SKILL to the copy.

0. What's in this skill

The tool only writes .md files (plus _errors.log / _review.log) under the output folder. It never modifies the input PDFs.

1. Your job, end to end

1. Check prerequisites — Tesseract + language data, Python venv, best-accuracy models.
2. Self-test — run selftest.py; it generates degraded sample scans itself.
3. Ask the user for the input PDF folder, the output folder, and the language(s) of the documents.
4. Validate & confirm scope — dry-run, report the file count, get the go-ahead.
5. Run the conversion.
6. Report the result summary and where the .md files landed.

2. Prerequisite checks

2a. Tesseract + language data

which tesseract && tesseract --list-langs

• Binary found and the needed language codes listed (e.g. eng, deu) → continue.

• Otherwise install it:

  # macOS (Homebrew; tesseract-lang carries all extra languages, few hundred MB)
  brew install tesseract tesseract-lang
  
  # Ubuntu/Debian (one package per language)
  sudo apt-get install tesseract-ocr tesseract-ocr-deu

  These are sizeable downloads — run them in the background and wait. If no package manager is available, tell the user what to install and pause. Then re-run the check above and confirm the needed languages are listed.

2b. Python virtual environment + dependencies

System Pythons are often "externally managed", so always install into a local .venv — never into system Python — and always invoke the tool with $SKILL/.venv/bin/python.

First, if .venv already exists, test it:

"$SKILL/.venv/bin/python" -c "import fitz, pytesseract, PIL, tqdm; print('deps ok')"

• Prints deps ok → skip the install, go to section 2c.

• Missing, or the import errors → (re)create and install:

  python3 -m venv "$SKILL/.venv"
  "$SKILL/.venv/bin/python" -m pip install --upgrade pip
  "$SKILL/.venv/bin/pip" install -r "$SKILL/requirements.txt"

  Then re-run the deps ok check.

2c. Best-accuracy OCR models (recommended)

Quality is the priority, so fetch Tesseract's highest-accuracy models once (~35 MB, into scripts/tessdata_best/; the tool picks them up automatically):

"$SKILL/.venv/bin/python" "$SKILL/scripts/fetch_models.py" deu eng osd

Pass the language codes the user's documents need (plus osd, used for orientation detection). If the machine is offline, continue without them — the tool falls back to the system models and still passes the self-test; just tell the user that difficult scans read slightly better once the models are fetched.

2d. German language data (needed by the self-test)

The self-test fixtures are German on purpose — umlauts and ß make a demanding test case. If deu was not in the --list-langs output, install it (section 2a) even when the user's documents are in another language.

3. Readiness self-test (do not skip)

Prove the whole pipeline works before going near the user's data. The self-test needs no sample PDFs — it generates pages degraded like real bad scans (noise, low contrast, skew, uneven lighting), OCRs them, and scores the output against the known ground truth:

cd "$SKILL" && .venv/bin/python scripts/selftest.py

Expected: a per-fixture similarity table where every row says ok, then SELFTEST: PASS (takes ~30–60 s; it cleans up after itself).

The self-test must pass with or without tessdata_best/. If it prints FAIL, stop and debug (see section 7) before involving the user's files — a failing shadow fixture usually means the numpy/Pillow preprocessing is broken (reinstall, section 2b).

4. Ask the user

Ask directly in chat and wait for the answers:

1. "Which folder contains the PDFs? I'll search it and all its subfolders."
2. "Where should I save the Markdown files? I'll mirror the folder structure there."
3. "What language(s) are the documents in?" — map the answer to Tesseract codes (deu, eng, fra, …); mixed batches join with +, e.g. deu+eng.

Notes:

• Accept absolute paths, ~, and relative paths — expand ~ before using them.
• The two folders should be different. If the user gives an output folder that sits inside the input folder, that's fine (the tool skips its own output), but prefer a separate location to keep things tidy.
• If they have no preference for output, suggest a folder named mdfiles.
• If the language they name isn't installed yet, go back to section 2a for it.

5. Validate, confirm scope, then run

Substitute the answers for <INPUT>, <OUTPUT>, <LANG> below (keep the quotes — paths may contain spaces).

Dry run first — this lists how many PDFs would be processed and the src → dst mapping, without doing any OCR. It also errors clearly if <INPUT> isn't a real folder:

"$SKILL/.venv/bin/python" "$SKILL/scripts/ocr_pdfs.py" "<INPUT>" --out "<OUTPUT>" --lang <LANG> --dry-run

Tell the user the count ("Found N PDFs"). OCR is CPU-bound and runs across all cores; as a rough guide, dense 300-DPI scans take ~1–2 s per page per core. If N is large (say > 500) or it'll clearly run long, confirm with the user before the real run.

Real run:

"$SKILL/.venv/bin/python" "$SKILL/scripts/ocr_pdfs.py" "<INPUT>" --out "<OUTPUT>" --lang <LANG>

• Resumable: if it's interrupted (Ctrl-C, crash, machine sleep), just run the exact same command again — files already converted are skipped.
• Re-OCR everything (e.g. after tuning settings): add --overwrite.
• Quality is the default (orientation fix, deskew, escalation on low-confidence pages, best models). Only add --fast if the user explicitly prefers speed over accuracy.

While it runs, the tool prints a live progress bar with throughput, ETA, and running ok / skip / fail / ocr_pages / conf / review counts. Time estimate for the user: clean scans ≈ one OCR pass (~2–4 s/page/core); genuinely poor pages escalate and can take several times that — that's intentional.

6. Report back

When it finishes, summarize for the user:

• the final counts — ok / skipped / failed,
• pages OCR'd vs. taken from an existing text layer,
• the mean OCR confidence,
• the output folder path where the .md files now live,
• if any pages stayed below the confidence floor, point them to <OUTPUT>/_review.log — it lists every uncertain page (worst first) so they can spot-check those instead of proofreading everything,
• if failed > 0, point them to <OUTPUT>/_errors.log (one path <TAB> reason per line) and mention the top couple of failures.

7. Troubleshooting

8. Flag quick-reference

Run "$SKILL/.venv/bin/python" "$SKILL/scripts/ocr_pdfs.py" --help for the full list.

9. Guardrails

• The tool writes only under <OUTPUT>. Don't delete or overwrite anything outside it, and never touch the input PDFs.
• If the input path is missing or ambiguous, ask the user — don't guess.
• Don't kick off a large run without telling the user the file count first.
• Keep scripts/selftest.py and scripts/fetch_models.py in place — they are the readiness check and the model installer. (tessdata_best/ itself is disposable: re-create it any time with fetch_models.py.)
• Don't switch to --fast to "speed things up" on your own initiative; quality is the stated priority. Suggest it to the user only if they raise runtime concerns.