Carrel

Research environment toolkit for academics. A Python core library (carrel CLI), a Claude Code plugin (skills, agents, hooks, commands), and a portable skill pack (skills/carrel/) packaged as a cross-host plugin (plugins/carrel/) for GitHub Copilot, Codex and Kimi Code.

Quick Commands

# Core library
uv run pytest                                    # Full test suite
uv run carrel env doctor                         # Hardware + tools audit
uv run carrel env validate --vault .            # Validate environment.json + drift markers
uv run carrel env fix --safe --vault .          # Apply safe environment.json repairs
uv run carrel vault init /tmp/test               # Scaffold a vault
uv run carrel paper convert paper.pdf            # Convert PDF (liteparse default)
uv run carrel transcript create rec.m4a          # Transcribe audio (coli default)
uv run carrel capture url https://example.com    # Web capture (defuddle)
uv run carrel google export <google-docs-url>    # Google Docs export (gws)
uv run carrel vault cheatsheet --vault . --force # Regenerate _meta/cheat_sheet.md
uv run carrel vault dashboard --vault . --force  # Regenerate _meta/my-environment.md
uv run carrel vault automation-prompt --vault . --force # Regenerate _meta/automation-prompt.md
uv run carrel vault check-sync --vault .         # Check CLAUDE.md profile markers for drift
uv run carrel vault add-markers --vault .        # Append profile markers to CLAUDE.md
uv run carrel setup-state show --vault .         # Inspect setup phase (v0.5.3+)
uv run carrel setup-state advance --phase N      # Move to next phase atomically
uv run carrel setup-state complete --vault .     # Mark setup complete (idempotent)
uv run carrel trust check automation:propose --vault .  # Check if action allowed at current trust level
uv run carrel trust list --vault .  # See what current trust unlocks
uv run carrel paper convert foo.pdf --explain    # Print routing decision + rationale without executing
uv run carrel vault feedback export --redact-list redact.txt --vault . # Anonymized feedback digest (v0.9.0)
uv run carrel vault mirror --write --from-stdin --vault . # Idempotent dated mirror file (v0.9.0)
uv run carrel vault reflect-log --append --from-stdin --vault . # Atomic append to dated reflect-log (v0.9.0)
uv run carrel vault share generate --mode quick --for alice --sensitivity medium --vault . # Collaborator handbook (v0.9.0)
uv run carrel batch convert <folder> [--unattended] --vault . # Sequential PDF batch (v0.9.0)
uv run carrel batch transcribe <folder> [--unattended] --vault . # Sequential audio/YouTube batch (v0.9.0)
uv run carrel automate configure --enabled --trust-level consultative --schedule daily --review-cadence quarterly --vault . # Typed-flag automation config, no prompts (v0.9.0)
uv run carrel migrate apply [--plugin-root <path>] --vault . # Walk migrations registry, update plugin-state.json (v0.9.0)
uv run pytest --cov                              # Coverage run (pytest-cov in dev group; config in pyproject [tool.coverage.*])
scripts/sync-skill-repo.sh [--push]              # Publish skills/carrel → linxule/carrel-skill via git subtree (dry-run default)
python scripts/sync-agent-plugin.py [--check]    # Regenerate plugins/carrel/ from skills/carrel (--check fails on drift)
python scripts/build-skill-zip.py                # Build dist/carrel.zip for Claude app skill upload

Architecture

Skills (markdown)     → human judgment, loaded by any AI
Core library (Python) → deterministic operations, no AI
Transports (thin)     → plugin, CLI, MCP, agent SDK apps

The core library NEVER asks questions or makes judgment calls. Skills handle that. If a required parameter is missing, the library returns an actionable error.

Host-split (decision 2026-07-10): the typed CLI (src/carrel, provisioned by install.sh via uv tool install) is the Claude Code plugin's sole runtime — hooks, slash commands, and the 13 workflow skills call carrel .... The portable skill pack (skills/carrel/: SKILL.md + stdlib-only scripts/carrel.py runtime) is the standalone engine for every other host (Codex, GitHub Copilot, Kimi Code, Gemini CLI, OpenCode, Cursor, Claude app zip) — sandbox-native because it travels as files + python3. The pack is a strict subset; tests/test_runtime_parity.py is the drift alarm (same-input comparisons on shared surfaces + an enumerated allowlist of intended vault init divergences: setup-state.json typed-only, agent-context.md portable-only). Distribution: scripts/sync-skill-repo.sh --push subtree-publishes skills/carrel/ to linxule/carrel-skill for npx skills / .agents/skills/ installs. Never edit carrel-skill directly — it's a publish target, not a second source of truth; skills/carrel/ here is canonical. The same pack is also packaged as a cross-host plugin at plugins/carrel/ (Agent Plugins 1.0 plugin.json for Copilot + Codex, kimi.plugin.json for Kimi Code; catalogs at .github/plugin/marketplace.json and .agents/plugins/marketplace.json; repo-root kimi.plugin.json points Kimi at the packaged skill only). plugins/carrel/skills/carrel/ is generated — edit skills/carrel/, then run python scripts/sync-agent-plugin.py; tests/test_packaging_contract.py fails on drift and pins each host's manifest rules. For Claude app uploads, scripts/build-skill-zip.py builds carrel.zip (root folder must be carrel/ — Claude rejects mismatched folder names); .github/workflows/skill-zip.yml republishes it to the skill-latest release on every skills/carrel/ change on main.

Directory Structure

carrel/
├── src/carrel/           # Python core library
│   ├── cli/              # typer CLI (paper, transcript, vault, env, capture, google, setup-state, trust, automate, batch, migrate)
│   ├── convert/          # PDF/doc conversion (router, adapters, filer, pipeline)
│   ├── transcribe/       # Audio/video transcription (router, adapters, filer)
│   ├── google/           # Google Workspace export (gws CLI integration)
│   ├── vault/            # Vault scaffold, organize, templates, markers, dashboard, automation_prompt
│   ├── env/              # Audit, profile, install, validation, healing, platform
│   ├── policy/           # Sensitivity routing (16-row matrix) + trust enforcement
│   ├── feedback/         # Anonymized feedback digest exporter (v0.9.0)
│   ├── migrate/          # Migration runner — registry walk + plugin-state.json writer (v0.9.0)
│   ├── share/            # Collaborator handbook synthesizer + sensitivity redaction (v0.9.0)
│   ├── models.py         # Pydantic models (options, results, enums, AutomationConfig, ResearcherProfile, SetupState)
│   ├── safe_path.py      # Vault containment helper
│   ├── youtube_url.py    # Unified YouTube URL parser
│   ├── source_hash.py    # Unified source-hash helper for idempotency
│   └── errors.py         # CarrelError with actionable hints
├── tests/                # pytest suite
├── templates/            # Vault templates (loaded by vault/templates.py)
├── skills/               # plugin skills (including the portable carrel skill pack)
├── agents/               # Plugin agents (setup-interviewer, research-partner)
├── hooks/                # 4 plugin hooks: SessionStart (check-environment.js), SessionEnd (session-reflect.js), UserPromptSubmit (inject-context.js, v0.9.0), PreToolUse Bash matcher (sensitivity-gate.js, v0.9.0)
├── commands/             # 15 plugin slash commands (/carrel-*); 7 are thin wrappers per CONVENTIONS.md
│   └── CONVENTIONS.md    # ${ARGS} (skill-constructed) vs $ARGUMENTS (raw) convention (v0.9.0)
├── planning/             # Specs, reviews, prompts, reports (multi-model review workflow)
├── plugins/carrel/       # Generated cross-host plugin (Copilot, Codex, Kimi) — never edit; run scripts/sync-agent-plugin.py
├── scripts/              # sync-skill-repo.sh (publish carrel-skill), sync-agent-plugin.py (regenerate plugins/carrel), build-skill-zip.py (Claude app ZIP)
├── bootstrap.sh          # Deprecated shim — hands off to install.sh
├── install.sh            # macOS/Linux one-line installer (curl | bash)
├── install.ps1           # Windows installer (first-class as of v0.7.0)
└── pyproject.toml        # pydantic + typer + rich + httpx + markitdown + youtube-transcript-api (defuddle, liteparse, coli are bun tools)

Tool Routing

PDF conversion

liteparse (local default) > mineru (cloud, explicit --tool mineru) > markitdown (non-PDF only)

No markdownify fallback for PDFs. Missing liteparse → clear error with install command.

Audio transcription

coli (local default) > groq (cloud, explicit --tool groq)

No markdownify fallback for audio. Missing coli → clear error.

YouTube

youtube_captions (network-lite default, fetches existing public captions with timestamps) > gemini (cloud, explicit --tool gemini or cloud_consent)

Captions are a network-lite fetch, not a cloud upload — no vault data egresses, so they route on LOW/MEDIUM without cloud_consent. HIGH sensitivity blocks caption fetches by default but honors an explicit --tool youtube_captions override with a warning (the researcher asserts the URL is not sensitive). gemini is a true cloud tool (uploads audio/video) and stays blocked on HIGH regardless of consent. Policy lives in src/carrel/policy/sensitivity.py (NETWORK_TOOLS). Decision 2026-07-10.

Web pages

defuddle (local default, smart content extraction) > markitdown (fallback if defuddle not installed)

Key Design Rules

• Cloud consent: --tool <cloud-tool> IS consent. No separate --cloud flag. Profile cloud_consent controls auto-routing only.
• Idempotency: SHA-256 source hash in output frontmatter. Re-run = skip. --force to overwrite.
• Filesystem: Write only inside vault. Read from anywhere.
• Subprocesses: asyncio.create_subprocess_exec (never shell=True). Configurable timeouts.
• No AI imports: Core library is deterministic. AI lives in the transport/skill layer.
• Router validation: Routers validate tool+input combinations, not just enum membership. --tool gemini on a local file or --tool coli on a YouTube URL is rejected with an actionable error. The router is the validation boundary — transports trust it.

Scheduled Automation

Carrel can run unattended through Claude scheduled tasks (the Scheduled page; Cowork is now part of the regular Claude app) or a credential-dependent claude -p scheduler. The automation skill defines the contract; /carrel-automate configures the profile, and carrel vault automation-prompt generates the prompt separately. In Claude, use the Scheduled page or /schedule in a conversation. A task that needs a local vault requires the connected folder and Claude Desktop availability when local access is needed.

• Trust levels: Advisory (suggest only) → Consultative (propose, researcher approves) → Delegated (act on new items, experimental) → Partnership (reorganize, experimental)
• AutomationConfig in models.py: per-capability booleans, trust level, model, schedule, review cadence
• Session-start hook: surfaces morning briefs, active plans, pending decisions/approvals (gated on _meta/briefs/ existence)
• Generated prompt: _meta/automation-prompt.md — per-researcher, uses vault detection (no absolute path)
• Two-track sync: environment.json + vault CLAUDE.md must both reflect automation preferences

Commands: /carrel-batch (sequential file processing), /carrel-automate (configure), /carrel-mirror (research self-portrait)

Collaboration (v0.5.1)

/carrel-share generates a vault-specific handbook for an incoming collaborator (RA, co-author, lab member) by synthesizing friction log, capability log, reflections, configured tools, sensitivity rules, and active threads. Distinct from /carrel-setup (new researcher's own vault) and from Claude Code's /team-onboarding (generic Claude Code usage tips). Output: _meta/handbook/[YYYY-MM-DD]-for-[name].md.

Skill: collaborator-onboarding. Profile fields that drive it: collaborators: bool, team_context: str | None. Asked in setup interview; surfaced in Phase 9 handoff when collaborators == true.

Model Teammates (v0.8.1)

/carrel-teammates brings Codex (ChatGPT), Gemini, and Kimi into Claude Code via community plugins (openai/codex-plugin-cc, thepushkarp/cc-gemini-plugin, linxule/kimi-plugin-cc). Profile field model_teammates: dict[str, ModelTeammateStatus]. Skill: skills/model-teammates/SKILL.md. Spec: planning/specs/013-model-teammates.md.

Vocabulary: teammates ≠ Claude-side agents/ (setup-interviewer, research-partner) ≠ human collaborators (co-authors/RAs via /carrel-share).

Knowledge Wiki

Optional synthesis layer: agent-maintained entity/concept pages that compound knowledge across sources. Adapted from Karpathy's LLM Wiki + Hermes Agent.

• Trust-gated: wiki:propose drafts an exact batch, wiki:apply-approved applies that batch at Consultative trust, and autonomous wiki:write requires Delegated trust.
• Folder mapping: papers/, transcripts/, inbox/ = source layer. wiki/ = synthesis layer. No separate raw/.
• Operations: ingest, query (trust-gated filing), lint. See skills/knowledge-wiki/SKILL.md.
• Dialogue: researcher callouts (> [!researcher]) on wiki pages — agent reads, never overwrites. Wiki = field's voice; notes/ = researcher's voice; gap = contribution.
• Ephemeral agent: log.md includes reasoning per decision (cross-instance handoff). Lazy orientation (full context only for writes).
• Provenance and quality: optional confidence/contested/contradiction signals, source-body digests, and paragraph source markers on 3+ source pages.
• Automation: wiki_maintenance in AutomationConfig. At delegated trust, briefs include quality/drift counts, per-file revert instructions, and one synthesis insight.
• Model fields: wiki_enabled, wiki_preference, wiki_proposal_deferred_until on ResearcherProfile. Read by Claude, not code-enforced.
• Detection: check wiki/SCHEMA.md existence (not just wiki/). Upstream in capability-registry.md.

Feedback Loops

Interview preferences flow into two systems that must stay in sync:

• environment.json → CLI router (structured, mechanical). The CLI reads cloud_consent, sensitivity, etc.
• Vault CLAUDE.md → Claude's judgment (narrative, contextual). Claude reads this every session.
• setup-state.json (.carrel/setup-state.json, added v0.5.2) → tracks last_completed_phase so /carrel-setup can pause and resume. Written by carrel vault init (initial: phase 4); managed via the carrel setup-state CLI (added v0.5.3 — advance --phase N, complete, show, reset); completed_at set at phase 9. Hook surfaces a resume prompt if paused. The SetupState Pydantic model enforces phase ∈ [4,9], semver version, ISO completed_at, and the mutual-implication invariant phase == 9 ⟺ completed_at is set.

The setup SKILL (Step 5) instructs Claude to write a personalized CLAUDE.md from the interview. When preferences change, update BOTH files. The session-start hook surfaces preferences so Claude has immediate context.

Architecture Normalization (v0.9.0)

Spec 014 (CC-only re-scope) normalizes the three-layer rule for 7 high-value command paths. Deterministic file I/O moved to carrel <subcmd>; orchestration moved into skills; slash commands shrunk to thin !carrel <subcmd> ${ARGS} wrappers.

• Deterministic CLI surfaces: vault {feedback export, mirror, reflect-log, share generate, automation-prompt}, batch {convert, transcribe}, migrate apply, and automate configure. Skills retain interviews, synthesis, approval, and scheduling guidance.
• 7 wrapper shrinks (564 → 35 body lines): /carrel-{feedback,migrate,mirror,reflect,share,batch,automate}. Convention: ${ARGS} (skill-constructed) vs $ARGUMENTS (raw user input) documented in commands/CONVENTIONS.md.
• Skill enrichments absorbed the freed orchestration prose: automation (configuration interview + explicit prompt lifecycle + Claude Scheduled/claude -p scheduling + unattended contract), convert+transcribe (pre-batch routing), collaborator-onboarding (mode + sensitivity tiers), self-improve (migrate orchestration), research-partner (mirror 5-dimension synthesis), new session-reflection skill (reflect + feedback read/write symmetry).
• 3 CC plugin feature adds: marketplace metadata (keywords, category, license, repository); UserPromptSubmit hook (hooks/inject-context.js) for per-turn vault context; PreToolUse Bash matcher (hooks/sensitivity-gate.js) for sensitivity ask-gate before cloud subprocesses (bypass via # bypass-gate comment).

Spec: planning/specs/014-cc-plugin-v090.md. Cross-CLI port (Codex + Kimi) parked as future work; investigation artifacts preserved.

Version & Migration

Plugin version is tracked in .carrel/plugin-state.json in each vault. The /carrel-migrate command compares this against the plugin's current version, assesses the environment, and suggests improvements.

Migration files live in migrations/ with a registry.json index. Each migration is a markdown file describing what's new, automatic steps, and manual steps. Add new migrations when releasing breaking changes or significant features.

When bumping the plugin version in .claude-plugin/plugin.json, also update .claude-plugin/marketplace.json, pyproject.toml + src/carrel/__init__.py, and the cross-host manifests (plugins/carrel/plugin.json, .github/plugin/marketplace.json, kimi.plugin.json, plugins/carrel/kimi.plugin.json) to match — tests/test_version_consistency.py checks all of them, and add a migration file if the update affects the user's vault or config. The check-version.js hook module reads plugin_version from plugin-state.json and is wired into the session-start hook.

Gotchas

• markitdown (Microsoft's library) is the non-PDF converter — NOT the old MCP tool names
• defuddle is the web capture tool — much better than markitdown for URLs
• Install constants are centralized in env/install.py — don't duplicate
• coli install uses bun add -g @marswave/coli (not npm)
• gws (Google Workspace CLI) requires Google Cloud project + OAuth — high friction setup, see references/gws-setup-guide.md
• Cheat sheet regeneration: carrel vault cheatsheet --vault <path> --force (added v0.5.2; the legacy generate-cheatsheet.js was removed). Renderer beefed up in v0.5.3 with configured-tools matrix, common workflows, and next steps sections.
• youtube-transcript-api >= 1.0 uses .fetch() not .get(), returns objects not dicts
• Wiki preference fields (wiki_preference, wiki_proposal_deferred_until) are on the Pydantic model but read by Claude via skill instructions, not enforced by hooks — consistent with all carrel preferences
• Cross-platform (v0.7.0): all Tier 1 tools work on macOS, Linux, Windows. Install paths are platform-keyed in env/install.py; audit detects the platform in audit.py; decision-tree and cheatsheet render OS-aware.
• Setup-state changes go through carrel setup-state (added v0.5.3) — never edit .carrel/setup-state.json by hand. The CLI is the validation boundary.
• Sensitivity gate (v0.7.0): the full 16-row (sensitivity × cloud_consent × requested_tool) matrix is implemented in src/carrel/policy/sensitivity.py:select_tool. HIGH blocks cloud regardless of consent (gws included — Google API calls send data). MEDIUM requires explicit --tool <cloud> to route to cloud. LOW defaults local-first; cloud_consent=True auto-routes when local unavailable. consent.py:resolve_cloud_consent is now a thin backward-compat wrapper around policy.sensitivity. Use --explain on carrel paper convert / transcript create / google export to see the routing decision + rationale without executing.
• Vault writes go through safe_path.safe_vault_join (v0.5.4) — resolves the path and rejects anything escaping the vault root. Used by all filers + scaffold + google export.
• Trust enforcement (v0.6.0): writes that require Consultative+ trust now go through carrel trust check <action> — see spec 008-trust-enforcement.md and carrel trust list --vault . for the action matrix. The check is the single boundary; never bypass it. One documented exception (decision 2026-07-10): carrel automate configure permits an Advisory→Consultative bootstrap without an approval artifact on disk — the skill's interview-approval flow is the contract, and the runtime validates and persists that single transition in one write. It is by design, not a hole: a fresh Advisory profile may transition only to Consultative, never directly to Delegated/Partnership. See automate.py bootstrap comment, skills/automation/SKILL.md, and skills/carrel/references/contracts/trust-levels.md.
• Environment drift (v0.7.0, spec 006): carrel env validate --vault . is the full validator and carrel env fix --safe --vault . applies deterministic repairs to .carrel/environment.json only. Hook runs env validate once per 24h and surfaces /carrel-fix when drift needs review.
• Profile sync (v0.7.0): _meta/my-environment.md and _meta/automation-prompt.md are now deterministic (regenerate via carrel vault dashboard|automation-prompt --force). Vault CLAUDE.md uses HTML-comment markers (<!-- carrel:field -->value<!-- /carrel:field -->); carrel vault check-sync still exists for marker-only inspection.
• Wrapper convention (v0.9.0): 7 slash commands are now thin !carrel <subcmd> ${ARGS} shells (no body prose). ${ARGS} = skill-constructed typed-flag arg list; $ARGUMENTS = raw user input. Documented in commands/CONVENTIONS.md. The other 8 commands keep their existing shape; shrink later if they accumulate orchestration prose. tests/test_command_wrappers.py enforces the structural template for the 7 wrappers.
• Hook debugging (v0.9.0): inject-context.js (UserPromptSubmit) and sensitivity-gate.js (PreToolUse Bash matcher) silent-fail on any error path so they don't disrupt user turns. Set CARREL_HOOK_DEBUG=1 to enable [carrel:<hook-name>] <reason> stderr logging when diagnosing why a hook isn't firing. The sensitivity gate also rejects (passes through silently) when the command contains shell control characters (;, &, |, <, >, `, $) — use # bypass-gate in those cases.
• Mirror filename (v0.9.0): carrel vault mirror --write writes to _meta/mirror/<YYYY-MM>.md (monthly, idempotent same-month). Reflect-log writes to _meta/reflections/reflection-<YYYY-MM-DD>.md; feedback-digest writes to flat _meta/feedback-digest-<YYYY-MM-DD>.md. These three paths form a read/write loop (feedback-export reads reflections, mirror reads reflections + friction-log + capability-log) — session-reflection skill owns reflect + feedback; research-partner skill owns mirror synthesis.
• Gemini transcription model (2026-07-10): default gemini-3.5-flash (GA), overridable via CARREL_GEMINI_MODEL. The Interactions response parse walks ALL output items (thought-items-first responses are handled) — don't revert to output[0][...] indexing.
• Corrupt-profile recovery (2026-07-10): vault init on UNPARSEABLE JSON backs the file up to .carrel/environment.json.corrupt-<timestamp> and proceeds fresh (loud warning); parseable-but-invalid profiles hard-fail with an env validate → env fix --safe hint. Consent repairs are asymmetric by design: auto-repairs may only NARROW consent ("false"→false auto-fixed; "true" defers to the human).
• Feedback digest headings (2026-07-10): vault-relative paths with the structural sweep prefix (_meta/<sweep-dir>/) never redacted and every user-named segment + basename always redacted — no codename leaks via paths, no structural garbling. Identical in both runtimes; parity-asserted.
• carrel-skill repo is publish-only: regenerated by scripts/sync-skill-repo.sh --push (dry-run default, refuses dirty trees). Edit skills/carrel/ here, never the standalone repo.

Capability Absorption

Carrel grows by absorbing capabilities from the ecosystem (skills repos, MCP servers, CLI tools) and by learning from what researchers actually need. The self-improve skill owns this process — see skills/self-improve/SKILL.md for the full evaluation criteria, absorption process, and tracking mechanism.

One-plugin policy: researchers install Carrel, everything works. No companion dependencies. We absorb and curate, not delegate.

Registry: skills/self-improve/references/capability-registry.md tracks what's been absorbed, from where, and when to review upstream.

Relationship to ItDepends

Carrel is the on-ramp to ItDepends (abductive research agent). Same Python + pydantic + typer + rich stack. Carrel modules may become ItDepends components (profile → Mneme, vault papers → Scholia).

Spec & Review Process

Each spec gets adversarial reviews before implementation. Default reviewer set: Codex (deep adversarial pass) + Kimi (independent second-pair-of-eyes) + a feasibility/architect pass. Some specs go through multiple rounds. Spec "Open Questions" must be locked (decision + rationale) before delegated implementation.

Full work log — every spec, review, report, research artifact with one-line summary — lives in planning/README.md. Don't duplicate it here.