Istara — Agent Contract

Canonical instructions for every coding agent (Claude, Codex, pi, others). Claude-specific operating rules live in CLAUDE.md. Keep this file short: facts here must be verifiable in the repo; move narrative to Tech.md or docs/architecture/.

Reference map (read on demand, not up front):

1. Istara Research Spine Contract

Istara is a research system. Every product feature that ingests, creates, processes, retrieves, summarizes, validates, visualizes, routes, promotes, or reports user research data is an extension of the same research-validity spine:

Sources -> Evidence Units -> Independent Multi-Model Atomic Extraction + Open Coding -> Reliability + Grounding -> Reconciliation -> Accepted Atoms/Nuggets -> Facts -> Insights -> Recommendations -> In Review -> Human-Approved Done -> Reports.

Atomic Research is not a pre-validation summary layer. Bias reduction happens before trust: model output can create candidate/provisional artifacts, but no feature may treat nuggets, facts, insights, recommendations, design decisions, tasks, or reports as reportable until the source-grounded coding, reliability, reconciliation, and Done-task gates have accepted them. Evidence units come from raw source spans, not synthesized nugget prose, unless exact source spans are preserved and the artifact remains provisional until validated.

This spine is not optional and not limited to Findings, Tasks, or Reports. It applies to skills, task creation and execution, ReAct/tool calls, chat, documents, interviews, surveys, AURA-style research, integrations, deployments, interfaces, autoresearch, self-evolution, RAG/GraphRAG, compute donation, benchmarks, simulations, and any future feature that touches research data.

Before changing any feature, use Compass Forge impact output as a starting map, then follow dependencies and feature relationships until you know whether the change touches the research spine. If it does, the feature must enter or respect the pipeline in docs/architecture/research-validity-contract.md. Do not treat parallel data paths, raw finding creation, synthetic benchmark shortcuts, or feature-specific objectives as acceptable substitutes for the spine unless the path is explicitly unit-scoped, non-research, or documented as a governed exception.

If a feature currently bypasses the spine, classify it as architecture debt and either fix it in scope or report it explicitly. Never describe the system as fully aligned while any research-data path bypasses evidence units, coding, reliability, reconciliation, human review, route evidence, or Done/report gates.

Code markers to preserve: research_spine_eligible: False (transcription, file processor, validation) and can_bypass_research_spine: False (autoresearch_engine.py, self_improvement_policy.py). Guard tests: tests/test_research_validity_contract.py, tests/pi_production/test_w3_research_spine.py, test_scenario_research_spine.py, test_research_spine_donor_routing.py.

Self-Improvement Governance Contract

Self-improvement exists only to improve the Research Spine. Telemetry observes, ReasoningBank stores process lessons, Memento Skills stores validated skill memory, Autoresearch runs sandboxed experiments, Meta-Hyperagent proposes project-scoped variants, and Self-Evolution applies only governed promotions. RAG/BM25 retrieves exact evidence, GraphRAG synthesizes and traces dependencies, Prompt-RAG adds supporting context, and LLMLingua compresses only when protected protocol/codebook/gate/schema blocks remain intact.

None of these systems may create report evidence, silently rewrite protected methodology, weaken authorization, mutate global process state from project-scoped evidence, or learn strong positive skill/model signals from raw tool success. If a self-improvement path touches research data or process policy, it must preserve project scope, route/evidence handles, verification state, governance status, and Research Spine gate status. The durable contract is docs/architecture/self-improvement-governance-contract.md.

2. Blast-radius protocol (Compass Forge CLI)

Use the CLI (compass-forge); the MCP server is optional and often unavailable. Graph queries are intelligence code-graph|graph-query|impact — there is no graph command. Spec ids in old docs are unreliable; confirm with compass-forge spec.

# before editing
compass-forge status
compass-forge index status            # stale -> compass-forge index refresh
compass-forge agent-brief "<request>"
compass-forge intelligence impact --path <file> [--symbol <name>]
compass-forge intelligence test-impact --path <file>
compass-forge gate before
# after editing
compass-forge index refresh && compass-forge gate after

Impact output is the starting map, not the answer. Follow dependencies until you know whether the change touches the spine, auth, or a cross-surface contract. Attach command, gate, and test evidence to the CF task before finishing. Security-sensitive paths are listed in recipes/istara-main/recipe.toml.

Cross-surface coupling to check on every change:

3. Branches, CI, promotion

• Default: work lands on testing (feature branches → PR into testing). ci.yml runs on push/PR to main, staging, testing; release-gate aggregates all jobs; qa-artifact.yml builds the QA image on testing pushes.
• main changes through promote-testing.yml (manual, testing-promotion environment, source_sha must equal origin/testing HEAD, green checks + QA artifact evidence, never auto-merges), or through a PR into main that the owner explicitly authorizes (as for #44–#46 and #36 on 2026-09-26). Such a PR still needs every required check green, and in the same session testing is brought up to main with a sync PR so the two trees agree. Promotions are squash merges, so commit counts between main and testing are not meaningful — compare trees (git diff --stat main origin/testing).
• main protection: 17 required contexts (testing/required-checks.json, checked by scripts/check_required_checks.py), linear history, no force-push.
• Pushes to main trigger sync-staging.yml (force-syncs staging), pages.yml (docs site), build-installers.yml (releases), badge-sync.yml, scorecard.yml.
• Never push to main, force-push, delete remote branches, or run git gc --prune=now without the owner (unreferenced certification SHAs must survive).

4. Releases, packages, website

• Release: scripts/prepare-release.sh [--bump|<ver>] (integrity, CI governance, harness, security readiness, security benchmark, production rehearsal, then set-version.sh). build-installers.yml builds macOS/Linux/Windows Tauri installers, latest.json for the updater, and a v<CalVer> GitHub Release.
• VERSION must not fall behind the latest tag (scripts/check_version_drift.py, PRs into main): run scripts/set-version.sh --bump in the promotion. The Homebrew tap updates only when the HOMEBREW_TAP_TOKEN secret exists; macOS is ad-hoc signed until APPLE_* secrets exist.
• Known drift: no GHCR images are published; wiki/ is not deployed.
• Website: docs/features/ → python scripts/feature_docs.py --seed-missing --generate-site --check and pytest tests/test_feature_docs.py -q → GitHub Pages via pages.yml.

5. Containers

Protected QA Containers and Databases (Never Delete)

The Mac Studio QA host keeps containers that hold research results from testing and multi-model runs used for consultation. They carry the never-delete-official- name prefix and must never be deleted:

• never-delete-official-istara-qa-readiness5-20260910-qa-backend — QA backend run data (DB in tmpfs /tmp/istara-qa.db, feature tree in /app/data).
• never-delete-official-w3-live-backend, never-delete-official-w3-live-backend-fix — W3 candidate live runs (istara-w3.db + run artifacts in /app/data).
• never-delete-official-w4-live-pi, never-delete-official-w4-live-legacy — W4 long-horizon multi-model telemetry (host dataset at ~/w4-scratch-20260910/data).

Rules:

1. Never run docker rm, docker container prune, or docker system prune against these containers, and never delete their databases or images.
2. Their data surfaces are tmpfs (in RAM): before stopping any protected container, snapshot /app/data and /tmp/*.db* to ~/never-delete-official-data/<container>/<UTC-timestamp>/ on the host. Snapshots are permanent records — never delete, move, or prune them.
3. Code updates happen by renewing the codebase with new commits only: rebuild images from repo HEAD into a new QA run (QA_RUN_ID=<branch>-<date>, unique project per docker-compose.qa.yml contract). All existing databases and data snapshots for all features must be kept across redeploys — a redeploy never erases prior run data.
4. The W4 multi-model telemetry dataset at ~/w4-scratch-20260910/data (istara-w4-pi.db, istara-w4-legacy.db, run JSONs/logs) is protected research data: never delete, move, or prune it.

Protected Local Artifact Folders

LLMs/ and Model_Finetuning/ are local, gitignored model/training artifact folders. Never delete, prune, move, or clean them during agent work.

6. Full UI Testing Suite Contract

Every novel feature, sub-feature, addition, or behavior change must ship with coverage in the container-first user-journey suite (tests/simulation/ scenarios + tests/real_user_benchmark/ where long-form/team/donation flows apply). A feature is not done until the suite drives it the way a real user would, through a container, with dated verdicts.

When adding or changing product behavior, the author must:

1. Add or extend a Playwright scenario in tests/simulation/scenarios/ (registered in lib/scenario-registry.mjs) that performs the feature's real browser acts — navigate, click, fill, upload, send — not API calls with a screenshot attached. API-behind-browser steps must be labeled as such in the scenario.
2. Cover the matrix: roles (admin/researcher/viewer/stranger where auth-adjacent), light/dark, 375px reflow, keyboard Tab + visible focus, and loading/error/empty states. Mutations use synthetic data only — never golden data, no destructive ops.
3. Record the verdict in the scenario's summary plus the coverage map (tests/simulation/lib/scenario-registry.mjs and any coverage-matrix.json): pass/fail with date, screenshots/HAR paths, and the QA lane used (docker-compose.qa.yml ui profile, loopback publish only).
4. Extend the long-form layers when the feature touches them: team/role flows → tests/real_user_benchmark/lib/persona.mjs; donated-compute/model paths → donor/model-management probes; Research Spine paths → spine probes (gates stay non-bypassable, evidence stays provisional until Done-task acceptance).
5. Keep it green without live dependencies: new scenarios must pass in the credential-free lane or declare their live requirements (donors, models, third-party keys) and fail closed with not_runnable — never silently skip, never fabricate.
6. Attach suite evidence as Compass Forge command evidence (scenario command + verdict summary) before finishing the feature's tasks, and update TESTING.md/testing/TEST_HISTORY.md when the suite topology or release-relevant behavior changes.

Stale scenarios are architecture debt: if a feature change breaks a scenario's selectors, copy, or flow, updating that scenario is part of the feature — not a follow-up.

Suites and lanes (commands in TESTING.md):

• Broad suites (run the ones your change touches; --engine pi|legacy where chat is involved): all menus/views (09, 90 — every sub-tab, both themes, roles — plus test_phase9_broad_24view_sweep.mjs), agentic chat and real work (05, 12, 17, 21, 48, 70–71, 76, 79), model ensemble (35, 37), research spine (07, 16, 47, 58), security/auth/2FA (32, 42, 56, 64, 67, 68, 74), compute donation (34 + tests/petals_bridge/), voice (77, 78), long-form (npm --prefix tests/real_user_benchmark run probe:deep).
• CI lane is credential-free: ui-journeys runs only testing/ui-journeys.smoke.json + proven-extra.json against the QA ui profile with the provider stub. Scenarios needing live models/donors declare it and fail closed with not_runnable — never skip or fabricate.
• Desktop: CI only runs cargo check in desktop/src-tauri; there are no desktop UI tests. State that gap rather than claiming desktop coverage.

7. Telemetry and benchmarks

• Current state: custom spans (backend/app/core/telemetry.py, telemetry_export.py → telemetry_spans, agentic_usage_rows, <base>_spans.jsonl); run aggregation via qa/scripts/w4_aggregate_telemetry.py (per-turn/tool/model latency, tokens, cost, errors; prompts stored as sha12 + length only). No OTLP export and no gen_ai.* attributes yet.
• New or changed LLM/agent instrumentation must follow the OpenTelemetry GenAI semantic conventions: spans chat/invoke_agent/execute_tool; attributes gen_ai.operation.name, gen_ai.provider.name, gen_ai.request.model, gen_ai.response.model, gen_ai.usage.input_tokens/output_tokens, gen_ai.response.finish_reasons, gen_ai.conversation.id; eval results as gen_ai.evaluation.*. Map onto the existing span tables rather than forking a second store, propagate trace context across pi-runtime ↔ backend ↔ relay, and pin collector images.
• Benchmark results are the engine scorecard: refresh and commit them after agentic engine changes (comparison-Istara-pi/, qa/runs/<run-id>, artifacts/, testing/TEST_HISTORY.md), always keyed by commit SHA, engine, and model.

8. pi (earendil) runtime

• pi-runtime/ (Node worker; wire contract pi-runtime/PROTOCOL.md) wraps @earendil-works/pi-ai + @earendil-works/pi-agent-core, exact-pinned and kept in lockstep with labs/pi-replacement. Backend adapter: backend/app/core/pi_runtime/.
• Upgrade routinely with the runbook: python scripts/pi_bump_diff_proof.py proof <X> --report ... → install both surfaces with exact pins → python scripts/generate_pi_catalog.py → classify every changed surface intended-upstream or istara-fix → pi_bump_diff_proof.py verify → update EXPECTED_PINS in tests/pi_migration/test_version_provenance.py.
• Verify: cd pi-runtime && npm ci && npm test, pytest tests/pi_compat tests/pi_migration -q, then a pi-engine UI chat scenario. New pi capabilities go through the adapter seams (seams.py, tools.py), never around the spine or tool-authority checks.

9. Safety rules

Live LLM and Model Loading Safety

Do not start live backend/frontend servers, send chat-completion probes, or trigger model loading without explicit user permission. Passive LLM status/discovery checks must stay passive. Active model loading belongs only on deliberate request paths and must be bounded to one configured target so agent work never loads multiple heavy models at once.

Use gitignored environment files, process environment, or macOS Keychain for live LLM endpoints and tokens. Never commit or paste private LLM server URLs, tokens, connection strings, or endpoint fingerprints that could identify a private server.

Security Benchmark Gate

Auth, authorization, session, WebAuthn, connection string, pooled compute, MCP, webhook, LLM-provider, autoresearch, self-evolution, and agentic-memory changes must run the tracked security benchmark:

python scripts/security_benchmark.py --fail-on-threshold

Update security/control_matrix.json, security/SECURITY_BENCHMARK.md, and tests/test_security_benchmark.py when a security control, evidence path, standard version, or trigger pattern changes. Attach the scorecard output as Compass Forge command evidence before finishing security-sensitive tasks.

Repository hygiene

• Never commit the literal checkout path (scripts/public_repo_quality_audit.py).

Skills library

This project has access to a shared, vendor- and model-neutral Agent Skills library (the open Agent Skills format — a folder per skill, each with a SKILL.md). It lives at:

/Users/user/Documents/Skills

Before starting a task, check whether a skill there applies, then use it:

1. Discover — list the subfolders of the library and read the description: in each SKILL.md. Match your task against those descriptions.
2. Load — read the full SKILL.md whose description fits.
3. Execute — follow its instructions, gates, and output style for the whole task.
4. Deepen on demand — open a skill's references/, scripts/, or assets/ files only when its SKILL.md points you there.

A skill is instructions, not code — reading its SKILL.md and acting on it is invoking it. No special runtime is needed. For the full contract, per-harness wiring, and the standards for adding or editing skills, read /Users/user/Documents/Skills/AGENTS.md. To add, edit, or re-install this library, load the /Users/user/Documents/Skills/skills-librarian skill.

If you ever register a skill into a harness's own skills dir (e.g. ~/.codex/skills/, ~/.claude/skills/), symlink the library folder — never copy it — and repair any existing copies with /Users/user/Documents/Skills/skills-librarian/scripts/sync-harness-skills.sh.

Available skills (auto-generated on install — re-run the skills-librarian install to refresh):

Compass Forge Agent Workflow

Project root: <REPO_ROOT> (repo-relative; the literal checkout path must never be committed — see scripts/public_repo_quality_audit.py rule machine_checkout_path) Recipe: istara-main Runtime: Rust-only (compass-forge mcp).

1. Call forge.status, then forge.agent_brief for the user's request.
2. For meaningful changes create, clarify, plan, and task a durable spec.
3. Use impact, graph, model, zones, context packs, and a role-specific work order before editing.
4. Run gates before and after; attach command, gate, and review evidence.
5. Preserve independent blind review: freeze the reviewer sheet before revealing implementation evidence, then reconcile.
6. Never invoke a legacy runtime or silently mutate global configuration.

Execution mode (owner rule, 2026-09-18)

This overrides anything else in this file about subagents or review loops.

• No subagents: the session that takes the work does it, and delegates to no other agent.
• No re-review loops: at most one independent review, and only when the owner starts it.
• Self-verify with evidence: regression red before and green after, exact commands with quoted output, the last check run after the last edit.
• Flag Full-risk work (security, data, trust boundaries, cross-machine state) to the owner as self-verified only.
• A pipeline that launches agents runs only when the owner starts it.
• Model ids go only in a plan's ledger and decision log.