Research Results Synthesis and Explanation

Public showcase attribution

This is a Gavin/Hermes local workflow skill, substantially shaped by Gavin-specific research projects, artifact conventions, and repeated claim-hygiene corrections. It is not presented here as a stock upstream Hermes package skill.

Use this skill when the user asks to explain, interpret, summarize, or review results from a research project, especially when the results are stored across local notes, CSVs, markdown reports, plots, generated decks, or run directories.

This is for research interpretation, not just file summarization.

Project-specific references

• For Value Action / Phase 3.1 ConsVal artifact curation and tier explanations, see references/value-action-consval-artifact-scope-and-tiering.md. Use it to avoid conflating top-32 triage artifacts with full-support dense artifacts, to explain lenient/default/strict percentile bars, and to sanity-check order-direction/order-robust labels before answering.
• For nonlinear SAE paper-readiness / old-theory questions about false positives, false negatives, support-selection errors, angular distance, mean-direction divergence, parameter-matched wider-linear 60-row maps, or whether ResNet solves a linear-encoder pathology, see references/nonlinear-sae-support-selection-pathology-metrics.md. Lead with the exact readiness/CUDA answer, then frame the claim as “nonlinear encoders improve sparse support selection under strict budgets,” not “nonlinear SAEs are universally better.”

For terse Discord project status follow-ups (e.g. “again”, “is it still running?”, “check the jobs”), use the fast GPU/process/log/progress-file recipe in references/quick-gpu-status-checks.md: infer the project from the channel/router, map GPUs→PIDs→logs via nvidia-smi, ps, and /proc/<pid>/fd, verify liveness with log mtime/progress JSON rather than a single utilization sample, then answer with a compact verdict and small aligned fenced-text table.

Interrupted Long-Run / Partial-Result Recovery Pattern

When a long research evaluation/training process is reported as completed with a signal exit (for example exit 143/SIGTERM) or otherwise stopped mid-run, do not assume failure or restart the full job. Use a recovery-first workflow:

1. Inspect the run log tail and searchable milestones to identify whether it stopped at a Python exception versus external termination/interruption.
2. Count completed output artifacts by eval/task family and compare them against the manifest rows or planned workload.
3. Identify the first missing row(s) and whether partial eval families lag behind others (e.g. one eval has 58 outputs while two have 60).
4. Create small resume manifests for exactly the missing prefix/next tranche instead of rerunning already-completed rows.
5. Relaunch a bounded continuation on an idle/appropriate GPU, reusing the same output root when the runner skips existing artifacts by default.
6. Verify the new process is actually using the intended GPU and record the process id/log path in the user-facing status.

Pitfall: for multi-eval manifests, interruption can leave eval families at different offsets. Resume the lagging eval(s) for the last partially completed rows separately before continuing the full eval set.

Behavior / Format QA Gates Before Activation Collection

When synthesizing mechanistic-interpretability dataset QA that decides whether to collect answer-token/answer-marker activations, treat the behavior-only format gate as a hard prerequisite, not a side diagnostic. Verify the scored-row count, category rates, grouped breakdowns, and representative generations before recommending activation collection. If a large fraction of rows fail to start with the intended marker, explicitly say not to run answer-marker activations on that render and identify the best surviving prompt-format lane plus remaining failing controls. A good compact readout is: total rows scored, first-token-valid rate, critical bad-first-token rate, failure categories, artifact paths, best/worst format lanes, and the next repair. See references/answer_format_qa_gates.md for a reusable checklist and Value Action example.

The goal is to reconstruct: what was tested, what changed, what the numbers mean, what is causally supported, what is merely correlational, and what claim is now safe to make.

Trigger Phrases

Load this skill for requests like:

• "explain the results comprehensively"
• "what do these results mean?"
• "summarize the experiment outcome"
• "interpret the plots/CSV/report"
• "what claim can we make from this?"
• "write a status memo from the latest artifacts"
• "review the run and tell me the takeaway"
• "adapt this benchmark / what metrics are included?"
• "set up a checkpoint sweep / estimate benchmark runtime or cost"

Workflow

1. Find the right project and artifact root.
   • If the user did not name a project, infer from the Discord channel, current working directory, project mapping notes, recent chat summaries, and modified/latest artifacts.
   • Prefer project-local RECENT_CHAT.md, README.md, summary/, analysis/, results/, reports/, and presentation folders.
   • For Gavin's research projects, project mapping notes may live under /home/<user>/work/hermes-agent-work/.
2. Read synthesis docs before raw tables.
   • Start with recent-chat notes, final synthesis markdown, analysis reports, plans, manifests, and deck READMEs.
   • Then inspect raw CSV/JSON tables to verify counts, means, top rows, and edge cases.
   • Do not rely on memory for counts or arithmetic; compute or read them from artifacts.
3. For live experiment status, separate running state from completed results.
   • Check live processes/GPU/tmux first, then progress files for durable done/failed counts, then logs for the current in-flight run, then artifact mtimes.
   • Also check user systemd services when runners were deliberately launched outside Hermes background processes, e.g. systemctl --user list-units 'sae-*' --all, and map PIDs back to the project root.
   • Use runner docstrings/configs to recover intended sweep size and ordering.
   • Treat status: "running" in a progress file as only a durable hint until corroborated by a live process/GPU/log; stale running entries should be reported as stopped/stale, not active.
   • For detached/systemd/nohup launches, a stale stdout log alone is not enough to call the job dead; map the live PID to its command/project root, check GPU utilization, and inspect output/checkpoint mtimes.
   • Ignore buffer-refill progress bars when estimating training progress; use the latest true training tqdm line and progress.json elapsed times.

• For quick x/100 progress checks, reconstruct the denominator from the runner's plan/config rather than len(progress.json), because progress files may only contain touched runs; report done_count + current_run_fraction only when the running entry is live-corroborated, and also name the integer run index (run n/100, p%). Start with a compact count table before narrative: done / running / failed / not-started, plus the current run ID. If multiple sweeps are active, report per-sweep counts first, then a combined total only after the denominators are explicit.
• When a user asks what failed means in a progress file, answer concretely: the runner caught an exception, wrote status: "failed", skipped/continued to later rows, and usually did not create checkpoint/metrics artifacts unless verified otherwise. Immediately list the exact failed run IDs, their plan indices ([97/100] etc.), architecture/setting/sparsity values, first-line error, elapsed time if recorded, and whether expected artifacts such as sae_weights.safetensors, cfg.json, final_metrics.json, and meta.json exist. Then state runner behavior from code: e.g. if the loop skips only done rows, a retry runner will reattempt failed rows after reaching them.
• When many live SAE/ML runs suddenly fail with generic CUDA errors, distinguish a model/config failure from a GPU/NVML/driver failure before retrying. If nvidia-smi reports an unknown GPU handle or fresh Torch processes cannot initialize CUDA even on a different GPU, stop retries, mark stale running entries honestly, preserve unaffected existing CUDA jobs, and recommend retrying failed rows only after a GPU/driver reset; see references/live-sae-cuda-nvml-failure-triage.md.
• See references/live-experiment-status-checks.md for a reusable read-only workflow and answer shape, references/live-experiment-stale-running-progress.md for handling stale/detached running progress entries and incomplete progress-file denominators, and references/nonlinear-sae-auxk-live-status-20260529.md for the nonlinear SAE AuxK systemd-service / fractional x/100 example.

4. For benchmark/evaluation pipeline setup, treat launch readiness as part of synthesis.
   • Use this same skill when Gavin asks to adapt an external benchmark, evaluate many model/SAE/checkpoint variants, prepare a GPU/API sweep, inventory metrics, estimate cost/runtime, or aggregate benchmark JSON into matched comparisons.
   • Resolve the authoritative benchmark repo/version; inspect source schemas/configs rather than README summaries; record branch/commit and compatibility caveats.
   • Inventory eval families and metric names before launching. Separate included, excluded, optional-but-needs-smoke, and incompatible metrics; answer “is X included?” with yes/no first.
   • Build manifest-driven runs with one row per checkpoint/run and explicit pairing metadata (model, layer/hook, architecture, baseline family, width, sparsity/k, checkpoint/config/output paths, human labels). Avoid ad-hoc shell loops.
   • Add adapter layers instead of mutating original artifacts; smoke both representative baseline and new-architecture checkpoints before full launches.
   • Treat LLM-judge/API evals as separate batches with cost/rate-limit pilots; do not bundle them with ordinary GPU metrics unless requested.
   • Shard with runner/manifest shard flags, use independent logs, and launch long bounded runs as tracked background processes with completion notification.
   • Aggregate pairwise before claiming wins; compare matched pairs first and do not collapse metric families into a single score until directions/relevance are explicit.
   • See references/research-benchmark-pipelines.md, references/saebench-current240.md, and references/cuda-sweep-progress-eta.md for the absorbed benchmark-pipeline workflow and concrete SAEBench/CUDA ETA examples.
5. Reconstruct definitions and sign conventions.
   • Explicitly state what each option/site/token/metric means.
   • For causal metrics, state the objective and sign convention before interpreting numbers.
   • Separate target labels, model choices, generated answer formats, action spans, rationale text, and intervention sites.
6. Compute concise verification statistics.
   • Use a small script to calculate counts, percentages, means/medians, positive/negative rates, and top/bottom examples from CSV/JSON artifacts.
   • If pandas is unavailable, use Python stdlib csv, json, statistics, and collections rather than stopping.
7. Distinguish evidence classes.
   • Separate raw activation patterns, validated feature explanations, global correlational behavior, case-study examples, and causal/intervention evidence.
   • Mark which results are strong, suggestive, mixed, or likely confounded.
8. Explain as a research narrative.
   • Start with a bottom line.
   • Then cover: scope, definitions, main quantitative results, qualitative examples, causal evidence, limitations, and the cleanest claim.
   • Include enough detail for the user to trust the interpretation, but avoid dumping every table.
9. End with claim hygiene.
   • Provide a conservative claim the user can make.
   • Identify what would be overclaiming.
   • Name the next validation experiment only if useful.

Pitfalls

• For Discord status/result tables, avoid Markdown pipe tables. Gavin finds | table syntax hard to read in Discord. Render comparison tables as fenced text code blocks with aligned spaces. Keep this formatting especially for counts, run summaries, and metric comparisons.
• Keep match units explicit. When comparing experiment variants, state the key being matched and whether rows are run-level or collapsed. Example: AuxK SAE comparisons may have width + L1 + aux coefficient rows, while no-Aux standard-vs-linear uses only width + L1; do not compare denominators without explaining the extra axis.
• For Gavin's SAE beat/loss questions, use Pareto beat/loss and split tradeoffs by direction. A Pareto beat means the candidate is non-worse on both matched metrics of interest and strictly better on at least one (typically higher/equal EV and lower/equal L0, with at least one strict improvement). A tie on one metric plus a win on the other is a beat, not a tie. A Pareto loss is the analogous non-better-on-both with at least one strict worsening. Tradeoffs only mean strict EV-vs-L0 disagreement; report the direction explicitly, e.g. candidate EV up / L0 worse versus candidate EV down / L0 better, instead of collapsing all tradeoffs into one unlabeled count. If L0/sparsity is tied, EV decides the Pareto outcome: tied L0 + EV win is a beat; tied L0 + EV loss is a loss, not a sparse-only trade. Consider adding L0 ties and L0 nonworse columns when Gavin asks whether tied sparsity is being handled correctly. For the OPT-125M non-Aux core5 TopK/BatchTopK comparison, tied L0 in all 20 rows makes wins/losses pure EV outcomes; see references/nonlinear-sae-core5-resnet-topk-batchtopk-design.md.
• For nonlinear SAE percentile-normalized comparisons, compute percentiles in fair pools, not inside pairs. When Gavin asks for EV/L0 percentiles as a fair comparison layer, define a pool such as family_group × width, compute EV_pct, reversed Sparse_pct from L0, and a harmonic balanced score H2 = 2*EV_pct*Sparse_pct/(EV_pct+Sparse_pct). For matched ResNet-vs-Linear comparisons, compare ΔH2 = H2_candidate - H2_linear_baseline after matching on width + sparsity setting (+ AuxK axis where relevant). Do not compute percentiles within each 2-row pair; that degenerates to 0/100 and adds little. Percentile transforms are monotone, so clean percentile beat/loss aligns with Pareto beat/loss; the new value is ranking tradeoffs and selecting balanced EV/L0 knee checkpoints. Include exact formulas and export summary + pair-row CSVs when making slides. See references/nonlinear-sae-percentile-matched-comparison.md.
• For nonlinear SAE architecture comparison decks, keep candidate/baseline orientation stable. When Gavin is comparing nonlinear/ResNet variants against matched linear baselines, label rows and slides as ResNet vs Linear, not Linear vs ResNet; compute deltas as candidate minus baseline (ΔEV = ResNet EV - Linear EV, L0 improvement = Linear L0 - ResNet L0). When both AuxK and no-Aux variants are shown, put AuxK rows first, then no-Aux. If asked for top runs across all architecture/settings by EV and L0, use a transparent rank-sum shortlist (EV rank descending + L0 rank ascending) and state that it is not a Pareto frontier proof. For PPTX layout details and the config sanity-check pattern for apply_b_dec_to_input, see the presentation skill reference gavin-research-presentations/references/nonlinear-sae-resnet-vs-linear-pptx-comparison-decks.md.
• For nonlinear SAE fair metric-system questions, layer percentile normalization after matching and Pareto beat/loss. Percentile scores are useful for selecting balanced EV/L0 knee checkpoints, but they are not standalone win/loss evidence. First match by fair keys and compute Pareto beat/loss; then compute EV and reversed-L0 percentiles inside local pools such as family_group × width, combine with a harmonic mean, and report dead-feature percentile as a separate sanity gate. Keep all-run top lists separate from nonlinear-candidate top lists. For top N across all runs/architectures, do not rank local family×width percentile scores as if they were absolute utilities: local percentiles can violate obvious cross-family Pareto dominance (higher EV and lower L0 can rank lower than a worse raw point from another pool). Instead compute a global EV/L0 percentile score over the declared candidate universe after applying any dead-feature eligibility filter, or clearly label the local-pool score as only a within-family knee selector. Dead neurons should be a filter/sanity gate, not part of the EV/L0 ranking score, unless Gavin explicitly asks for a dead-inclusive score. For the OPT Phase 1 result and GPT-2 Phase 2 replication plan, see references/nonlinear-sae-global-ev-l0-ranking-and-phase2.md; for parameter-matched wider-linear and pure nonlinear/no-skip control planning, see references/nonlinear-sae-parameter-matched-baselines.md; for the older local-pool percentile layer, see references/nonlinear-sae-percentile-comparison-layer.md.
• For nonlinear SAE architecture-design questions, inspect implementation before inferring intent from names. Matryoshka ResNet TopK in the current OPT-125M core5 code is a shared ResNet encoder with prefix reconstruction losses, not separate nested sub-ResNets; default levels are [d_sae//4, d_sae//2, d_sae]. Current/old nonlinear_resnet_gated uses an asymmetric design: ResNet gate path plus linear magnitude/value path. For linear gated specifically, explain that the gate and magnitude are parallel from sae_in but share W_enc; it is not gate-on-main-output and not two independent encoder matrices, so the parameter delta is only vector-sized (about +2*d_sae). nonlinear_resnet_dual_gated is a valid high-capacity ablation with independent ResNet gate and magnitude/value paths, but it is not the closest direct analogue of ordinary linear gated. If Gavin asks for the “actual corrected” or “core/final” ResNet-gated design, propose/label a tied/shared ResNet-gated architecture: one shared ResNet encoder output z, featurewise gate_pre = z + b_gate, featurewise mag_pre = z * exp(r_mag) + b_mag, and acts = (gate_pre > 0) * relu(mag_pre). Do not collapse this to (z > 0) * relu(z), which degenerates to standard ReLU ResNet. Treat dual-gated as a separate dual-path ablation, not the canonical fixed gated result unless Gavin explicitly decides otherwise. For parameter-matched wider-linear or makeup-run planning, combine trainable-param counts with a rough CUDA fit screen and verify fresh PyTorch CUDA init before launching if NVML/GPU health is suspicious. When adding new custom SAE architectures for these sweeps, verify both the training registry and the inference/base SAE registry before launch; dry-run config validation alone may miss final-save KeyError failures. See references/nonlinear-sae-core5-resnet-topk-batchtopk-design.md, references/nonlinear-sae-gated-design-and-cuda-fit-check.md, references/nonlinear-sae-tied-resnet-gated-and-reboot-run-plan.md, references/nonlinear-sae-tied-gated-design-and-runner.md, and references/nonlinear-sae-custom-architecture-runner-pitfalls.md.
• For nonlinear SAE live runner launches, keep status compact and make runs resumable. Before starting additional long sweeps, check live GPU/process state and progress files, avoid historically unstable CUDA2 unless explicitly needed, and use one physical GPU per runner with CUDA_VISIBLE_DEVICES=<physical_gpu> plus --device cuda:0. Use conservative thread caps (TORCHINDUCTOR_COMPILE_THREADS=4, MAX_JOBS=4, OMP_NUM_THREADS=4) for multi-day jobs. If a runner dies or is killed, back up progress.json, mark stale running rows as failed only after live-process corroboration fails, and relaunch; these runners should skip done rows and retry failed rows. Report done/running/failed/not-started and current run ID first. See references/nonlinear-sae-custom-architecture-runner-pitfalls.md.
• When AuxK is involved, compare each method pair separately and label scope. For early standard-family AuxK pilot results, produce separate rows/plots for AuxK ResNet vs linear, AuxK ResNet vs no-Aux ResNet, and, if relevant, no-Aux ResNet vs linear, with pilot denominators clearly labeled. For expanded all-core5 AuxK sweeps, recompute exact all-core5 matches (family + width + sparsity parameter/value, with the AuxK coefficient fixed/explicit) and update all denominators from live artifacts rather than reusing old pilot counts. If only one side of a fourth comparison is partial, e.g. ResNet AuxK vs Linear AuxK, report that pair with its current matched denominator separately and do not fold it into the completed three-way PNG/table.
• Do not collapse sites. In interpretability results, action spans, pre-answer tokens, generated answer tokens, and generated rationales can mean different things. Keep them separate unless the artifact explicitly combines them.
• Do not treat generated rationales as the decision mechanism. They can be post-answer traces and should be interpreted separately.
• Do not interpret raw activation rank as semantic importance. Generic syntax, pronoun, formatting, and scaffold features can dominate raw top-k lists.
• Do not ignore answer-format confounds. If a run varies the answer format, compare choice distributions and answer-site feature stability before interpreting answer-token features.
• For final live-status checks, verify artifacts, not just progress counts. When a sweep transitions from active to complete, report live process/service state first, then compare progress.json done counts with expected final_metrics.json artifacts. A run can be marked done and have weights/meta/log completion but lack final_metrics.json; report that as a metrics-file recovery caveat rather than a training failure if logs and checkpoint files corroborate completion. If Gavin says a previously-live GPT/nonlinear-SAE tail is now finished and asks to move into analysis, re-verify the final state first, then regenerate summaries/decks from final metrics rather than relying on provisional artifacts; see references/nonlinear-sae-final-status-before-analysis.md. For nonlinear SAE makeup-sweep completion and CUDA visibility recheck details, see references/nonlinear-sae-live-multisweep-status-20260604.md. For GPT-2 pure-nonlinear no-Aux final completion after a registry/final-save fix, verify both progress.json and all 100 checkpoint artifact dirs across 2layer_4k and 2layer_8k before moving to analysis; see references/nonlinear-sae-gpt2-pure-nonlinear-final-completion-20260630.md.
• For nonlinear SAE dominance/DAME/redundancy follow-ups, start with fixed-k TopK-family rows and require ranking sanity gates before expensive downstream metrics. Gavin prefers the first dominance-aware interpretability pass to focus on BatchTopK, corrected Matryoshka TopK, and original TopK because fixed k makes ResNet-vs-Linear wins/losses much cleaner: sparsity is controlled and EV decides the row. Treat standard/gated L1-family rows as secondary/appendix unless a separate L0-binned/thresholded comparison rule is defined. Use a manifest-first sequence, but do not proceed from manifest to DAME/redundancy until dominance ranking passes basic invariants: eval L0/k matches final_metrics.json, active feature counts are plausible, sample size is sufficient for rank/quantile splits, and split-half rank/tertile stability is reported. When dead neurons or redundancy are used as “not too bad” filters in ResNet-vs-Linear tables, distinguish the side with the problem: mark a true candidate failure only if the ResNet/candidate side violates the threshold; Linear-only high-redundancy rows should be labeled as baseline issues and still compared on EV/DAME. Include standalone redundancy/dead-neuron count summaries before interpreting whether redundancy is getting better or worse. The failed v1 ranking used only 10 batches / 5,120 positions and had thresholded OPT BatchTopK/Matryoshka rows with high EV/L0 encode as all-zero/near-zero; downstream DAME and redundancy from those ranks were quarantined. Replacement should be named Dominance V2 / sanity-checked dominance ranking, use larger samples (about 50–100 batches for claim-facing ranks), checkpoint-type-aware encoding for thresholded BatchTopK/Matryoshka, and separate v2 output directories. Current DAME selection may use tertiles rather than quartiles when Gavin chooses presentation simplicity; build tertiles over active features only, but if active features are implausibly zero relative to final L0, mark the row invalid and rerun/fix the encoder rather than treating it as science. DAME score should be explained as token/sentence pairwise ranking accuracy from generated explanations, not percent of features correct. For “top 10 overall/Q1” requests, rank all runs ignoring architecture and count ResNet vs Linear unless Gavin explicitly asks for ResNet-minus-Linear advantages. For improved claim-facing DAME reruns after V2, do not rerun dominance ranking if Dominance V2 already passed sanity; clarify that DAME “collection” means gathering activation examples for explanation/eval, not ranking. Gavin's current preferred V3 design is High-dominance top50 as primary, Mid-dominance top25 as robustness, Low omitted unless explicitly requested, with 100 collection batches/checkpoint, 30 explanation examples, and up to 40 token + 40 sentence MC questions; see references/nonlinear-sae-dame-v3-high50-mid25.md. For redundancy metric choice, prefer high-dominance Top100 decoder-cosine redundancy @0.5 as the headline count/rate, plus nearest-neighbor duplicate fraction and tertile-wide high/mid/low rates as robustness; do not use raw top100 pair count alone. When Gavin asks for EV+DAME comparison with dead-neuron/redundancy included only as “not too bad” filters, treat dead/redundancy as pair quality gates rather than ranking scores, and show separate green/red beat/loss columns for EV and each DAME metric instead of collapsing the row into one verdict; see references/nonlinear-sae-ev-dame-quality-filter.md. For dominance-level distribution / top-10 or top-30 cumulative dominance questions, answer the definition first: DAME-paper dominance is mean per-input activation share, dominance_i = E_x[|z_i(x)| / sum_j |z_j(x)|], so top-N accumulation sums normalized shares after ranking features by this value. Do not describe it as raw accumulated activation magnitude or firing frequency. Also distinguish pure cumulative-dominance slides from redundancy slides that use dominance-ranked Top100 features but plot decoder duplicate counts. See references/nonlinear-sae-dominance-level-distribution-v2.md. For the full pattern and row/table presentation preference, see references/nonlinear-sae-dominance-aware-topk-metrics.md; for the 2026 tertiles/top25 DAME runner edge cases and GPT-2 interim analysis lessons, see references/nonlinear-sae-dame-tertiles-run-and-analysis-202606.md; for the v1 quarantine and Dominance V2 rerun standard, see references/nonlinear-sae-dominance-v2-quarantine-20260627.md. If Gavin asks whether to run DAME/redundancy on weak nonlinear ablations versus making ResNet DAME more robust, skip expensive interpretability metrics for variants that are not competitive on matched EV/L0, and for ResNet use a targeted DAME V3 robustness plan rather than a blind full rerun: increase high-dominance feature coverage, examples/questions per feature, and sampling/CI stability checks; keep Low-tertile claims caveated because V2 low-dominance coverage was poor. See references/nonlinear-sae-dame-v3-robustness-plan.md.
• Do not overstate small-bundle interventions. Exact Shapley over a selected bundle is exact for that bundle, not for the full feature basis.
• For Value Action public SAE/transcoder/crosscoder basis-search status, separate scientific results from job state and separate the existing benchmark from newly tried pairs. This is an RQ0-style preservation screen, not the full Keishii S/V/V3/V4 pipeline. Answer first whether a candidate is BENCHMARK/current, STRICT, STRICT_DENSE, RELAXED, running, FAIL, or BLOCKED before science; do not list pending or blocked candidates as winners. When Keishii L11 appears in the table, label it as BENCHMARK/current pair, not as a newly discovered candidate; count totals as benchmark/current, new/other screened, new strict passes, strict-dense numeric passes, relaxed candidates, and fails so Gavin does not think the current pair is being rediscovered. For near/relaxed rows, always show the exact percentage preservation plus the two corresponding scores that produced it, e.g. raw acc 0.781 → sparse acc 0.656 = 84.0%, and do the same for macroF1 and matched-N when available. Use aligned text-table columns: LLM name/size, basis repo/type/width, layer site, checkpoint size, raw acc/macroF1, sparse acc/macroF1, sparse/raw preservation, raw top-N MI, sparse top-N MI, matched-N preservation, L0/active features, and decision. Keep two bars distinct: strict/gold promotion generally needs sparse acc or macroF1 >=0.70 and >=80–85% sparse/raw preservation; Gavin may also ask for relaxed exploration around sparse acc/F1 >=0.60 and preservation >=80% (GemmaScope L12-style candidates). A roughly 30-row RQ0 public-basis screen is enough for basis-selection evidence, but not final causal/steering claims; after such a screen, promote only representative passing settings rather than every correlated L0 checkpoint. If a candidate passes strict numeric thresholds but has very high observed L0/active feature count, label it STRICT_DENSE and caveat that it may be less interpretable / less cleanly sparse than Keishii. If a candidate cannot run due gated model/auth, label it BLOCKED before scientific result, not a scientific failure. See references/value-action-public-basis-result-table.md, and references/value-action-public-basis-30-screen-20260626.md.
• For Value Action dataset redesign / value-action-gap dataset validity, require genuine two-sided value conflict and baseline specificity before strong claims. If Gavin raises concerns that current Task1/Task2 items are not controversial enough, answer that the existing dataset should be treated as a pilot/warning signal: weak Task1↔Task2 overlap may reflect prompt format, option position, context artifacts, generic moral-question features, or weak value-action mapping rather than a real value-action gap. The preferred new unit is a paired controversial value conflict: Task1 asks abstract priority between two defensible values, and Task2 presents a concrete dilemma where each action instantiates one value. Current paper-candidate target from the 2026-07-02 plan is 30 promoted value pairs × 50 concrete dilemmas = 1,500 core dilemmas, rendered into Task1, Task2 implicit-action, Task2 explicit-label, random-question, and same-format non-value prompts; A/B vs 1/2 and F0/F1/F2 answer-ending overlays are controls, not silent changes to the canonical count. Required baselines: cross-value, random A/B-or-1/2 question, same-format non-value tradeoff, normal/reversed option order, final-answer-format variants, explicit-value-label ablation, marker-format controls, and shuffled mapping. Use Codex/LLM generation freely for candidate drafting, but separate generation from critique/revision/manual validation and run behavior-only sanity checks before feature extraction. Before any answer-token SAE/transcoder extraction, run answer-format compliance QA: strict answer-token site is the first generated nonspace token; if the model emits prefixes like Option, Answer:, or The correct answer is before A/B/1/2, revise the fixed prompt ending and rerun behavior QA before collecting activations. See references/value-action-controversial-value-conflict-dataset.md and references/value-action-answer-format-qa-gate.md.
• For Value Action option-choice feature analyses, separate detection from choice-correlation and causal impact. Do not average Option1-vs-Option2 activation gaps across final-answer classes and call the result a preference feature. Report conditional option-text gaps by final answer, answer-token gaps, pre-answer probe evidence, and causal/Shapley margin effects as distinct evidence classes. Scaffold features can hit many detection metrics without being the value mechanism. When ranking Q2/Q3/Q4/Q5 metrics, state the formal formula and the comparison set; avoid rank among eight selected features as evidence except for within-bundle Q4-vs-Q5 consistency. Prefer all-candidate descriptive baselines or broader/random causal baselines. If Gavin asks whether a metric measures an option gap, answer the exact yes/no first and verify the formula before interpreting. Current Value Action S3/Q5 pre-answer probe influence is coef × unconditional mean activation, not an answer-conditional contribution gap; use S4 for coef × (mean_pre | Option1 answer − mean_pre | Option2 answer) and validate S4-selected candidates with V3/V4. See references/value-action-choice-correlated-features.md, references/value-action-option-choice-metric-baselines.md, and references/value-action-s4-v3-v4-gap-correction.md.
• For Value Action cross-value follow-up after a worked single-value slice, build the full Pool D/E feature-expansion atlas before per-value Q-metrics. Gavin corrected that the core next step is not “pick Benevolence and analyze features” but first covering all selected value families/subvalues with cross-value Pool D/E discovery, then giving every newly surfaced feature a Power-style explanation and held-out validation score. Only after the manifest has explanation/validation records for all candidate features should you run per-value Q1/Q2/Q5, select bundles, or make Q3/Q4 causal claims. Treat severely imbalanced per-family Pool D directions as discovery signals only. See references/value-action-cross-value-pool-de-atlas.md.
• For Value Action after the Power V7 worked example, run the cross-value Pool D/E atlas before single-value interpretation unless Gavin explicitly asks for a one-slice pilot. Gavin corrected that the core next step is full Pool D/E feature expansion across value families, with explanation and validation scores for newly surfaced features, before per-feature/per-value Q1-Q5 or causal analysis. Reuse Power V7's explanation/held-out domain-binned validation method; no feature should enter main claim tables without an explanation/validation record or an explicit unexplained / needs repair status. See references/value-action-cross-value-pool-de-atlas.md.
• For Value Action after the cross-value atlas is complete, choose the first Q1/Q2/Q5 focus slice by choice-balance before semantic preference. Do not automatically pick Benevolence/Universalism as the next contrast if their Option2/minority counts are too small; inspect per-family counts and prefer a non-Power slice with enough minority examples for directional metrics and Q5 probing. Check whether a cleaner answer-format variant, especially bare-digit Variant B, changes the balance before deciding. Then run candidate-universe Q1/Q2/Q5 over the atlas features, write manual_agent_feature_review.md, and only promote semantic/action features to Q3/Q4 after separating generic transition/discourse rows and scaffold/syntax controls. See references/value-action-post-atlas-qmetrics-focus-selection.md.
• For Value Action once the cross-value Pool D/E atlas is complete, the next step is cheap/site-local Q metrics, not causal tests or a deck. If Gavin asks “what is the next step?” around the completed atlas state, answer directly: compute per-value/family Q1 answer-format metrics, Q2 full action-span metrics using max activation primary plus mean/hit/count diagnostics, and Q5 pre-answer probe influence over the completed manifest; then select Option1/Option2/control bundles; only after that run Q3/Q4 causal and Shapley checks. Ground this in todo.md and RECENT_CHAT.md if needed, but do not ask Gavin to restate the project context.
• For Value Action after the paper/plan shifts to item-level analysis, treat the 56 value items as the primary next-step unit. If Gavin asks whether the plan changed away from broad 10-family slices, answer yes when local files support it: the current paper framing says the analysis should run first at the 56-item level and aggregate upward later. The TODO may still say “value/family,” so distinguish conceptual plan from TODO wording. Do not automatically recommend another 10-family pass; first run/organize the cheap item-level Q1/Q2/Q5 screen and choose a balanced item. In the checked 2026-05-30 state, Daring was the best continuity candidate after Stimulation, while Independent was the best new-family item-level candidate. See references/value-action-56-item-first-next-step.md.
• For Value Action Task1 Q-system / final feature-union questions, separate Task1 endorsement from Task2 option-choice directions. Task1 uses a 1–4 endorsement scale (1/2 endorse, 3/4 reject), not a 1–10 score; official agreement means exact parsed option match, and mean absolute delta is on the 1–4 option-number scale. The primary mechanistic branch is clean per-value option-only; official all-56 JSON is a faithful-task comparison branch. The first executable Task1 Q1/Q2/Q5 script is scripts/43_compute_task1_clean_option_q1_q2_q5_candidate_metrics.py, with Task1 sites answer_choice_token, value_name, value_statement, value_block, and pre_answer. When Gavin asks “how many Task1 selected?” distinguish: all 56 items complete, 771 candidate atlas features scanned per item, Task1 shortlist rows are typically 56 × (30 endorse + 30 reject + 20 scaffold/control) = 4,480, unique shortlist features were 493 in the checked Phase 1.4 state, but Task1 added 0 genuinely new feature IDs to the final union; the final 777 union was old 771 plus 6 Task2-reversed-only features. When he asks which token was used, answer directly: Q1 is on the generated answer digit/choice token, Q5 is the last pre-answer token before generation, and Q2 is over value-name/value-statement/value-block prompt spans; do not conflate these with rationales or free-generation tokens. For future naming, prefer S1/S2/S3 for selection screens, P0 for slice-level pre-decision predictability, and V1/V2 for decisiveness validation; emphasize that Task1 S2/value-text association is weaker and not equivalent to Task2 action-span evidence. Keep final feature union branch-labeled (task2_normal, task2_reversed, task1_clean_option, optional task1_official_json) and do not claim broad free-generation steering just because Task2 Q3/Q4 causal ablations exist. If Gavin asks to confirm what Task1 vs Task2 are, answer directly that Task1 is stated/espoused value endorsement under country+context, not situation+action accept/reject, while Task2 is the concrete two-action choice; show compact templates/examples from references/value-action-task-templates.md. See references/value-action-task1-qsystem-feature-union.md, references/value-action-qmetric-renaming-and-predictability.md, and references/value-action-task-templates.md.
• For Value Action new-dataset design, build around explicit controversial value conflicts, not one value plus two weak actions. If Gavin questions whether current Task1/Task2 results support a value-action-gap claim, lead with the conservative claim: current results are a pilot/warning signal because weak overlap may reflect prompt format, option salience, one-sided value/action mapping, or context artifacts rather than a true stated-vs-enacted gap. For replacement data, use paired value-conflict cases: Task1 asks abstract priority between two defensible values; Task2 presents a concrete dilemma where each action manifests one side. Require steelmanned options, minimal context by default, multiple wording variants, normal/reversed order, and baselines: same-pair, cross-value, random-question, same-format non-value, shuffled mapping, implicit/explicit value-label, and wording variants. It is fine to use Codex/LLMs heavily for candidate generation, but separate generation from critique, revision, and manual/paper-facing validation. Start small and clean before scaling. See references/value-action-controversial-value-conflict-dataset.md.
• For Value Action Task1↔Task2 alignment/overlap analysis, case-stratify before individual feature stories. Start from behavioral classes (Task1 endorse/reject × Task2 value-consistent action chosen/rejected), then split by Task2 normal/reversed order and no-reason/with-reason variants. Report feature activation percentages and top-K/weighted overlap across Task1 sites (value_name, value_statement, value_block, pre_answer, answer_choice_token) and Task2 sites (action span, pre-answer, answer token, reason tokens separately). Classify features as shared/enacted, espoused-only, action-only/inhibiting, semantic-action follower, position-bound, reason/rationale-only, or scaffold. If Gavin asks about “interesting high S3V12 with Task1,” answer that Task1 S1/S2/S3 exists but the individual high-S3 plus V1/V2 Task2-causal feature audit is a separate next analysis unless local artifacts show it has since been written. For original RQ mapping, overlap tables answer RQ5/RQ6, Task2-only/Task1-only classes start RQ7/RQ12, Task1 V1/V2 is needed for RQ2, cross-task steering is needed for RQ8, and Keishii L11-only work does not settle RQ9. See references/value-action-task1-task2-alignment-overlap.md.
• For corrected Value Action Phase 2 after the Phase 1.7 S4 correction, make S4/S3∩S4 first-class but start with denominators before overlap claims. Gavin corrected that Phase 2 should first redo Task1↔Task2 overlap with Task1-S4, Task2-S4, V3/V4, and S3∩S4 as explicit lanes, not treat Phase 1.7C as an appendix. Compute S3/S4/S3∩S4 at their intended site (pre_answer for the corrected S lanes); do not average across sites. Before the full overlap atlas or more steering, run a per-value denominator and S1234 coverage audit: count Task1/Task2 examples under each condition, check direction/gap balance, and compute selected-feature count share plus positive-activation-magnitude share at each lane's intended site. Sequence: Phase 2.1R denominator + S1234 coverage audit, then broad condition-stratified activation/overlap atlas, then candidate mining, then dose/flip screens and V6-style free-generation on the small survivor/control set, and only then aggregate value-action gap/no-gap mechanisms. Treat strict S3∩S4 overlap as high-precision candidates, not automatically stronger mechanisms, until dose and order-flip screens support them. See references/value-action-phase2-1R-denominator-and-coverage-audit.md, references/value-action-phase2-1r-denominator-first-planning.md, references/value-action-phase2-s4-s3s4-first-overlap.md, and references/value-action-phase2-s4-s3s4-bridge-dose-screen.md. If the user wants to explore the 56-value denominator/activation distribution interactively, route to the Value Feature Browser pattern in gavin-research-presentations/references/value-action-phase2-1r-activation-audit-browser.md rather than forcing dense slides. When Gavin asks what the old RQs are, or how the 2.6% bridge-overlap number relates to gap/no-gap baselines, separate the prompt-level gap/no-gap overlap analysis from the Phase 2.4 selected category-record census; see references/value-action-rq-overlap-denominator-synthesis.md. If Gavin asks to focus on Layer 2 or “find qualitatively interesting things first,” switch from broad quant tables to manual mixed-value motif mining: use the naturally mixed values, inspect pre-answer S3/S4/S3∩S4 candidate rows, pull concrete Task1/Task2 examples and feature interpretations, label semantic motifs vs scaffold controls, and produce a shortlist memo before proposing quantitative validation. Prefer a focused interactive audit/table over slides when there are many examples: rows should include value, feature id, Task1 direction, Task2 setup/direction, lane memberships, feature explanation, qualitative motif, bridge story, scaffold/control risk, and promote/backup/control tier. If Gavin pushes for the actual feature behavior inside value-action gaps, use the six-category taxonomy before anecdotes: categories 1/2 are aligned controls; categories 7–10 are the main gap mechanisms; categories 3–6 are diagnostics; category 8/10 action-side overrides are usually the highest-priority steering candidates. Explain lane cards as Task1 lanes | Task2 lanes | feature explanation. If a six-category steering screen is run, do not promote a candidate just because it moves Task2 logits: require semantic Option1/Option2 consistency across normal and reversed display order, and distinguish clean semantic gap-closing from order/display-position effects. When S3/S4/S3∩S4 steering results look order-sensitive, audit before concluding: S4 is a semantic pre-answer gap metric, so do not say “S4 features are display-order features.” Instead report that the causal decoder-direction intervention behaved display-digit/display-position-like in the forced-choice setup, then verify sign conversion, exact canonical S3/S4 provenance, order-specific qroots, and both semantic and display margins. The completed Phase 2.3A six-category run was mostly negative for clean semantic gap-closing, and the stricter Phase 2.3B rerun found source-order effects but 0/41 both-order semantic Task2 positives. For Phase 2.4-style recovery, broaden before steering: build an all-56 robust S3/S4/S3∩S4 candidate census, keep Task2 both-order semantic prefiltering separate from later causal steering, and manually inspect Tier A Cat8/Cat10 S4/S3∩S4 rows before a compact Phase 2.4B dose screen. When interpreting the completed Phase 2.4B category-specific S3/S4/S3∩S4 steering run, do not collapse to one aggregate success/failure rate: report scope/filter counts first, then category/value/S-bin results; the safe conclusion is meaningful but category-specific, with Cat7 value-side transfer strongest, Cat8/Cat10 action-side survivor leads, Cat1/2 bridge-control negatives, and display/order caveats especially in Cat10; see references/value-action-phase2-4b-category-s34-steering-synthesis.md. After Gavin's clarification, always separate natural activation/asymmetry from steering/capability for categories 1/2/7/8/9/10. Cat1/2 bridge controls ask whether natural both-side rows are causal-positive on both sides. Cat7–10 are already interesting because of one-task natural recruitment/asymmetry; steering is only a follow-up diagnostic. Do not call Cat7/9 +α positives gap-closing unless the manifest target is explicitly the closure direction: in the Phase 2.4B run Cat7 targeted Task2 Option2 and Cat9 targeted Task2 Option1, i.e. encoded/gap-side directions, so closure requires the opposite direction or suppression readout. See references/value-action-natural-gap-vs-steering-reframing.md and references/value-action-category-direction-and-overlap-lessons.md. Before presenting category steering, list fake examples and verify direction logic step-by-step; report raw/selected/S3∩S4 overlap denominators first. When building/interpreting the steering manifest, use category-specific requirements rather than a single gap-data rule: Cat1/2 are bridge controls and may be tested without gap-only data; Cat7/9 require value-side Task1 S4/S3∩S4 evidence; Cat8/10 require action-side Task2 both-order semantic S4/S3∩S4 evidence. If the GPU run completes but the shell exits after missing strict manifest, check Task1/Task2 manifest combo counts before relaunching; often only the summarizer RUN path needs patching. See references/value-action-phase2-1r-layer2-qualitative-mining.md, references/value-action-six-category-gap-feature-dose-screen.md, references/value-action-phase2-3a-steering-pipeline-audit.md, references/value-action-phase2-4a-robust-candidate-census.md, and references/value-action-phase2-4b-category-specific-steering.md.
• For Value Action Task1 Phase 1.5 stated-value analysis, keep Task1-specific manual triage separate from cross-task interaction. Gavin corrected that the individual important S1/S2/S3 feature analysis belongs in a Task1-specific report first. Treat Phase 1.5 as Task1 espoused-value S1/S2/S3 screening plus manual feature triage; Phase 1.6 as Task2 normal/reversed and no-reason/with-reason robustness; and Phase 2 as Task1↔Task2 espoused/enacted interaction. When building this report, aggregate Task1 clean-option shortlist rows, enrich with V2.1/moral-primary metadata, manually mark value-relevant candidates and scaffold/artifact controls, extract concrete Task1 top-activation examples, update the static browser page, and update todo.md/RECENT_CHAT.md. Do not claim Task1 V1/V2 causal evidence unless a separate Task1 causal mini-pass using endorsement/exact-option/ordinal margins has been run. See references/value-action-task1-phase1-5-manual-analysis.md.
• For Value Action Phase 1.6 Task2 condition-overlap completion, use the existing A/B/C × normal/reversed robustness artifacts as the source and keep it descriptive. The completed source run has 43,680 prompt cells, 5.24M site top-feature rows, and 3.07M reason-token rows. Report normal↔reversed semantic/display agreement, A/C no-reason↔with-reason answer changes, the primary 2×2 condition matrix, feature top-k overlap, same-answer vs changed-answer overlap, rank-gap diagnostics, reason-token artifact candidates, and heuristic feature-stability labels. Avoid launching steering or claiming Task1↔Task2 interaction in Phase 1.6. For large site scans, avoid all-pairs per-example loops over all 5M rows; compute aggregate condition×semantic-site×feature summaries plus exact per-example top-k Jaccard for the primary pairs/sites. If Gavin asks for value-feature overlap, S1/S2/S3, V1/V2, or pushes back on all-value overlap, treat value_item as the primary unit: distinguish raw full-basis top-20 activation overlap from S/V-selected feature-set agreement, split same-answer vs changed-answer pairs, and do not present a collapsed all-values raw top-20 chart as the per-value result. If he asks whether S/V-selected top features preserve when answers match vs change, do not average S/V stages across unrelated sites; evaluate each metric at its intended site family (e.g. S3 at pre_answer, V1/V2 action rows at matching semantic target action site) and show same/changed denominators plus active/preservation rates. See references/value-action-phase1-6-task2-condition-overlap.md, references/value-action-phase1-6-per-value-condition-overlap.md, and references/value-action-intended-site-sv-answer-match.md.
• For Value Action Task1 V2/Shapley selection questions, state the selection boundary before interpreting scores. V2 is exact Shapley only within the selected small value×target-direction×site bundle, not an all-feature/global causal score. Missing V2 usually means not selected/tested, not zero effect; changing the bundle can change credit for correlated features. For the full selected Task1 V1/V2 run, selection started from S1/S2/S3 browser rows, prioritized S3/S2/S1, causal site, target direction, rank/score, manual-interest labels, moral-domain labels, and penalized artifact/control rows; it then capped coverage per value and capped exact-Shapley bundles at 4 features. Use wording like “bounded causal validation of screened features,” not “perfect/global Shapley decomposition.” See references/value-action-task1-v2-selection-dependence.md.
• For Value Action feature-explanation pipeline questions, separate explanation validation from decision validation. Start by explaining the two evidence layers: (1) held-out domain-binned feature-explanation validation asks whether a written explanation predicts Keishii L11 activation bins on ETHICS/Moral Stories examples; (2) Q1-Q5 / Q3-Q4 asks whether that explained feature directionally or causally supports Option1/Option2 in VAL-style choices. Do not imply that explanation validation is causal decision evidence. Current v8 anchors: 771 atlas features all have explanation records; 139 newly surfaced features have domain-binned validation; all-56 Q1/Q2/Q5 and Q3/Q4 are complete; browser data has 262,480 metric rows. See references/value-action-feature-explanation-validation-pipeline.md.
• For Value Action Phase 1.4 Q125 union / V2-delta questions, keep branch provenance explicit and do the narrow delta, not a full re-interpretation. In the checked Phase 1.4 state, Task2 normal Q125, Task2 reversed Q125, and Task1 clean-option Q125 are complete; the final branch union is 777 features: old 771 plus 6 genuinely new Task2-reversed features (f151, f2701, f8312, f12048, f30538, f32048). Task1 clean-option adds 0 new feature IDs; it adds metrics over the existing universe. For V2 delta work, collect WikiText/global and moral-domain records for only those 6, build dual-source packets, run V2.1 axis-separated explanations, then export a compact summary. Keep option-position binding audit and final synthesis artifact definitions crisp. See references/value-action-phase1-4-q125-v2-delta.md.
• For Value Action feature-category provenance questions, say categories are heuristic labels, not complete manual labels or learned quantitative clusters. Labels like semantic-ish, scaffold/syntax, format/scaffold, and mixed/uncertain come from a rule-based semantic_category() pass over each feature's short_explanation. The upstream discovery, validation, and Q metrics are quantitative, and later manual/agent synthesis audits important rows, but the semantic_category field itself is a cheap triage/filtering convenience. Treat semantic-ish as weak evidence only; stronger category claims require rubric-based relabeling or consistency-checked classifier/LLM judging. See references/value-action-feature-category-provenance.md.
• For Value Action SAE-vs-raw/PCA control questions, compare decision-deciding units, not raw activation ranks, and do not collapse raw/PCA unless Gavin explicitly asks. Gavin's intended comparison is top raw layer-coordinate and PCA signed-pole units vs top SAE/transcoder features under the same directional Q stack: Q1 answer-token direction, Q2 selected option/action-span direction, Q5 pre-answer probe influence, then Q3/Q4 causal margin movement/Shapley. Use signed positive/negative poles for raw/PCA coordinates, fit PCA unsupervised on the same moral-domain L11 hook-site activations, and keep rank denominators explicit. For Q1/Q2/Q5 screens, report SAE vs raw and SAE vs PCA separately; avoid a pooled dense_best framing unless it is only an explicitly-labeled challenger screen. When Gavin asks whether the result is an average or a top comparison, state the unit precisely: e.g. value_item × question × direction × site, top-1/top-5/top-10 within each basis, not an average across all features. For top-k robustness, compare top-k means (or another clearly named top-k statistic) for SAE, raw, and PCA separately and name the candidate denominators (SAE atlas features vs 8192 raw signed poles vs 8192 PCA signed poles). Do not directly compare SAE decoder-ablation magnitudes to raw/PCA input-coordinate ablations unless intervention type is matched or caveated; prefer empirical rank-1 decoder vectors for raw/PCA causal controls. Keep the conclusion layered: Q1/Q2/Q5 can show dense bases are stronger descriptive/probe predictors, while Q3/Q4 after explanation validation is the decision-causal test. Use the compact synthesis when useful: dense raw/PCA coordinates can be better screening separators because they preserve broad polysemantic surface information, but SAE/transcoder features give clearer validated causal decomposition after Q3/Q4. See references/value-action-raw-pca-dense-basis-controls.md and references/value-action-raw-pca-topk-and-causal-followup.md.
• For Value Action SAE-vs-raw/PCA sanity-check plans, use dense bases as matched controls, not as a different task. Compare the existing Keishii L11 SAE Q1-Q5 pipeline against raw layer-11 coordinates and PCA coordinates at the same hook site. Treat raw/PCA signed coordinates as signed-pole pseudo-features (pos=max(score,0), neg=max(-score,0)), fit PCA on natural moral-corpus activations rather than VAL labels, and keep explanation validation separate from decision validation. For causal Q3/Q4, do not directly compare SAE decoder ablations with raw/PCA input ablations unless the intervention type is matched/caveated; the recommended primary control is an empirical rank-1 decoded-output ablation for raw/PCA units. Start with a challenger pilot before all-poles LLM explanation. When synthesizing continuation runs, verify phase-1 via durable files (basis fit + 112 item/basis status rows), treat dense Q1/Q2/Q5 wins as numerical challengers only, and require held-out explanation validation plus rank-1-decoder Q3/Q4 before any “decision-deciding” verdict. See references/value-action-raw-pca-basis-comparison.md.
• For Value Action V2.x feature-explanation audits, produce machine-readable manual labels and update the browser, not just prose. Join V2.x axis-separated explanations, source-rank validation, V1/V2 comparison, and exported top-k browser metric rows; use the exported top-k table for top-feature appearance counts rather than full metric_long rank rows. Keep SAE/transcoder manual-interest labels separate from raw/PCA dense-control labels, and regenerate no-server browser HTML if it exists so embedded data is not stale. Required outputs should include CHANGED_INTERPRETATION_TABLE.csv, INTERESTING_FEATURE_LABELS.csv, raw/PCA challenger review/table, browser update notes, and deck notes. See references/value-action-v2-1-feature-audit-browser-deck.md.
• For Value Action Phase 1.4 Q125 union deltas, interpret only genuinely new union features and keep branch provenance explicit. After Task2 normal/reversed and Task1 clean-option Q125 are complete, separate original atlas/V2-covered features, branch-only candidate rows, and genuinely new feature IDs. In the checked Phase 1.4 state the original atlas had 771 features, Task2 reversed contributed 16 reversed-only candidate rows but only 6 genuinely new features, Task1 clean-option added 0 new union features, and the final union was 777. Run V2 record collection/explanations only for the genuinely new delta, then verify exact feature-count coverage before claiming all-union interpretation. See references/value-action-phase1-4-q125-delta-v2.md.
• For Value Action Q3/Q4 after a Q1/Q2/Q5 focus slice, keep target-support sign and controls explicit. Use the selected answer format's margin (e.g. Variant B bare digit: M1=log p("1")-log p("2")), convert ablation deltas into target-support signs (Option1=-delta_m1, Option2=+delta_m1), split action vs pre-answer site outputs, run exact Shapley only over small bundles, and include scaffold/random atlas controls before claiming semantic specificity. If a feature is anti-target under Q3/Q4, drop/invert it even if Q1/Q2/Q5 ranked it highly. See references/value-action-tradition-q3q4-causal.md.
• For Value Action after Phase 1.3 top-15/robustness/Task1-bridge work, do not imply broad feature steering is already done. Phase 1.3 produced the top-15 Task2 audit, Task2 A/B/C normal-vs-reversed robustness, Task1 prompt comparison/clean-option Q-system, and branch-union plan; existing Q3/Q4 is VAL-local ablation/Shapley, not dose-response or free-generation steering. If Gavin asks what is missing, name Phase 1.4 as the top-feature steering and option-position confound study: semantic shortlist of the 15, value/item + Option1/Option2 direction labels, calibrated positive/negative decoder-direction patches, unrelated free-generation prompts, and a normal/reversed audit separating semantic-action followers from displayed Option1/Option2 position detectors, answer detectors, and scaffold/format features. See references/value-action-phase1-4-feature-steering-and-option-binding.md.
• For Value Action top-15 steering deck Q&A/revisions, answer the reader-facing interpretation first and keep deck metadata self-explaining. When Gavin asks about the Keishii L11 top-15 steering slides, clarify that feature titles are selected 56-value items; use S1/S2/S3 and V1/V2 with old Q names in parentheses; explain base O1/O2 as α=0 semantic baseline-choice counts; define the two dose-plot panels (display-order split vs baseline-choice split); and translate raw verdict strings into friendly slope-summary labels. For flipped-answer tables, highlight the expected/toward-target direction (O2→O1 for target Option1, O1→O2 for target Option2). Free-generation snippets should be handpicked/scored for readable interpretation-linked examples, while quantitative claims stay tied to replicated keyword/semantic-drift tables. See references/value-action-top15-steering-deck-qa.md.
• For Value Action V6 fixed-completion/free-generation relatedness significance, test paired clusters rather than rows. Rows are nested by feature × template × alpha × sample, so row-level p-values are overconfident. Use feature × template as the paired unit against the same cluster's α=0 baseline. Primary outcomes should include direction-weighted relatedness (supportive_relatedness = score if direction=supporting else 0, opposite_relatedness = score if direction=opposite else 0), score≥3, and direction-specific score≥3. Global paired-cluster bootstrap/permutation is the primary claim; per-feature tests are diagnostic heterogeneity maps; per-template tests are exploratory. See references/value-action-v6-freegen-significance.md.
• For Value Action S4-corrected Task2 top-15 refreshes, run the actual latest V6 dose/free-generation pipeline and keep S3/S4 lanes separate. Gavin's phrase “latest / most updated Phase 1.4 Task2 top-15” means the real V6/V5-style all-example dose + role-diverse fixed-completion free-gen + blind judge + significance deck spine, not a compact S/V evidence deck. If the target manifest changes to S4/V3/V4 candidates, rerun dose/free-gen/judge on that manifest; do not reuse old V6 rows. Describe the selected list as S4-selected, V3/V4-positive, non-scaffold, manually/heuristically interesting feature–value–target pairs — not pure S4-score top-15. Use readable one-claim-per-slide overlap addenda and replace placeholder option-summary text with actual target pair, feature interpretation, manual rationale, and selection evidence. See references/value-action-s4-v6-dose-freegen-refresh.md.
• For Value Action V5 all-example steering revisions, lead with all-example qualitative per-feature evidence when Gavin asks for it. Gavin corrected that the next V4/V5 interesting-pair version should focus on all-example denominators because it is a qualitative per-feature study; K-balanced can remain appendix/internal robustness, but should not be pushed as the main deck. State that current V4 patches the Keishii L11 transcoder decoder direction at the selected site and lets downstream layers run normally; this propagated L11 intervention is the main behavioral causal test. If he asks about “only that layer” or restoring later layers to baseline, frame it as a separate layer-local/direct diagnostic, not a replacement or reselection criterion by default. Prefer a cleaner V5 alpha grid around ±1/±3/±5/±7; treat ±10 as stress/OOD. Before building V5, first produce a manual/LLM-assisted free-generation relatedness audit from V4 outputs with scored examples and reasons for snippet selection. See references/value-action-v5-all-example-steering-and-freegen-audit.md.
• For Value Action V6 fixed-completion significance questions, avoid row-level pseudo-replication. The generated rows are nested by feature × template × alpha × sample; use paired feature_idx × template_id clusters against each cluster's own α=0 baseline, with feature-level/per-template tests only as diagnostics. Prefer direction-aware outcomes such as supportive_relatedness, opposite_relatedness, hit_ge3, and signed_directional_relatedness; slope-style alpha tests can be more informative than coarse positive/negative band means. If Gavin says to “play around” with significance, save standalone completion_significance/ tables/plots and discuss claim hygiene before editing slides. See references/value-action-v6-completion-significance.md.
• For Value Action steering-template inventory questions, inspect manifests and separate active vs historical templates. If Gavin asks what templates/prompts are currently being used, first check the Phase 1.4/top-feature steering manifests/CSV rows rather than answering from memory. Report active V5 open-ended/free-generation prompts and V5 fixed neutral completion templates first; then label older smoke prompts as historical and keep the VAL Task2 forced-choice template separate from free-generation/completion templates. Current checked inventory and paths are in references/value-action-steering-template-inventory.md.
• For Value Action V4/top-feature deck selection, require an interesting feature-interpretation ↔ target-value pair, not just high Q3/Q4/Q5 evidence. Gavin corrected that a candidate should be selected because the feature interpretation and the value item/direction form an intuitive story, whether supportive or opposing. Do not confuse provenance tags like prior_power_v7_priority with the actual target value row. Example: f22192 as empathy/conflict-resolution is weak when paired with Clean/Security because Clean is a literal 56-value item meaning neat/tidy/orderly, but strong as a V4 candidate when paired with Social Power/Power Option2 because empathy pushing against dominance/status-power is interpretable. Also reject metric-positive but semantically weak/generic pairs such as Creativity + possessive/close-relationship, Creativity + specific prior entity, or Successful + possessive/genitive unless inspected examples prove a concrete mechanism. Before building PDFs, send a compact manual shortlist of ~15 such pairs with separate columns for feature ID, target option/direction, value/family, feature interpretation, and why this value-direction pair is interesting; do not merge feature interpretation and rationale into one column because Gavin cannot tell which is which. See references/value-action-v4-interesting-feature-value-pairs.md.
• For Value Action Phase 1.7B Task2 dose-target selection, default to interesting-only manual selection rather than score ranking. Gavin corrected that the purpose of the experiment is to find the most interesting feature–value–target-option pairs and then run dose/free-generation on them; S4/V3/V4 scores can be provenance/evidence but should not drive the shortlist. Do not include metric-positive but semantically weak rows such as Creativity + possessive/close relationship, Creativity + specific prior entity/concept, or Successful + possessive/genitive unless examples prove a concrete mechanism. In Discord tables, keep Feature interpretation and Why this pair is interesting as separate columns; do not merge them into one “brief/why” column, because Gavin needs to distinguish the feature label from the rationale. See references/value-action-phase1-7b-interesting-only-shortlist.md.
• For Value Action Phase 1.7A Task1 interesting-top-15 proposals or completed tables, keep Task1 scope and semantic-interest selection explicit. Phase 1.7A means Task1 S4 endorse-vs-reject gap candidates plus V3/V4 validation, with final row selection based primarily on feature interpretation × Task1 value statement × endorse/reject direction rather than raw score. For completed outputs, include the short Task1 value definition in the table and keep S4/V3/V4 as evidence columns; do not silently remove semantically interesting rows just because bounded V3/V4 is weak or negative. If the new phase1_7a_task1_s4_v3v4_* artifacts are not present yet, give only a provisional/manual-interest shortlist grounded in existing Task1 Phase 1.5 / V1/V2 / top_interesting_features artifacts, and explicitly say final rows may change after the actual S4/V3/V4 run. For quick Discord proposals, use a compact aligned text table with Feature, Target option, Value/family or Value definition, and Brief interpretation; collapse Task1 target options as Endorse = O1/O2 upward, Reject = O3/O4 downward. See references/value-action-phase1-7a-task1-interesting-shortlist.md.
• For Value Action Phase 1.7B Task2 interesting-top-15 proposals, answer as a concise feature-value-option shortlist, not a metrics report. First check docs/PHASE1_7_MAKEUP_RUNS_PLAN.md and docs/VALUE_ACTION_INTERESTING_FEATURE_VALUE_PAIR_DEFINITION.md, then use the latest S4/V3/V4 and/or S3/S4 two-lane candidate artifacts as evidence filters. The promoted unit is feature × value item/family × target Option1/Option2; repeated feature IDs are acceptable when the value-option story differs. Primary ranking is manual interesting-pair quality: an intuitive, specific feature interpretation × target value × target option story beats a higher-scoring vague row. For Gavin’s quick Discord ask (“propose 15 interesting features”), lead with the list only: columns #, Feature, Target, Value, and Brief interpretation / why interesting; keep it compact, mention promote vs backup only briefly after the table, and avoid dumping S4/V3/V4 numbers unless requested. Demote rows like abstract “specific entity/concept” unless examples show a concrete mechanism for that target value-option pair.
• For Value Action Stimulation after the post-atlas Q1/Q2/Q5 pass, treat Q3/Q4 as a lead-plus-control warning rather than a clean mechanism. Variant B made Stimulation balanced enough (Option1=208, Option2=186), and Q3/Q4 found a strong semantic/action lead f4754 on Option2 action, but the largest effects were scaffold/pronoun controls (f24246, full Option2 scaffold bundle). Also note that high-Q5 features can invert causally (f29594, f15596, f27056 in Stimulation), so revise bundles by causal sign before claiming anything. See references/value-action-stimulation-q3q4-causal.md.
• For Value Action after a completed-but-limited value checkpoint, move to the next balanced value rather than over-optimizing by default. If the current value has a useful Q3/Q4 result but scaffold/random controls limit specificity, choose the next slice by minority-class balance and run cheap Q1/Q2/Q5 first. After Tradition, Stimulation was the best next slice: Variant B balanced to Option1=208, Option2=186, unparsed=0; Q5 probe balanced accuracy was 0.667. Treat its reviewed bundles as pre-causal candidates until Q3/Q4 confirms signs with controls. See references/value-action-stimulation-qmetrics-next-slice.md.
• For tokenizer-free / apperception experiments, verify apparent gains with fixed large checkpoint eval before over-interpreting train-time metrics. Train-time metrics.jsonl may use changing eval seeds or small samples, so report final-checkpoint fixed held-out results first, keep best-checkpoint selection bias explicit, and include per-task split diagnostics before claiming a slot/workspace mechanism. See references/tokenizer-free-fixed-checkpoint-eval.md.
• For tokenizer-free / apperception results, run example-level calibration diagnostics before claiming a clean binding/workspace mechanism. Round2 showed that aggregate wide-local gains can hide boolean yes-bias, answer-range extrapolation confounds, and weak raw character-level slot traces. Compare same-seed baseline vs candidate predictions, report yes/no or digit prediction distributions, split scope binding from digit extrapolation, and aggregate slot attention by parsed clauses/entities rather than raw characters. See references/tokenizer-free-round2-example-diagnostics.md.
• Do not ask for context when project-local notes can identify the result. First inspect likely recent-chat/project mapping/result artifacts.
• For live paper-reading Q&A, answer the exact confusion before broad synthesis. If Gavin points to a figure/caption and asks “what are these?”, first identify whether the visual elements are experiment conditions, toy explanatory panels, metrics, or axes. Define the symbols in one sentence each, then map each subpanel/row to the paper’s claim. Avoid turning the answer into a full paper summary unless asked.
• For synthetic training runs with changing eval samples, run fixed checkpoint eval before making the claim. If training-time eval uses small or per-step-changing samples, treat best_hard/training-log comparisons as provisional. Reload the saved checkpoints, evaluate all variants on the same fixed seeds with larger held-out sets, compare final checkpoints first, and then inspect per-task metrics to see whether the split-level gain supports the intended mechanism. If the fixed eval weakens the initial story, recommend a narrow multi-seed pass over the leading variants rather than expanding architecture variants. See references/tokenizer-free-fixed-checkpoint-eval.md.
• For branch-specialization, do not overclaim clean functional modularity from role-affinity evidence. Larger-head-count controls can preserve structural specialization while weakening the stronger functional-module claim. Small validated batteries (e.g. 5 families / 10 templates, or less) are not enough for a functional-modularity conclusion even if ontology-label alignment improves; treat them as limited role-geometry signals unless label-free clustering and matched controls also improve. Keep structural role affinity, toy functional modularity, and real-model causal validation as separate evidence tiers. See references/branch-specialization-claim-hygiene.md.
• For branch-specialization Pythia all-role questions, do not require every role to pass. Gavin's criterion is that some roles/families can have strong selected-vs-random Q1/Q2 evidence. Current canonical Pythia-160M seed-family aggregation has five 9/9 Q1/Q2 roles (kv_lookup, local_copy, repeat_match, copy_suppression, natural_repeat_8g) and two noisy/negative assays (ioi_name_mover, delimiter_boundary). Treat the initial copy/retrieval/memory Q3 geometry as a module-map lead, and distinguish it from the stricter causal bundle-ablation follow-up: copy_memory is 5/5 for Q1/Q2 but 4/5 for Q3 bundle (copy_suppression drops out), while boundary/entity controls are 0/1 and were not Q3-bundle-tested. For Gavin's “X out of X roles from each family” requests, answer with a fenced text table. See references/branch-specialization-pythia-all-role-module-search.md and references/branch-specialization-phase5-family-pass-tables.md.
• For branch-specialization “are these roles working?” questions, answer role status first. Gavin corrected that he does not need a broad specialization/global verdict when he asks whether roles are working. Start with a direct role-status table: repeat_match, local_copy, natural_repeat_8g, and kv_lookup working strongly; copy_suppression working individually but not cleanly in the same copy-memory bundle; ioi_name_mover and delimiter_boundary noisy/not working yet. Only after that, clarify whether the evidence is pretrained Pythia causal validation or a small from-scratch hetero-vs-uniform LM. See references/branch-specialization-direct-role-status.md.
• For branch-specialization Pythia causal-role answers, state “evidence for what” before using validation/module language. Gavin found phrases like “pretrained ordinary Pythia head evidence” and “validated causal module” confusing when the main project question is hetero-vs-uniform architecture. Always separate the evidence classes up front: Pythia ordinary-head ablations validate target roles/functions in a real pretrained LM; they do not validate heterogeneous head dimensions. Hetero-vs-uniform training is the separate architecture test. Define “causal” as selected-head/bundle ablation changing endpoint behavior more than matched random heads, and define “module” as overlap/clustering/bundle effects among related selected heads, not a final claim of hetero architectural success.
• For branch-specialization real-language LM sweeps, require the metric directions to agree before calling a win. Real-language next-token training is a stronger substrate than synthetic prompt-target training, but a narrow role readout can still show only weak structural affinity. Aggregate by config and paired seed deltas; check validation-loss matching first; call specialization positive only if top-mass rises and effective-head count falls. If affinity is positive but specialization/modularity are not, recommend a diagnostic/readout upgrade rather than simply scaling the same narrow probe run. For broad matched Phase 4C-style sweeps, report Q1/Q2/Q3 separately with pass counts by operation/coarse-family/axis and do not convert wider-head enrichment into a functional-module claim. See references/branch-specialization-real-language-phase4c.md and references/branch-specialization-phase4c-v4-broad-matched-real-lm.md.
• For branch-specialization synthesis, produce stratified effect maps before global verdicts. Do not collapse hetero-vs-uniform outcomes into one positive/negative claim when the artifacts contain role, family, format, topic, or operation metadata. State which axes are actually available in the current experiment: e.g. Phase 4C real-language WikiText has 20 role probes across 5 families, but no format/topic metadata; Phase 4B synthetic open-module has operation/topic/format metadata but is not real-language evidence. Treat strong subset effects as meaningful rather than averaging them away. When asked which role/family is positive on all project questions, separate (a) strict family-mean wins, (b) subset-valid family wins where some roles inside the family pass, and (c) role-level hits combined with the family modularity axis. Positive affinity/specialization/modularity for some roles or families is a valid finding even when global averages are mixed. See references/branch-specialization-stratified-effect-maps.md and references/branch-specialization-all-three-effect-map.md.
• For Gavin's branch-specialization modularity questions, keep the module axis open-ended. Do not assume the relevant modules must be known LLM mechanisms. If Gavin asks about expanding role/function ontology or “what kind of modularity,” propose a bottom-up module-discovery battery with crossed labels (operation, topic/domain, format, cognitive primitive, target type, token-statistics controls), then apply the same three questions — structural affinity, specialization, and modularity — to any top-down ontology or discovered cluster. See references/branch-specialization-open-ended-module-discovery.md.
• For branch-specialization status confusion, inventory before interpreting. If Gavin asks “where are all the roles/families,” “how many have we finished,” or pushes back on incremental small-role phases, start with explicit counts for each inventory: completed real-language hetero-vs-uniform roles/families, broad registry probes/operation families/coarse families, and pretrained-Pythia causal roles. Do not mix Pythia causal validation with hetero-vs-uniform completion. If the workflow is unchanged, prefer proposing one organized broad sweep plus one consolidated report rather than more tiny slices. See references/branch-specialization-role-inventory-big-sweep.md.
• For branch-specialization Phase 8 / role-atlas planning, do not narrow to the currently clean Pythia copy-memory roles. Gavin corrected that the readout-registry roles and axes should be used rather than ignored. Treat registry labels as candidate roles/module axes to validate causally across actual pretrained LMs, not as already-validated endpoints and not as disposable synthetic artifacts. The next-phase framing should be a multi-LLM ordinary-head role atlas: include all registry operations/metadata axes, the current Pythia-clean roles, and literature-derived roles such as induction, copy suppression, IOI/name-mover, successor, retrieval, function-vector, factual-recall, syntax/dependency, and attention-sink heads. See references/branch-specialization-multi-llm-role-atlas.md.
• For branch-specialization Phase 8 specificity, selected-vs-random is only a foothold, not a final module. Before promoting role-atlas rows into hetero-vs-uniform architecture tests, compare each selected bundle against off-role and matched off-role endpoints. Report selected-vs-random pass, self-above-off-role mean, strict self-above-all-off-roles, self-above-matched-off-role mean, and cross-model portability separately. If many rows pass selected-vs-random but few are strict-specific, recommend contrastive head selection (self_effect - matched_offrole_effect) and rerun the specificity matrix before architecture transfer. See references/branch-specialization-phase8b-specificity-matrix.md.
• Do not treat synthetic prompt-target batteries as a real modularity test. For branch-specialization, synthetic/toy prompts can support architectural signals like head-dimension affinity and per-role specialization, but they are not sufficient evidence for or against functional modularity in LLMs. Before interpreting modularity, check whether the model was trained/evaluated on real language data or pretrained real-LM weights. If the data are fixed synthetic prompt→single-token examples, state that modularity must be redone under real-language LM conditions; do not frame a modularity failure there as evidence against real LLM modularity. See references/branch-specialization-real-language-modularity.md.
• For probe-registry LM sweeps, explicitly separate training scope from evaluation granularity. Before interpreting modularity failures, state whether each model was trained jointly on the full probe battery/registry or separately per task, then state that head ablations/modularity metrics are evaluated per probe after training. Do not say only “trained on the task” when the run used many registry tasks; say “trained jointly on all registry tasks, then evaluated per probe.” Also separate “solved this narrow target objective” from “trained as a broad natural LM,” since fixed prompt-target registries can be memorization-like even when all probes reach 100% accuracy. See references/branch-specialization-open-module-training-eval.md.

Scope-explicit tier classification

When classifying research evidence into tiers, every tier must state its scope/unit (row-level, value-pair-level, task-family-level, format/global, etc.) and whether counts are pooled or format-robust. For sparse-feature analyses, use references/scope-explicit-tier-classification.md for the tier/count reporting template and the baseline-active-but-contrast-survives rule.

Recent Gavin-specific corrections

• Value Action Phase 1.7/Phase 2 S4-first correction: When Gavin asks about 1.7C, S4, V3/V4, S3∩S4, or whether 1.7A/B/C are finished, lead with the correction: S4 and S3∩S4 are central lanes, not appendix material. Verify local artifacts before answering, and note that the Task2 1.7B-equivalent outputs may live under legacy phase1_6_s4_* or phase1_4.../s4_v6_actual_dose_freegen/ names. For Phase 2 Task1↔Task2 overlap, distinguish “S4 used as attached stability evidence” from “Task1 S4 and Task2 S4/S3∩S4 included as first-class overlap stages.” If the current package only does the former, recommend/build the latter. See references/value-action-phase1-7-s4-first-phase2-overlap.md.

Output Structure

A good comprehensive explanation usually has this shape:

1. Bottom line / safest interpretation.
2. What was tested and what artifacts were used.
3. Definitions and metric sign conventions.
4. Main quantitative findings.
5. Mechanistic or qualitative story.
6. Causal/intervention evidence.
7. Confounds and limitations.
8. Clean claim to make / what not to claim.

References

• references/capacity-baseline-gap-framing.md — how to frame nonlinear/ResNet SAE wins versus ordinary linear and stronger trainable-parameter-matched big-width linear baselines using signed EV gaps and low-k slices.
• references/research-benchmark-pipelines.md — absorbed class-level benchmark/evaluation pipeline workflow: authoritative benchmark resolution, metric inventory, manifests, adapter layers, smoke gates, API cost/runtime planning, sharding, and matched-pair aggregation.
• references/saebench-current240.md — SAE Bench adaptation notes from the nonlinear SAE current-240 setup, including metric inventory and AutoInterp cost/runtime planning.
• references/cuda-sweep-progress-eta.md — patterns for mapping active CUDA PIDs to logs/progress artifacts and estimating full-run ETA for multi-GPU research sweeps.
• references/value-action-natural-gap-vs-steering-reframing.md — Value Action Phase 2 S3/S4/S3∩S4 category interpretation correction: separate natural activation/asymmetry from steering/capability, treat Cat7–10 asymmetry existence as a result, and do not call Cat7/9 +α positives gap-closing when the encoded target is the gap-side option.
• references/value-action-category-membership-evidence.md — Value Action category membership evidence and wording: use “candidate feature×value×direction record” instead of bare “row”, distinguish value-level candidates from global feature claims, report S3/S4/S3∩S4 counts separately, and separate detection from corrected-direction steering.
• references/value-action-phase2-1R-denominator-and-coverage-audit.md — Value Action Phase 2.1R pre-overlap audit: count per-value/task/condition denominators and measure S1/S2/S3/S4 selected-feature count and magnitude coverage at the lanes' intended sites before building the condition-stratified overlap atlas or collecting more data.
• references/value-action-phase2-1r-layer2-qualitative-mining.md — Value Action Phase 2.1R Layer 2 qualitative workflow: focus on naturally mixed values, manually inspect pre-answer S3/S4/S3∩S4 feature stories and concrete Task1/Task2 examples, separate semantic motifs from scaffold controls, and write a shortlist memo before quantitative validation.
• references/value-action-six-category-gap-feature-dose-screen.md — Value Action correction for Layer 2 gap-feature reporting and Phase 2.3A steering: use categories 1/2 and 7–10, demote 3/4/5/6 to diagnostics, explain Task1 lanes | Task2 lanes, and judge dose-screen success category-specifically.
• references/value-action-phase2-3a-steering-pipeline-audit.md — Value Action Phase 2.3A audit lesson after Gavin challenged the S3/S4 steering interpretation: distinguish semantic S4 provenance from display-digit-like causal intervention behavior, verify canonical S3/S4 provenance and order-specific qroots, and report semantic and display margins separately before final claims.
• references/value-action-phase2-s4-s3s4-bridge-dose-screen.md — Value Action Phase 2.2 bridge-dose pattern after Gavin's S4/S3∩S4 correction: use the last-three S4/S3∩S4 lanes, keep metrics at pre_answer, run Task1 and Task2 normal/reversed dose screens before V6-style free-generation, and treat overlap as a filter rather than proof.
• references/value-action-phase1-7-s4-first-phase2-overlap.md — Value Action correction for Phase 1.7/Phase 2 overlap questions: S4, V3/V4, and S3∩S4 are central; verify whether Task1 S4 and Task2 S4/S3∩S4 are first-class overlap stages or only attached as stability evidence.
• references/value-action-condition-overlap-phase-plan.md — Gavin-specific plan for Task2 order/reason overlap robustness and later Task1↔Task2 condition-aware value-action overlap studies.
• references/live-experiment-stale-running-progress.md — live experiment status-check pattern for distinguishing corroborated active jobs from stale status: "running" progress entries, including conditional ETA wording for stopped jobs.
• references/nonlinear-sae-auxk-live-status-20260529.md — nonlinear SAE OPT-125M core-5 live-status example: systemd-service runners, progress-file counts, latest true tqdm parsing, and fractional x/100 reporting (done + current_run_fraction) for Gavin's quick Discord checks.
• references/nonlinear-sae-live-multisweep-status-20260604.md — nonlinear SAE fast-path pattern for bare "check again" requests when core5 AuxK plus corrected 40-point makeup sweeps are live together; includes failed-entry handling, 40-vs-100 denominator hygiene, systemd/log paths, and compact Discord answer shape.
• references/nonlinear-sae-live-multisweep-status-20260606.md — updated nonlinear SAE live-status example after all four 100-point core5 sweeps reached 100/100; only the two corrected 40-point makeup services remained active on CUDA 2/3, with CUDA 1 idle and CUDA 0 occupied by a non-SAE service.
• references/branch_specialization_phase4c_v2_naturalistic_readout.md — branch-specialization Phase 4C-style diagnostic pattern: checkpoint reuse, eval-only readout controls before scaling, stratified axis interpretation, and claim hygiene for weak/subgroup natural-language modularity signals.
• references/branch-specialization-pythia-all-role-module-search.md — branch-specialization Phase 5 Pythia all-role causal module-search synthesis: all roles need not pass; five copy/retrieval/memory-like roles pass Q1/Q2 across 9/9 seeds, IOI/delimiter are noisy assays, and Q3 selected-head geometry is a module-map lead rather than final bundle proof.
• references/branch-specialization-phase5-family-pass-tables.md — compact Phase 5 family/role pass tables for Gavin's “X out of X roles pass Q1/Q2/Q3” questions, including the stricter Q3 bundle-ablation distinction where copy_memory is 4/5 and copy_suppression drops out.
• references/value-action-controversial-value-conflict-dataset.md — Value Action dataset-redesign pattern after Gavin challenged current Task1/Task2 item quality: treat current results as pilot/warning evidence when dilemmas lack genuine two-sided conflict; design paired abstract/concrete controversial value-conflict cases with steelmanned options, minimal context, wording/order variants, cross-value/random/format/shuffled baselines, and Codex/LLM generation separated from critique/revision/manual validation.
• references/value-action-task1-qsystem-feature-union.md — Value Action Phase 1.3/1.4 Task1 clean-option Q-system pattern: 1–4 endorsement definitions, exact agreement/delta interpretation, branch-labeled Task1/Task2 feature union, first executable Task1 Q1/Q2/Q5 script, sklearn binary-coef pitfall, and causal-vs-free-generation steering wording.
• references/value-action-v7-power-site-identity.md — session-specific example of synthesizing a Value Action V7 Power/Social Power + Wealth analysis with answer-format controls, feature validation, and small-bundle Shapley.
• references/value-action-cross-value-pool-de-atlas.md — Value Action workflow correction for cross-value Pool D/E atlas building before per-value Q-metrics, including explanation/validation gate, malformed explanation repair, resumable JSONL, and imbalanced Pool D caveats.
• references/value-action-choice-correlated-features.md — Value Action feature-analysis pattern for separating option/answer-token detection from choice-conditioned activation gaps, pre-answer probe evidence, and causal/Shapley decision impact.
• references/value-action-option-choice-metric-baselines.md — formal Q2/Q3/Q4/Q5 definitions and baseline/ranking rules for Value Action option-choice feature analyses, including the warning not to treat within-selected-bundle rank/8 as global evidence.
• references/value-action-cross-value-pool-de-atlas.md — Value Action post-Power-V7 workflow correction: build the cross-value Pool D/E feature-expansion atlas across value families, then run Power-style explanations and held-out validation before per-value Q1-Q5 or causal claims.
• references/value-action-post-atlas-qmetrics-focus-selection.md — Value Action post-atlas workflow: choose the next focus slice by choice-balance, use Variant B when it improves balance, run candidate-universe Q1/Q2/Q5, and require manual/agent bundle review before Q3/Q4.
• references/value-action-tradition-q3q4-causal.md — Value Action Q3/Q4 causal pass pattern after a Q1/Q2/Q5 focus slice: target-support sign conversion, action/pre-answer split outputs, exact small-bundle Shapley, scaffold/random controls, and Tradition interpretation lessons.
• references/value-action-stimulation-q3q4-causal.md — Value Action Stimulation post-atlas Q3/Q4 synthesis: Variant-B balance, f4754 as strongest semantic/action lead, f24246 scaffold dominance, Q5-to-causal sign inversions, and revised bundles.
• references/value-action-stimulation-qmetrics-next-slice.md — Value Action post-Tradition next-slice pattern: move on when a value has a useful but control-limited checkpoint, select the next value by choice balance, and use the Stimulation Q1/Q2/Q5 outputs/bundles as the concrete example.
• references/nonlinear-sae-auxk-comparison.md — nonlinear SAE status/comparison example covering Discord-friendly text tables and denominator hygiene for AuxK-vs-baseline matched comparisons.
• references/nonlinear-sae-pareto-beat-loss.md — nonlinear SAE pairwise Pareto beat/loss definitions, match keys, and plotting recipe for AuxK/no-Aux/linear comparisons.
• references/nonlinear-sae-percentile-matched-comparison.md — nonlinear SAE fair-percentile layer: compute EV/L0 percentiles inside family_group × width pools, use harmonic H2 for balanced shortlist ranking, and compare matched ResNet-vs-Linear pairs by ΔH2 without degenerating to 2-row pair percentiles.
• references/nonlinear-sae-global-ranking-phase2.md — nonlinear SAE correction for global top-run rankings: use global EV/L0 percentiles over the eligible run universe, use dead neurons only as a filter/sanity gate, frame OPT Phase 1 as the architecture frontier sweep, and plan GPT-2 no-Aux replication plus public-SAE baseline validation.
• references/nonlinear-sae-expanded-auxk-allcore5.md — expanded OPT-125M core5 AuxK comparison pattern: update old pilot/64 denominators to 100-row all-core5 matches when the three main sweeps are complete, keep partial Linear AuxK comparisons separate, and deliver a verified PNG plus backing CSV.
• references/nonlinear-sae-corrected-makeup-baseline-comparison-20260608.md — completed OPT-125M corrected makeup comparison: dual-gated and matryoshka makeup vs matched linear/linear-AuxK baselines, gated-specific answer pattern, and the caveat that gated AuxK rows are metric-identical to no-Aux because no real dead-latent AuxK mechanism is present.
• references/nonlinear-sae-auxk-coefficient-selection.md — nonlinear SAE AuxK pilot-to-expanded-sweep example: select one coefficient vs doubling the grid, preserve denominator hygiene, and document reuse of existing checkpoints.
• references/branch-specialization-claim-hygiene.md — branch-specialization example for separating structural role affinity from functional modularity claims and choosing real-model causal validation as the next step.
• references/branch-specialization-open-ended-module-discovery.md — branch-specialization guidance for Gavin's bottom-up modularity framing: do not restrict modules to known LLM head roles; build a factorized probe registry, score multiple metadata axes, and apply affinity/specialization/modularity to any discovered family or cluster.
• references/branch-specialization-role-inventory-big-sweep.md — branch-specialization status pattern after Gavin's correction: separate completed real-language hetero-vs-uniform roles/families from broad registry probes and Pythia causal roles; when workflow is unchanged, propose one organized big sweep and consolidated report instead of confusing incremental slices.
• references/branch-specialization-multi-llm-role-atlas.md — branch-specialization Phase 8 planning correction: use all readout-registry roles/axes as candidate ontology, add literature-derived LLM head roles, and validate a broad actual-LLM ordinary-head role atlas before narrowing to hetero-vs-uniform architecture tests.
• references/branch-specialization-phase8b-specificity-matrix.md — branch-specialization Phase 8B specificity gate: selected-vs-random ordinary-head effects are causal footholds, but require off-role/matched-off-role specificity and contrastive selection before promoting roles into hetero-vs-uniform architecture tests.
• references/branch-specialization-real-language-modularity.md — branch-specialization claim boundary from Gavin's correction: synthetic prompt-target batteries can support affinity/specialization but not real LLM modularity; move to real-language LM conditions before modularity claims.
• references/branch-specialization-real-language-phase4c.md — completed real-language WikiText hetero-vs-uniform synthesis pattern: check validation-loss matching, separate weak structural affinity from specialization/modularity, and prefer readout diagnostics over scaling narrow probes.
• references/branch-specialization-phase4c-v4-broad-matched-real-lm.md — full broad matched 288-row real-language branch-specialization synthesis pattern: Q1 structural affinity positive, Q2 specialization negative, Q3 scientific modularity unsupported; includes safe claim boundary and Discord-friendly pass-count table shape.
• references/branch-specialization-stratified-effect-maps.md — branch-specialization correction for stratified synthesis: Phase 4C supports role/family effect maps but not format/topic axes; Phase 4B supports synthetic operation/topic/format maps; report supported axes rather than one global verdict.

Research Results Synthesis

Gavin Value Action / ConsVal RQ synthesis

When Gavin asks for Value Action paper conclusions, old-RQ status, gap/no-gap interpretation, or Phase 3.x ConsVal results, first consult references/value-action-rq-synthesis.md. Key rule: do not make a paper-level conclusion from a single overlap metric or from intuition. Build the behavioral majority/gap table first, mark ties as TIE/weak, then run a metric multiverse that separates raw activation overlap, signed side-specific contrast alignment, row selection, and causal steering.