Representation and Parameter Analysis

Advantage

Features and weights provide a single low-dimensional handle for both analysis and control, with no retraining needed. Once a concept direction is identified, reading becomes a projection and editing becomes an addition — the same primitive supports monitoring, classification, steering, model editing, and capability composition. This is far cheaper than the intervention sweeps in Causal Attribution or full fine-tuning. Linearity also makes operations compose: directions can be subtracted to contrast concepts, and task-level weight edits can be added or negated to mix in or remove capabilities, each step with a clear semantic meaning.

Limitation

The approach assumes target concepts are linearly encoded. When the encoding is non-linear, distributed, or entangled with other concepts, a single direction misses much of the signal and additive edits spill into neighboring features. Results are also basis- and checkpoint-specific — directions rarely transfer across models — and large edit strengths push the model off-distribution, breaking the linear regime the method relies on.

Submethods

The category typically takes 4 forms:

• Representation Engineering: Reads, monitors, and edits LLM behavior by extracting concept directions from hidden-state features using contrastive prompt pairs. RepReading projects features onto the concept direction to detect attributes such as truthfulness, emotion, fairness, harmlessness, or memorization, while RepControl writes the same direction back into the residual stream during generation to steer the model toward or away from the concept. You can find a demo for this method in ./representation-engineering. This demo shows representation-engineering: Use this skill when working with Representation Engineering (RepE) for AI transparency, monitoring, or controlling internal representations of large language models including truthfulness detection, emotion control, harmlessness steering, and memorization analysis.
• Steering Vectors: A focused instance of directly editing features at inference time: steering directions are built from contrastive activation pairs (Contrastive Activation Addition / CAA) and added to a chosen layer's residual stream during generation. It is especially useful for behavioral control of chat models, where concepts of interest are easy to frame via paired examples of desired vs. undesired behavior. You can find a demo for this method in ./steering-vectors. This demo shows steering_vectors: Work with Llama 2 model steering using Contrastive Activation Addition (CAA) for behavioral control, including generating steering vectors, evaluating model behavior, and analyzing activation patterns.
• Steering features: Directly intervenes on the target feature rather than adding a separately built steering vector to the residual stream. The feature that corresponds to the target behavior is first located and scored for how strongly it drives the output — keeping features with a high output effect over merely input-correlated ones — and during generation its activation is directly amplified or shrunk to steer behavior. When the feature lives in a Sparse Autoencoder (SAE), this takes the encode → amplify target feature → decode form (Feature Clamping): the SAE encodes the residual stream into feature space, the target feature's activation is scaled, and the result is decoded back. The SAE is not essential, though — the same direct-intervention idea applies to a feature identified in the model's own backbone activations, where its activation is scaled in place with no encode/decode step. Because control acts on one isolated, near-monosemantic feature, it is more targeted and less entangled than dense contrastive steering vectors. You can find a demo for this method in ./steer-features. This demo shows steer-features: Use this skill when working with Sparse Autoencoders (SAEs) for model steering, feature analysis, or interpretability research. Particularly useful for selecting and scoring SAE features to influence model outputs toward desired concepts.
• Parameter-Space Task Vectors: Lifts the same idea to weight space and edits weights directly: a task vector is the displacement between a fine-tuned checkpoint and its pre-trained checkpoint, and arithmetic on these vectors — addition for multi-task composition, negation for unlearning, and analogy-style combinations across related tasks — modifies model behavior without any further training. You can find a demo for this method in ./parameter-space-task-vectors. This demo shows task-vectors: Use this skill when working with task arithmetic for editing neural network models, including creating task vectors from pre-trained and fine-tuned checkpoints, combining them via arithmetic operations (negation, addition, analogies), and applying them to CLIP vision models for multi-task learning.

Practical Notes

Default choice. If the user has not specified a particular method, recommend Steering features first — it is the simplest to set up and run.

Check for an available SAE. Before implementing, search whether a Sparse Autoencoder has already been released for the target model (and for which layers). If one exists, use it to locate the target feature: SAE features are more monosemantic/interpretable than directions read off the raw backbone activations, and intervening on them usually gives a cleaner, stronger effect. If no SAE is available, fall back to a feature identified directly in the model's own activations — the same direct-intervention procedure applies, without the encode/decode step.