Mechanistic interpretability asks a deceptively simple question: what is happening inside a large language model when it produces a token? Recent work has given us a powerful lens — sparse transcoders and attribution graphs, which decompose a forward pass into thousands of interpretable features and trace the causal paths between them. My independent research project began where that lens stops being useful. An attribution graph for a single prompt can surface hundreds of candidate features and confirming that any one of them matters requires an intervention: ablate the feature, rerun the model, measure how far the output distribution moves. Each test costs a forward pass. Nobody can run them all, so in practice researchers rank features by attribution score and probe the top few. That heuristic is cheap — but nobody had asked whether it is a good way to spend a limited experimental budget.
Neuroscience has been asking a structurally identical question for over a decade. Under the active inference framework, a perceiving agent does not passively receive data; it chooses actions that minimise expected free energy, trading the value of reaching a goal against the value of resolving uncertainty. The canonical example is a saccade: where should the eye look next to learn the most about the scene? Swap “eye” for “interpretability researcher” and “scene” for “transformer circuit”, and it is the same problem wearing different clothes. Testing that correspondence is what the project set out to do.
Active Circuit Discovery (ACD) implements it. A partially observable Markov decision process (POMDP) agent, built on pymdp and sitting on top of Anthropic’s circuit-tracer library, maintains a multi-factor generative model over three hidden state factors: a feature’s importance, its role in the layer hierarchy, and its causal influence. At each step the agent selects both which feature to target and which type of intervention to run, by minimising expected free energy over that belief state. Crucially, it does not assume it knows how interventions map to outcomes. The observation model is learned online through Dirichlet parameter updates, so the agent’s model of the model improves as it experiments.
I evaluated ACD on Gemma-2-2B and Llama-3.2-1B across four settings: indirect object identification, multi-step reasoning, feature steering, and a multidomain benchmark. Every number comes from real feature-intervention calls against GemmaScope and Llama transcoders — no simulated environment, no synthetic data. On Gemma IOI, the ablation-only agent reached 82.0% of oracle efficiency within a budget of twenty interventions per prompt, and it beat random feature selection consistently. That advantage did not reach statistical significance at the sample sizes an MSc project can afford, and it did not hold uniformly on Llama-3.2-1B, whose smaller transcoders produce a sparser and noisier attribution graph. Both facts are in the paper, stated plainly.
The result I find most useful is a negative one. In the steering experiments, features selected by circuit analysis were only marginally — and not significantly — more steerable than random active features at large multipliers. In other words, a substantial share of what looks like targeted causal control in feature-steering demonstrations is generic activation scaling: turn almost any active feature up tenfold and the model’s behaviour shifts. That is a real caution for a literature increasingly using steering as evidence of mechanistic understanding, and it only became visible because a pre-specified hypothesis forced a random baseline into the experimental design.
The whole pipeline is reproducible and runs on a free T4 GPU: code, Colab notebooks, Docker images and raw result files are all public. In collaboration with my supervisor, I subsequently published this research in the MDPI journal Symmetry and was invited to present our work to the Active Inference community.
The hardest lesson was learning to want the null result. I started out hoping to show that an active inference agent would decisively outperform attribution ranking, and for a few weeks I caught myself sketching analyses that would have been kinder to that hope than the data deserved.
The hardest lesson was learning to want the null result. I started out hoping to show that an active inference agent would decisively outperform attribution ranking, and for a few weeks I caught myself sketching analyses that would have been kinder to that hope than the data deserved. Committing to four hypotheses with pre-specified acceptance criteria, before running the full experiment suite, is what fixed it — and it changed the project. The steering finding, the contribution I am most confident in, exists only because a hypothesis obliged me to run a baseline I would otherwise have skipped.
The second lesson was that engineering is research infrastructure, not overhead. A large share of my time went into making the pipeline reproducible on hardware I do not own — Docker images for two architectures, a T4-compatible path, notebooks anyone can open in a browser. That work produced no results of its own, but it is the reason the reviewers, and now anyone reading, can check the claims rather than take them on trust.
Coming to interpretability from supercritical CO₂ power cycles and inertial navigation, I expected the domain shift to be the difficulty. It was not. The transferable skill turned out to be the habit of asking what would falsify this and then building the rig that could.
Leave a Reply