Elhage, Olsson, Nanda, and colleagues at Anthropic published A Mathematical Framework for Transformer Circuits, modeling the Transformer residual stream as a linear communication bus and factoring attention heads into QK (what attends to what) and OV (what moves forward) matrices to explain induction heads in small models.
Mechanistic Interpretability
Also known as: Mech Interp · Transformer Circuits · Sparse Autoencoders · SAEs · Dictionary Learning · Monosemanticity · Attribution Graphs · Cross-Layer Transcoders
Plain English
A research discipline that reverse-engineers neural networks from the inside out—mapping hidden weights and activations into human-understandable concepts and circuits. Rather than judging an AI model only by its final outputs, mechanistic interpretability examines the internal algorithms it executes.
Technical Overview
Mechanistic interpretability decomposes Transformer residual streams, QK/OV attention circuits, and MLP layers into sparse, causal components even when superposition packs far more features than neurons into polysemantic activations. Unsupervised dictionary learning—using sparse autoencoders (SAEs) and cross-layer transcoders (CLTs)—extracts monosemantic concepts and reconstructs multi-step attribution graphs across production models (Anthropic's Transformer Circuits, Google DeepMind's Gemma Scope 2). By 2025–2026, methods expanded beyond post-hoc SAEs to natively weight-sparse circuits (OpenAI) and Natural Language Autoencoders that verbalize activations directly for safety auditing and evaluation awareness detection.
Timeline (6)
2021
Paper2023
PaperBuilding on Toy Models of Superposition (
arXiv:2209.10652), Bricken et al. published Towards Monosemanticity, using sparse autoencoders (SAEs) to unpack a polysemantic 512-neuron MLP layer into over 4,000 monosemantic dictionary features—isolating crisp directions for code syntax, DNA sequences, and specific scripts.2024
PaperTempleton et al. published Scaling Monosemanticity, applying dictionary learning to a frontier production model (Claude 3 Sonnet) to isolate millions of multimodal and multilingual features and verify causal steering by clamping individual directions—demonstrated publicly two days later as "Golden Gate Claude."
2025
PaperAmeisen, Lindsey, and Anthropic's interpretability team published Circuit Tracing and On the Biology of a Large Language Model, using cross-layer transcoders (CLTs) to construct feature attribution graphs in Claude 3.5 Haiku—revealing multi-hop reasoning, advance rhyme planning, and shared cross-lingual concept spaces before open-sourcing
circuit-tracerin May 2025.- Release
Google DeepMind released Gemma Scope 2 (alongside OpenAI's November 2025 Weight-sparse transformers study,
arXiv:2511.13653), open-sourcing JumpReLU sparse autoencoders, skip-transcoders, and cross-layer transcoders across Gemma 3 models up to 27B parameters so researchers can audit refusal, hallucination, and safety circuits. 2026
PaperFraser-Taliente, Kantamneni, Ong, and colleagues introduced Natural Language Autoencoders (NLAs), jointly training an activation verbalizer and reconstructor with reinforcement learning so residual-stream states decode directly into natural-language explanations—surfacing unverbalized task strategies and evaluation awareness without manual feature labeling, alongside July 2026's Jacobian-lens Global Workspace study (
arXiv:2607.15495).
See also (10)
Open full graphMechanistic Interpretability