Up nextNext guide
Logit Lens and Intermediate Layer Decoding
Language AI
Technical GUIDE
Logit Lens and Tuned Lens
The logit lens and tuned lens are interpretability techniques that peek at a transformer's hidden states layer by layer to see what the model is 'thinking' before it produces a final answer.
Listen to this guideAI-narrated · ~2 min
On this page2 min read
Overview
They reveal how a prediction gradually forms as information flows up through the network.
Deep Dive
A transformer builds its answer incrementally: each layer adds to a running 'residual stream' that is only turned into word probabilities at the very end. The logit lens, introduced by nostalgebraist in 2020, shortcuts this by applying the model's final unembedding (and layer norm) directly to intermediate layers, so you can read out the network's best guess at every depth. This often shows the answer crystallizing in middle-to-late layers. The tuned lens (Belrose and colleagues, 2023) improves on it by training a small affine probe per layer to translate hidden states into the final basis, fixing the bias and inaccuracy the raw logit lens suffers, especially in early layers and across different model families.
Real-World Implementation
Using the logit lens to watch a factual answer like a capital city emerge in a model's middle layers
Applying the tuned lens to compare how different model families converge on a prediction across depth
Detecting that a model has internally 'decided' an answer several layers before the output
Diagnosing layers where harmful or biased token predictions first become dominant in the residual stream
The Future of Logit Lens and Tuned Lens
Lens techniques are becoming standard for tracing how facts, refusals, or biases emerge across depth, and for spotting when a model 'knows' an answer early. Expect them combined with sparse autoencoders and causal patching to move from describing predictions to explaining mechanisms. Research is also probing whether intermediate readouts reveal latent knowledge or deception a model conceals in its final output, making lenses a candidate building block for safety audits and early-warning monitoring.
Putting it into practice
Why it matters
- Cost and budget. Architecture decisions drive performance and operating cost for years.
- Clearer decisions. Technical education helps teams choose the right stack, not just the newest one.
- Quality control. Better engineering choices reduce reliability incidents in production.
Watch out for
- Optimizing one benchmark can hide broader system weaknesses.
- Infrastructure and maintenance costs are often underestimated.
- Security and observability gaps can grow as systems become more complex.
Next steps
- Define latency, quality, and cost targets before implementation.
- Benchmark under realistic load and data conditions.
- Instrument monitoring for errors, drift, and user impact.
- Prepare rollback and incident response paths before scaling.
Keep Exploring
Free newsletter
Keep up with AI in 3 minutes a day
One short email each weekday with the three AI stories that actually matter. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Logit Lens and Tuned Lens quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Frequently asked questions
What is Logit Lens and Tuned Lens?
The logit lens and tuned lens are interpretability techniques that peek at a transformer's hidden states layer by layer to see what the model is 'thinking' before it produces a final answer. They reveal how a prediction gradually forms as information flows up through the network.
What does the logit lens do?
The logit lens projects intermediate residual-stream states through the existing unembedding to see the model's predicted tokens at each depth.
What key improvement does the tuned lens add over the raw logit lens?
The tuned lens learns a per-layer linear translator, correcting biases and giving more faithful, lower-perplexity readouts than the raw logit lens.
What structure in transformers makes these lenses possible?
Each layer writes additive updates to a shared residual stream, so projecting that stream early approximates an intermediate prediction.
Who introduced the original logit lens, and roughly when?
The logit lens was introduced by nostalgebraist in a 2020 blog post analyzing GPT-2's intermediate predictions.
Where does the logit lens often show the final answer 'crystallizing'?
Readouts typically show the correct token gaining probability across middle-to-late layers as computation accumulates.
Keep learning
Related guides
More guides picked for this topic