In-Context Learning
In-context learning is the surprising ability of large language models to pick up a new task from a few examples placed in the prompt, without any retraining.
Overview
It is the reason you can 'teach' a model on the fly just by showing it what you want.
Deep Dive
Normally, teaching a neural network a new task means updating its weights through training. In-context learning is different: you write a few examples directly in the prompt (the 'context'), and the model infers the pattern and applies it to a new input. Nothing inside the model changes; the examples just steer the next-token prediction. You will hear 'zero-shot' (instruction only), 'one-shot' (one example), and 'few-shot' (several examples). This behavior was popularized by GPT-3 in 2020 and turned out to be an emergent ability: tiny models cannot do it, but past roughly 100-billion-parameter scale, accuracy on few-shot prompts climbs sharply. The model effectively learned to recognize and continue patterns during pretraining, so it can reuse that skill at inference time.
Technical Insight
Interpretability research traced much of this ability to 'induction heads' — attention circuits that emerge during training and perform fuzzy prefix-matching: they scan back for where a similar token appeared, then copy what followed it. So when your prompt shows 'apple -> fruit, carrot -> vegetable', the model matches the structure and predicts the right label for the next item. Crucially, no gradients flow and no weights update at inference. The examples simply reshape the activations that feed the next-token probability distribution.
Strategic Impact
Speed and scale
Language workflows can move faster without sacrificing consistency.
Access and reach
It expands access across languages and communication styles.
Clearer decisions
Teams can spend more time on judgment while automation handles repetition.
The Future of In-Context Learning
Expanding context windows (now hundreds of thousands of tokens) push in-context learning toward 'many-shot' regimes, where dozens or hundreds of examples can rival fine-tuning for some tasks, with no training cost. Expect tighter integration with retrieval, so relevant examples are fetched automatically, and better theory on when in-context learning fails or gets distracted. It will remain the fast, cheap way to adapt a model, complementing — not replacing — fine-tuning for stable, high-volume tasks.
Real-World Implementation
Giving a chatbot three example support tickets and their categories, then having it classify a new ticket the same way
Showing a model two before/after pairs of messy text reformatted into clean JSON so it converts the rest
Pasting a couple of sample product descriptions in your brand's tone so new ones match the style
Demonstrating a tricky math word-problem worked step by step so the model solves similar problems with the same reasoning format
Risks & Guardrails
Hallucinated facts can quietly enter reports, support flows, or research outputs.
Prompt sensitivity can create inconsistent results across similar requests.
Sensitive text data may be exposed if access controls are weak.
Implementation Roadmap
Define output format, tone, and quality standards before rollout.
Ground responses with trusted sources whenever accuracy matters.
Keep a human review checkpoint for high-stakes outputs.
Track failure patterns and retrain prompts or workflows regularly.
Keep Exploring
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the In-Context Learning quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Next guide
Context Windows
Frequently asked questions
What is In-Context Learning?
In-context learning is the surprising ability of large language models to pick up a new task from a few examples placed in the prompt, without any retraining. It is the reason you can 'teach' a model on the fly just by showing it what you want.
What fundamentally changes inside the model during in-context learning?
In-context learning happens entirely at inference. The examples in the prompt shape the model's activations and next-token predictions, but no weights are updated.
Why is in-context learning described as an 'emergent' ability?
The ability is weak or absent in small models and climbs steeply once models reach very large scale, which is why it is called emergent.
Which internal mechanism is most associated with in-context learning in transformers?
Interpretability work identified induction heads — attention circuits that find where a similar token appeared and copy what came next — as a core driver of in-context learning.