AI Glossary Term
What does “Transformer” mean?
Definition
A neural architecture that uses attention to model relationships across sequences in parallel.
How Transformer works
Example
A transformer reads a whole sentence at once. For each word it weighs how relevant every other word is, so in 'The bank by the river' it can use 'river' to interpret 'bank'.
Why it matters
The transformer, introduced in 2017, is the basis of most modern language models and many image and audio models.
Common questions
Which paper introduced it?
'Attention Is All You Need', published by researchers at Google in 2017.
Why did it replace recurrent networks?
It processes tokens in parallel, which trains faster on modern hardware, and handles long-range relationships well.
Is a transformer only for text?
No. Variants process images, audio, video, and protein sequences.