AI Glossary Term
What does “Encoder” mean?
Definition
The component of a model that transforms input into latent representations.
How Encoder works
Example
In a translation system, an encoder reads a French sentence and turns it into numeric representations that capture its meaning. A decoder then uses them to write the English sentence.
Why it matters
Encoders turn raw data into forms models can reason over. Many search and classification systems use encoder-only models for this reason.
Common questions
Do all language models have encoders?
No. Many chat models are decoder-only. Others, like BERT, are encoder-only, and some, like T5, use both.
What is the output of an encoder?
A set of vectors, called hidden states or representations, one per input token or one for the whole input.
Is an encoder the same as an embedding model?
An embedding model is typically built on an encoder, but the term embedding model emphasizes its use for similarity.