What is Attention Mechanism?

The component of a model that transforms input into latent representations.

A transformer mechanism that runs several attention operations in parallel to capture different types of relationships.

A positional bias method that penalizes attention scores based on token distance, helping models extrapolate to longer contexts.

A task where a model assigns an input to one or more predefined categories.

The maximum amount of input tokens a language model can process at once.

A shift in real-world input data over time that can degrade model performance.