AI Glossary Term

What does “Quantization” mean?

Definition

Converting model weights to lower precision formats such as 8-bit or 4-bit.

Browse the full AI Glossary Explore all free AI guides

How Quantization works

Example

A model file that is about 14 GB at 16-bit precision can shrink to roughly a quarter of that at 4-bit, so it can run on a laptop. Outputs may be slightly different from the original.

Why it matters

Quantization makes it practical to run models on cheaper or local hardware. Testing quality on your own tasks is still important.

Common questions

What do 4-bit and 8-bit mean?

How many bits are used to store each number. Fewer bits mean less memory and some loss of precision.

Is it reversible?

Not exactly. Original precision cannot be perfectly recovered from the quantized version.

Does it work for training too?

Mostly used for inference, though methods like QLoRA combine it with fine-tuning.

Model QuantizationReducing numeric precision of model weights to decrease memory and inference cost.QLoRAA fine-tuning technique that combines 4-bit weight quantization with LoRA adapters to reduce memory needs.BackpropagationThe core training algorithm that updates model weights by propagating prediction errors backward through the network.Convolutional Neural Network (CNN)A neural architecture optimized for processing grid-like data such as images.Feature ExtractionConverting raw data into informative features that a model can use.Generative AIAI systems that produce new content such as text, images, audio, video, or code.

Learn more in our free guides

Residual Vector Quantization SmoothQuant and Activation Quantization GPTQ and AWQ Post-Training Quantization Quantization

Test yourself

Residual Vector Quantization Quiz SmoothQuant and Activation Quantization Quiz GPTQ and AWQ Post-Training Quantization Quiz