AI Glossary Term

What does “QLoRA” mean?

Definition

A fine-tuning technique that combines 4-bit weight quantization with LoRA adapters to reduce memory needs.

Related terms

LoRA (Low-Rank Adaptation)
A parameter-efficient fine-tuning method that adds low-rank adapter matrices.
Quantization
Converting model weights to lower precision formats such as 8-bit or 4-bit.
Fine-Tuning
Continuing training on domain-specific data to adapt a pre-trained model to a specific task.
Instruction Tuning
Fine-tuning a model on instruction-response pairs to improve task following.
Model Quantization
Reducing numeric precision of model weights to decrease memory and inference cost.
Parameter-Efficient Fine-Tuning (PEFT)
Methods that adapt models by training a small subset of added parameters.

Learn more in our free guides

QLoRA and 4-Bit Fine-Tuning

See also

DPO (Direct Preference Optimization)Flash AttentionTree of ThoughtsMulti-Head AttentionReActPositional EncodingAgentic LoopRoPE (Rotary Positional Embeddings)
Browse the full AI GlossaryExplore all free AI guides