QLoRA and 4-Bit Fine-Tuning Quiz

Answers and explanations appear as you take the quiz.

1. What is the core memory-saving idea behind the '4-bit' part of QLoRA?
   • It quantizes the frozen base model's weights down to 4 bits each
   • It deletes 75% of the model's layers
   • It trains the model in only 4 epochs
   • It uses only 4 GPUs instead of many
2. During QLoRA fine-tuning, which parameters actually get updated by gradient descent?
   • All of the model's original weights
   • Only the small low-rank LoRA adapter matrices
   • The 4-bit quantization constants
   • The tokenizer's vocabulary
3. What is NF4 in the context of QLoRA?
   • A 4-layer neural network
   • A new optimizer algorithm
   • A 4-bit data type optimized for normally distributed weights
   • A 4-node GPU cluster format
4. What problem do 'paged optimizers' in QLoRA address?
   • Slow internet downloads of model files
   • Incorrect tokenization
   • Overfitting on small datasets
   • Out-of-memory spikes during long sequences
5. When are the 4-bit weights converted back to higher precision during training?
   • Just-in-time during the matrix multiplication, then discarded
   • They are never converted; math is done in 4 bits
   • Only once at the very start of training
   • Only after training completes
6. What did the original QLoRA paper demonstrate about quality?
   • 4-bit fine-tuning always beats full fine-tuning
   • Fine-tuning a 65B model on one GPU could match full 16-bit fine-tuning quality
   • Quantization makes models useless below 8 bits
   • Only tiny models can be fine-tuned this way
7. What does QLoRA’s double-quantization step compress?
   • The tokenizer vocabulary
   • The LoRA adapter rank
   • The quantization constants used to represent the 4-bit weights
   • The training examples’ text labels
8. How can one QLoRA base model support multiple task-specific versions?
   • They delete the optimizer state after every token
   • They increase the base model from 4-bit to 32-bit
   • They move all activations into the tokenizer
   • Reuse the frozen 4-bit base and swap small trainable LoRA adapters for different tasks