1-Bit and Ternary BitNet Models Quiz

Answers and explanations appear as you take the quiz.

1. Why is BitNet b1.58 described as using approximately 1.58 bits per weight?
   • Because each weight has three possible values (-1, 0, +1) and log2(3) is about 1.58
   • Because it rounds to 2 bits
   • Because it uses 1.58 GB of memory
   • Because 1.58 is the average of 1 and 2 bits
2. What makes BitNet's matrix operations cheaper than in standard models?
   • They skip the attention mechanism
   • Weights of -1, 0, +1 turn multiplications into additions and subtractions
   • They use a smaller vocabulary
   • They run only on TPUs
3. A key difference between BitNet and ordinary post-training quantization is that BitNet:
   • Only quantizes after training is complete
   • Cannot be trained at all
   • Is trained from scratch with low-bit weights
   • Uses 32-bit weights internally
4. During training, how does BitNet update weights despite the non-differentiable quantization step?
   • It freezes all weights
   • It trains without gradients
   • It uses reinforcement learning only
   • It uses a straight-through estimator with higher-precision shadow weights
5. What additional benefit does allowing the value 0 (ternary, not pure binary) provide?
   • Built-in sparsity, since zeroed weights can be skipped
   • Higher precision than floating point
   • Faster internet connectivity
   • Larger vocabulary size
6. What is the main hardware challenge for realizing BitNet's full efficiency gains?
   • There are no GPUs left in the world
   • Most current chips are optimized for floating-point math, not addition-only ternary ops
   • Ternary models need quantum computers
   • The models are too small to matter
7. If a BitNet weight is zero, what does it contribute to a dot product?
   • It multiplies the activation by negative one
   • It adds a full-precision matrix multiply
   • It contributes zero and can potentially be skipped by suitable hardware
   • It changes the number of model layers
8. Why are higher-precision shadow weights retained during BitNet training?
   • To perform inference with full-precision weights only
   • To store the tokenizer vocabulary
   • To replace the straight-through estimator with exact gradients
   • To maintain trainable values for gradient updates despite low-bit forward quantization