Small Language Models Quiz

Answers and explanations appear as you take the quiz.

1. What resource goal motivates a compact language-model implementation?
   • Guaranteeing it beats every larger model on every task
   • Reducing resource demands for a chosen task and device setup
   • Eliminating the need for all training data
   • Proving every phone runs every checkpoint identically
2. What training factor do the cited Phi studies investigate alongside model size?
   • Curated, filtered or synthetic training-data choices
   • Only random unfiltered data with no objective
   • Removal of all model parameters
   • A requirement to train only at night
3. What does teacher/student knowledge distillation train the student to learn from?
   • Only a new image-file extension
   • A guarantee of every possible teacher capability
   • The teacher’s outputs or distributions under a stated objective
   • Only a hardware-temperature label
4. What does weight quantization change in the storage setup described here?
   • Necessarily the number of model parameters
   • The language of every output sentence
   • Only the order of filenames
   • The numerical representation and bits used for weights
5. Which compression operation removes or suppresses selected network connections?
   • Pruning
   • Only changing a prompt’s font
   • Only enlarging the vocabulary
   • Only retaining every weight at its original value
6. What work does a key-value attention cache avoid repeating during autoregressive generation?
   • Every possible future training operation
   • All text-tokenization work for every application
   • Recomputing prior keys and values for subsequent token processing
   • The need to retain any state at all
7. Ignoring all overhead, how much raw storage is needed for 3.8 billion scalar weights packed at 4 bits each?
   • 3.8 billion bytes
   • 15.2 billion bytes
   • 0.475 billion bytes
   • 1.9 billion bytes
8. How many training tokens does the original Phi-3 report state for phi-3-mini?
   • 3.8 billion tokens
   • 3.3 trillion tokens
   • Only 3.3 million tokens
   • No training-token count was stated