Checkpoint Sharding and Resumable Training Quiz

Answers and explanations appear as you take the quiz.

1. Why are large-model checkpoints split into shards?
   • To make the model train faster on small inputs
   • To hide the model weights from users
   • Because sharding improves accuracy
   • Because a single checkpoint can be too large for one file or machine to handle efficiently
2. Which extra state is typically needed for training resumption beyond model weights?
   • Only the final accuracy score
   • Optimizer and scheduler state, step progress, RNG state, and relevant data-loader progress
   • Only a model-weight shard index
   • A screenshot of the training dashboard
3. What is the main benefit of resumable training for long jobs?
   • It removes the need for a validation set
   • It guarantees the model never overfits
   • A crash or preemption does not force restarting from scratch
   • It eliminates the optimizer
4. In a sharded distributed checkpoint, how can ranks avoid gathering the entire state on one worker?
   • By saving their assigned local tensor shards with checkpoint metadata
   • By each emailing a full model copy
   • By deleting optimizer state before every save
   • By disabling all state serialization
5. What role does an index file play in sharded checkpoints?
   • It stores the user's password
   • It maps each parameter name to the shard file that contains it
   • It compresses the images in the dataset
   • It schedules the learning rate
6. Which format is commonly used for safe, memory-mapped loading of model shards?
   • MP3
   • CSV
   • safetensors
   • ZIP-only archives
7. A package contains safetensors weight shards and an index, but no optimizer state. What can it support by itself?
   • Loading model weights, without fully restoring optimizer training history
   • Guaranteed exact resumption of every interrupted run
   • Recovering the next batch position automatically
   • Reconstructing every random-number-generator state
8. A sharded checkpoint loader supports a new rank layout. What should the team verify before trusting recovery?
   • That fewer ranks guarantee identical floating-point results
   • That the weight index alone includes every application state
   • That no restart test is necessary
   • That the necessary saved state loads correctly and the resumed training step behaves as expected