Mixtral and Sparse Models Quiz

Answers and explanations appear as you take the quiz.

1. In Mixtral 8x7B, how many experts process each individual token?
   • Two experts
   • All eight experts
   • Exactly one expert
   • A random number each time
2. What component decides which experts a token is sent to?
   • The tokenizer
   • The router (gating network)
   • The attention layer
   • The embedding table
3. Although Mixtral 8x7B has about 47B total parameters, roughly how many are active per token?
   • About 47 billion
   • About 7 billion
   • About 13 billion
   • About 70 billion
4. What must a Mixtral serving setup account for even when only two experts run per token?
   • Only the active parameters ever need to be stored anywhere
   • All experts must always fit in GPU memory with no offloading option
   • Unused experts can be deleted without changing the checkpoint
   • The full expert weights must be available to the system, with residency or offloading depending on deployment
5. Under which license did Mistral AI release Mixtral, fueling open adoption?
   • Apache 2.0
   • Proprietary, closed-source only
   • GPL v3
   • Creative Commons NonCommercial
6. What training steps does Mistral report for the Mixtral 8x7B Instruct variant?
   • It removes the need for any training data
   • Supervised fine-tuning and direct preference optimization (DPO)
   • It eliminates the need for GPUs entirely
   • It guarantees models never make mistakes
7. How many expert networks are available in each Mixtral 8x7B sparse mixture-of-experts layer?
   • Two experts
   • Seven experts
   • Eight experts
   • Twelve experts
8. Which languages does Mistral list for Mixtral 8x7B?
   • Yes; all unused expert weights are deleted
   • Yes; it means the model is dense with 12.9B total parameters
   • No; it means every parameter is active for every token
   • English, French, Italian, German, and Spanish