Mixture of LoRA Experts Quiz

Answers and explanations appear as you take the quiz.

1. What does the 'LoRA' part of Mixture of LoRA Experts refer to?
   • Low-Rank Adaptation, which trains small low-rank matrices instead of full weights
   • A reinforcement learning reward model
   • A long-range attention mechanism
   • A data-loading pipeline for adapters
2. What is the job of the gating/router network in MoLE?
   • To compress the base model weights
   • To decide which experts to activate and with what weights for a given input
   • To generate training data for the adapters
   • To convert text into embeddings
3. Why is MoLE often better than training one adapter on many mixed tasks?
   • It uses more GPU memory by design
   • It eliminates the need for any base model
   • It reduces interference and forgetting by keeping skills in separate composable experts
   • It only works on a single language
4. In LoRA, the weight update is typically expressed as which of the following?
   • A full dense matrix replacing W
   • A single scalar multiplier on W
   • A random noise matrix added each step
   • The product of two low-rank matrices, B times A
5. During MoLE training, what generally happens to the base model's pretrained weights?
   • They stay frozen while only adapters and the router are trained
   • They are fully retrained from scratch
   • They are deleted and rebuilt
   • They are quantized to one bit
6. In diffusion image models, what problem does hierarchical/per-layer gating in MoLE help solve?
   • Slow internet downloads
   • Multiple concept LoRAs overpowering each other when composed
   • Lack of training data
   • Incorrect file formats
7. How can a gated LoRA mixture differ from merging all adapter weights into one fixed update?
   • It must retrain the pretrained backbone for every prompt
   • It can use only one expert for all inputs
   • It can select or weight experts conditionally for a task or layer
   • It removes every low-rank matrix before inference
8. In a LoRA update with rank r, what does r control most directly?
   • The number of decoder tokens generated
   • The number of original model layers removed
   • The size of the training vocabulary
   • The dimension of the low-rank factors and thus the capacity and parameter cost of the update