Multi-Agent Reinforcement Learning Quiz

Answers and explanations appear as you take the quiz.

1. What makes the environment 'non-stationary' from one agent's perspective in MARL?
   • Other agents are simultaneously learning and changing their own policies
   • The reward function is randomly reset each episode
   • The state space grows without bound over time
   • Sensors return noisy observations
2. What does 'centralized training with decentralized execution' (CTDE) mean?
   • Agents are trained and run on a single central server
   • A shared critic uses global information during training, but agents act on local observations at deployment
   • One master agent issues commands to all others at runtime
   • Training data is centralized but rewards are decentralized
3. Which mathematical framework generalizes the single-agent MDP to multiple agents?
   • A linear program
   • A hidden Markov model
   • A Markov game (stochastic game)
   • A finite-state automaton
4. In a purely cooperative MARL setting, how is reward typically structured?
   • Each agent maximizes a reward that is the negative of another's
   • Only the fastest agent receives any reward
   • Rewards are assigned randomly each step
   • Agents share a common team reward they jointly maximize
5. Which training approach did OpenAI Five use to improve at Dota 2?
   • Self-play against copies of its own current and past policies
   • A fixed lookup table of human moves
   • A supervised-only classifier trained on image labels
   • A single policy trained only against a static scripted opponent
6. Why is 'credit assignment' especially hard in cooperative MARL?
   • Rewards arrive instantly and are easy to trace
   • It is difficult to determine which agent's actions contributed to a shared team outcome
   • There is only ever one agent receiving reward
   • Agents cannot observe any reward at all
7. What structural constraint does QMIX place on its mixing network?
   • It must be a linear sum with fixed weights
   • It ignores all individual agent values
   • The joint action-value is monotonic in each agent’s individual value
   • It requires communication at every action step
8. How does Value Decomposition Networks (VDN) form the team action-value?
   • By multiplying all rewards by the number of agents
   • By selecting the largest local reward only
   • By training a separate team for each agent
   • By summing the individual agents’ action-values