Inverse Reinforcement Learning Quiz

Answers and explanations appear as you take the quiz.

1. What does inverse reinforcement learning aim to recover?
   • A reward function that explains observed expert behavior
   • A faster optimizer for gradient descent
   • A compressed version of the demonstrations
   • A labeled image dataset
2. Why is the IRL problem described as ill-posed or ambiguous?
   • Because demonstrations are always perfectly optimal
   • Because many different reward functions can explain the same behavior
   • Because rewards must always be negative
   • Because it requires no expert data
3. What key advantage does recovering a reward have over behavioral cloning?
   • It needs no demonstrations
   • It always runs in constant time
   • It captures the underlying goal, generalizing to unseen states
   • It avoids using any policy
4. How does maximum-entropy IRL resolve reward ambiguity?
   • By forcing all rewards to be equal
   • By ignoring suboptimal trajectories completely
   • By removing the reward function
   • By modeling trajectory probability as exponential in total reward
5. After IRL recovers a reward function, what is typically done next?
   • Standard reinforcement learning derives a policy from that reward
   • The reward is discarded
   • The demonstrations are deleted to save space
   • The agent stops learning
6. In which scenario is IRL especially useful?
   • When the reward is already perfectly known
   • When you want to understand and generalize the goals behind expert behavior
   • When no behavior data exists
   • When you only need to memorize fixed actions
7. Why can a constant zero reward explain many different expert policies?
   • It assigns the highest reward only to expert actions
   • It proves the expert is optimal
   • Every policy receives the same return, so none is preferred by the reward
   • It makes the transition model deterministic
8. In maximum-entropy IRL, how are higher-reward trajectories treated relative to lower-reward ones?
   • They are assigned probability zero
   • They are treated as equally likely regardless of reward
   • They are removed before policy learning
   • They receive higher probability, with probability depending exponentially on total reward