Agent Scaffolding and Harnesses Quiz

Answers and explanations appear as you take the quiz.

1. What is an agent harness?
   • The model's neural network weights
   • The code around a model that runs the loop, executes tools and decides when to stop
   • A dataset for training agents
   • A type of GPU
2. Which is an example of a stopping rule?
   • Using a larger model
   • A maximum number of steps or a cost budget
   • Adding more tools
   • Changing the font of outputs
3. What did the SWE-agent project emphasise?
   • That models need no tools
   • An agent-computer interface designed for models, with concise viewers and immediate feedback
   • That bigger GPUs solve all agent problems
   • Removing the loop entirely
4. Why can the same model score differently on SWE-bench in different reports?
   • The benchmark changes every hour
   • Scores reflect the model plus harness, tools, step limits and retry policy
   • Models forget their training between runs
   • Scores are randomly assigned
5. How should a harness usually handle a failed tool call?
   • Crash the entire run immediately
   • Return the error to the model as a tool result it can react to
   • Silently ignore it
   • Restart the computer
6. What is the purpose of capping tool output size and marking truncation?
   • To hide information from users
   • To keep context manageable and let the model know to search or page for more
   • To make tools run faster on the GPU
   • To prevent the model from using tools
7. When fairly comparing two models as agents, what should you hold constant?
   • Only the model's name
   • The harness, prompt, tools, step limit and retry policy
   • Nothing; vary everything
   • The date of the test only
8. A harness pauses before large refund tool calls for human approval. What design principle does this show?
   • Context compaction
   • Gating side-effecting tools with permissions or confirmation
   • Parallel function calling
   • Embedding fine-tuning