quiz bu lëkkale ak tegtal · Diggante Niveau

Quiz d'échafaudage et harnais d'agent

Dafay saytu bouclage agent bi ak dispatch jumtukaay bi, taxawal sàrt yi ak njuumte yi, ak lu tax benn model bi di am poñ yu wuute ci harnais yi.

Yooni njiit yu jëm ciAgent Échafaudage et Harnais
Laaj 1 de 8

What is an agent harness?

Xoolaatal laaj bu nekk8 laaj

Sooy def quiz bi, tontu yi ak leeral yi dañuy feeñ.

  1. What is an agent harness?
    • The model's neural network weights
    • The code around a model that runs the loop, executes tools and decides when to stop
    • A dataset for training agents
    • Benn xeetu GPU
  2. Which is an example of a stopping rule?
    • Using a larger model
    • A maximum number of steps or a cost budget
    • Adding more tools
    • Changing the font of outputs
  3. What did the SWE-agent project emphasise?
    • That models need no tools
    • An agent-computer interface designed for models, with concise viewers and immediate feedback
    • That bigger GPUs solve all agent problems
    • Removing the loop entirely
  4. Why can the same model score differently on SWE-bench in different reports?
    • The benchmark changes every hour
    • Scores reflect the model plus harness, tools, step limits and retry policy
    • Models forget their training between runs
    • Scores are randomly assigned
  5. How should a harness usually handle a failed tool call?
    • Crash the entire run immediately
    • Return the error to the model as a tool result it can react to
    • Silently ignore it
    • Restart the computer
  6. What is the purpose of capping tool output size and marking truncation?
    • To hide information from users
    • To keep context manageable and let the model know to search or page for more
    • To make tools run faster on the GPU
    • To prevent the model from using tools
  7. When fairly comparing two models as agents, what should you hold constant?
    • Only the model's name
    • The harness, prompt, tools, step limit and retry policy
    • Nothing; vary everything
    • The date of the test only
  8. A harness pauses before large refund tool calls for human approval. What design principle does this show?
    • Context compaction
    • Gating side-effecting tools with permissions or confirmation
    • Parallel function calling
    • Embedding fine-tuning