Agent Scaffolding and Harnesses

A language model on its own produces text. An agent needs a program that repeatedly asks the model what to do, performs the action, and shows the model what happened. That program is the harness. Its core parts are the loop, tool definitions, tool dispatch, context management, stopping rules and error handling. The loop sends the conversation so far to the model. If the model requests a tool, the harness validates the arguments, runs the tool, and appends the result. If the model gives a final answer, the loop ends. Stopping rules add limits: a maximum number of steps, a token or cost budget, a timeout, or a check such as tests passing. Tool design has large effects. The SWE-agent project from Princeton researchers, published in 2024, argued that agents need an agent-computer interface designed for models, with concise file viewers, search commands and immediate feedback on bad edits, and showed this improved results over giving the model a plain shell. Output formatting matters too: truncating huge outputs, summarising errors clearly and returning structured results help the model act correctly. Context management decides what the model sees as the history grows: keeping recent steps, summarising older ones, or storing notes in files. Error handling decides whether a failed tool call crashes the run or is returned to the model as information it can react to. This is why benchmark numbers need context. On coding benchmarks such as SWE-bench, reported scores for one model vary across scaffolds, retry policies and step limits. A common misconception is that a leaderboard number measures the model alone; it measures the model plus harness plus settings. Products such as command-line coding agents are, in large part, carefully engineered harnesses around a model.