Research note · September 30, 2026

AI Model vs AI Agent vs Agent Harness

Hatsunama researches AI systems across the model, agent, and harness layers. Understanding the distinction between these layers helps explain why two AI systems using the same model can perform differently.

The three terms describe different things

An AI model transforms input into output. A language model receives tokens and predicts further tokens. Depending on its training and interface, that output might be prose, code, a structured object, or a proposed tool call. The model can be capable of reasoning about an action without having any ability to perform it. Its weights do not contain your current files, permissions, browser state, or the result of a command that has not run yet.

An AI agent is a system that uses a model, observations, and actions to pursue a task over one or more steps. The term describes behavior at the system level. A user asks for an outcome; the system decides what to do; it may call tools; it observes results; it continues or stops. Some agents are simple loops with one tool. Others coordinate many tools, memories, and execution environments. Calling a model an agent just because it can print a JSON object blurs the key distinction: the JSON still has to be interpreted and acted on.

An agent harness is the surrounding software that makes that cycle possible. It builds the model's context, exposes tools, validates tool arguments, enforces permissions, executes actions, returns observations, handles errors, and checks whether the result satisfies the task. The harness may also decide when to retry, ask for input, summarize old context, or stop. The model is one component inside it.

These definitions are useful because they assign responsibility. If a model emits a malformed tool call, the model or its prompting may be at fault. If the harness silently executes a destructive call without permission, that is a harness failure. If the tool succeeds but the agent reports failure because it misread the result, the observation and decision loop needs attention. If the agent claims success without checking the changed state, the verification step is missing.

One request, several layers

Consider the request: “Update the project README so the installation instructions match the current command.” A model can propose new Markdown and name a file-editing tool. That is only a proposal. To complete the request, the system must locate the repository, read the relevant files, identify the current installation command, edit the correct README, and verify that the change appears in the file. If the repository has tests or documentation checks, it may run them. The final response should describe the actual file change and any unverified assumptions.

The path can be represented as:

request → context → model decision → tool call → execution → observation → recovery → verification → response

Each arrow is a place where a system can fail. A stale README may be selected. The model may infer the wrong package manager. A tool call may target the wrong path. The edit may fail. A later step may overwrite the change. The final message may sound confident even though the file never changed. Looking only at the final message hides all of these failures.

This is why “the model knew what to do” and “the agent completed the task” are different measurements. Model evaluation can test whether the proposed edit is correct under ideal conditions. Agent evaluation must include the environment, tool interface, and observable outcome.

Tool calling is an interface, not execution

Many models are trained to emit structured tool calls. A model might output a tool name and arguments in a schema the harness recognizes. That ability is valuable: it can reduce ambiguity and make automatic execution possible. It does not mean the called tool exists, is authorized, or succeeded.

The harness needs to check whether the requested tool is available in the current session. It must parse and validate arguments. It may need to reject a path outside an allowed workspace or prevent a call that exceeds a user's authorization. It then invokes the tool and returns the tool's actual result to the model. If the tool errors, the agent needs a recovery path. If it returns success, the agent may still need to verify the intended effect independently.

For example, a file-writing tool might report that it wrote a file. A simple verification could read that file back and compare the relevant section. A browser action might return no error while leaving a form unsent; verification would inspect the page state. A database action may report that a query ran but affect zero rows. The meaning of “success” depends on the user's requested outcome, not just the transport-level status of one call.

My public Qwen3 tool-calling LoRA is an example at the model-interface layer. Its card reports hold-out and simplified BFCL checks for tool-call output, and explicitly notes a remaining miss: an unbound delete_database call. Those numbers describe a bounded tool-call test, not end-to-end task completion. The model card also says the BFCL numbers use a simplified AST check rather than the official scorer. That distinction matters when interpreting the result.

Harness design changes capability

Suppose two systems use the same language model. One gives it a single large prompt and asks for a final answer. The other gives it a file search tool, a file reader, a patch tool, execution feedback, and a verification step. On a repository task, the second system can observe current code and test a change. The first must guess from whatever context was placed in its prompt. The model weights are identical; the systems have different capabilities.

The difference can also go the other way. A weak harness can make a strong model appear unreliable. A tool description may be vague. A schema may have ambiguous field names. Search results may omit necessary context. Errors may be swallowed rather than shown. The agent may have no way to inspect whether a command changed the intended file. Improving the model alone will not repair those interface problems.

Good harness design therefore includes mundane engineering work: clear tool contracts, bounded permissions, stable state, useful observations, explicit error messages, and a way to verify outcomes. These details are less visible than a model benchmark score, but they directly affect whether the user gets the requested result.

Memory and context are system properties

Language models have finite context windows. An agent working on a long task must decide what to keep, retrieve, or summarize. That is a harness choice. A conversation summary can preserve goals and decisions, but it can also drop a constraint such as “do not publish this project yet.” A repository search can retrieve fresh facts, but it may miss the exact file where a setting is defined. Durable notes can help, provided they are updated and treated as evidence rather than unquestioned truth.

The most important context is often operational: which account is active, which branch is checked out, what the user authorized, which files were already changed, and what remains to be verified. Losing that context can cause duplicate work, contradictory edits, or an external publication at the wrong time. A capable agent needs a reliable way to carry it forward.

Permissions belong in the architecture

An agent should not treat every available tool as permission to use it for every purpose. The harness can expose a tool while still requiring authorization for a particular action. Read-only inspection, local editing, public publication, and financial transactions have different consequences. The system should make those boundaries clear to the model and enforce them at execution time.

Permission checks also need to account for the target. “Edit my README” may authorize a change to one repository, not a different organization or every project on the account. “Help me publish a website” may authorize a release but still require checking the destination and content. A model's plausible interpretation should not be the sole control for consequential actions.

Evaluation needs an outcome definition

Before measuring an agent, define what counts as completion. For a code change, completion might require the requested behavior, passing relevant tests, and a reviewable diff. For a browser task, it might require a visible confirmation on the destination site. For research, it might require citations that directly support the claims. The definition should be specific enough that a reviewer can inspect the evidence.

Useful agent evaluation separates at least four questions:

1. Decision: Did the model select a sensible next action?
2. Interface: Did it construct a valid tool call with appropriate arguments?
3. Execution: Did the tool run and change the intended state?
4. Outcome: Did the full task meet the user's criteria?

A system may pass one and fail the next. A correct tool choice with incorrect arguments is still a failed interface step. A successful tool execution on the wrong file is still a failed outcome. A final answer that says “done” without evidence should not be scored as success merely because it sounds coherent.

Negative results are valuable here. If a small model frequently produces valid calls but cannot recover from errors, that suggests a different limitation from a model that cannot select a tool at all. If a larger model performs well in a clean test but fails in a real application, the tool descriptions, context pipeline, or verification step may deserve investigation before changing models.

A practical way to inspect an agent

When an agent fails, reconstruct the path of the task rather than reading only the final answer. Start with the user's requested outcome. Inspect the context the model received. Look at the chosen tool and its arguments. Read the execution result. Check what the model observed afterward. Finally, compare the actual end state with the requested state.

This sequence helps identify the first point of divergence. If the context omitted a key requirement, later steps may be internally consistent but wrong. If the call was malformed, the model or schema may need work. If the execution result was ambiguous, improve the tool response. If every step worked but the final outcome was still wrong, strengthen the verification criterion.

The model, agent, and harness distinction is therefore practical rather than semantic. It tells us what to measure and where to intervene. Models produce decisions and content. Agents pursue outcomes through repeated decisions and actions. Harnesses make those actions possible and set the conditions under which they are safe and observable. Reliable AI work depends on the complete system.

About Hatsunama. Hatsunama is an independent artificial intelligence researcher and AI systems developer working across AI models, AI agents, agent harnesses, local AI, tool use, computer-use systems, AI evaluation, and applied AI engineering. Research · GitHub · Hugging Face.