Research note · September 29, 2026

Tool-Call Output Is Not Task Completion

A model that produces a valid tool call has cleared one step in an agent workflow. The call still has to be accepted, executed, observed, and checked against the user's requested outcome.

What a tool-calling test can show

A tool-calling evaluation can ask whether a model selects a suitable function, supplies parseable arguments, and avoids calls when no tool is needed. Those are useful questions. A structured call makes the model's proposed action easier for surrounding software to inspect and execute.

The Hatsunama Qwen3 tool-calling LoRA card reports a 15/16 hold-out result, including 9/9 cases where a call was required and 6/7 where it was not. It also reports BFCL-style numbers and states that these use a simplified AST check rather than the official scorer. The card identifies a remaining miss: an unbound delete_database call can still be emitted.

Those results describe a bounded test of model output. They should not be presented as evidence that a deployed agent reliably completed tasks. The named failure case is especially informative: producing a syntactically recognizable tool call can still be the wrong decision.

What happens after the call

The agent harness receives the proposed call and decides what to do with it. It must know whether the function exists, whether the arguments meet its schema, whether the action is authorized, and whether execution is safe in the current environment. A call can fail at any of those gates without reaching the external tool.

Execution creates another distinction. A tool can return an error. It can run successfully but affect the wrong target. It can report success while the intended state remains unchanged. The agent then needs an observation loop that exposes the result clearly enough to choose a recovery step or stop.

Consider a request to update a project file. The model might emit the correct editing function and well-formed arguments. Task completion still requires the correct repository, branch, and path; the intended edit; and a read-back or other check showing that the file now contains the requested change. A plausible final message is not a substitute for that check.

A fuller evaluation

For a tool-using AI system, it helps to score several stages separately:

1. Decision: Was a tool needed, and was the selected tool appropriate?
2. Call: Were the arguments valid and scoped to the right target?
3. Execution: Did the tool run, and what state did it change?
4. Observation and recovery: Did the agent interpret the result and handle failure?
5. Outcome: Did the real task meet the user's criteria?

This breakdown turns a single “agent succeeded” score into a diagnosis. If calls are well formed but outcomes fail, the next investigation may concern permissions, tool contracts, execution feedback, or verification. If the model calls a tool when it should abstain, a different intervention is needed.

The practical research question is therefore broader than whether a model can print a tool-call object. It is whether the complete system can move from a request to an authorized, observable, verified result.