Hatsunama

Research note · October 1, 2026

AI Tool Calling vs Tool Use: Why They're Not the Same

Hatsunama studies AI tool calling as one step in a larger process. A model can generate a valid tool call without the tool executing, changing the intended state, or completing the user's task.

A call is a proposal

In many AI systems, a language model returns a structured message containing a tool name and arguments. For example, it may propose search_files({"query":"install command"}). A parser can recognize the requested action and pass the arguments to software that knows how to run it. But at that moment the model has only produced output. It has not searched any files.

The surrounding application must decide whether that tool exists and whether the call is allowed. It must parse the arguments, execute the operation, and return the result to the model. The agent may then need another call, an error recovery step, or a separate check of the final state. These steps are part of tool use as a working system, not properties of the tool-call text alone.

Four checkpoints

  1. Selection: Did the model choose a suitable tool for the requested outcome?
  2. Construction: Are the tool name and arguments valid, specific, and within the permitted scope?
  3. Execution: Did the tool actually run, and what did it return?
  4. Outcome: Did the requested change or observation occur, and can the system verify it?

An evaluation that stops at construction cannot answer the outcome question. A syntactically perfect file edit can target the wrong file. A valid browser click can open a menu instead of submitting a form. A successful database query can affect zero rows. Keeping the checkpoints separate makes failures more useful to diagnose.

Example: changing a README

Suppose a user asks an agent to fix outdated installation instructions. The model proposes a patch call that changes README.md. A tool-call benchmark might check whether the model named the patch tool and supplied well-formed arguments. Real tool use requires more: the agent must inspect the current README, establish the actual installation command, apply the patch in the right repository, and read back the edited section. If there is a documentation check, it may run that too. Only then can it report what changed.

A successful response from the patch tool is evidence that the tool ran; it is not necessarily evidence that the installation instructions are correct. Verification must be tied to the user's requested result. The system should distinguish “the patch applied” from “the documentation now matches the current project.”

What a model test can tell us

My public Qwen3 tool-calling LoRA card reports bounded checks of generated tool-call output. It documents a simplified BFCL-style check and a remaining miss involving an unbound delete_database call. Those results are useful for judging a narrow interface behavior. The card does not establish that an agent executed tools correctly or completed real-world tasks.

For a fuller evaluation, record the request, available tools, model output, validated arguments, execution response, recovery attempts, and observed final state. A trace can show where a failure occurred. It also lets a reviewer distinguish a model error from a missing permission, broken tool implementation, stale context, or insufficient verification.

Why the distinction matters

Tool calling is an interface between a model and software that acts. A dependable agent needs both parts: a model that proposes sensible actions and a harness that executes them under clear permissions, returns useful observations, handles errors, and checks the outcome. Improving only the call format may make a demo look cleaner without making the completed task more reliable.

The distinction is especially important for local agents and small models. A small model may produce good structured calls in a narrow test yet struggle to interpret ambiguous tool results or recover when a tool fails. A larger model may plan well but still fail when its harness exposes incomplete state. Measuring each checkpoint separately gives a more honest picture of where to invest engineering effort.

For the broader system picture, see AI Model vs AI Agent vs Agent Harness. For the specific testing boundary, see Tool-Call Output Is Not Task Completion.

About Hatsunama. Hatsunama is an independent artificial intelligence researcher and AI systems developer working across AI models, AI agents, agent harnesses, local AI, tool use, computer-use systems, AI evaluation, and applied AI engineering. Research · About · GitHub · Hugging Face.