Tool / function calling explained

B5
Concepts · Core Building Blocks

Tool / function calling explained.

"Tool calling" named one mechanism in 2024 and names at least four in late 2026 — a plain JSON request, a schema-constrained one, a search over tool definitions, and code the model writes to call your tools inside a sandbox — and the vendors document that these are mutually exclusive. So the first decision in a tool-using system is not which tools to expose. It is which calling mechanism, because that choice fixes your schema guarantee, your round-trip count and your token bill before you write a single tool.

Step 1The model proposes; your code disposes.

The fact that survives every change of mechanism: the model does not call your function. It outputs a structured request asking you to call it. There is no code execution inside the model. Even the newest code-generating variants only move where the loop runs — into a sandbox you operate — not who executes the real work.

  • You describe the available tools (name, purpose, parameter schema) in the request; MCP is the open standard that lets any model reach any tool this way.
  • The model, instead of answering in prose, may emit a tool-use message: "call get_weather with {"city": "Paris"}."
  • Your application executes the real function, then sends the result back as a tool-result message. The model continues, now able to use that result.

Everything the model does is still next-token prediction. It was post-trained to emit a specific JSON-shaped structure when a tool would help; the "calling" is your code reacting to that structure. This is why the tool description is prompt engineering rather than boilerplate — the model's choice of whether and how to call a tool rests almost entirely on that text. A vague description ("does stuff with users") produces unreliable calling; a precise one ("Fetch a user by ID. Use only when you have a numeric user_id; do not guess IDs") produces reliable calling. Tool design for agents is the whole subject.

Step 2The wire shape.

# 1. You declare tools in the request
tools = [{
  "name": "get_weather",
  "description": "Current weather for a city. Use when asked about weather.",
  "input_schema": {
    "type": "object",
    "properties": {"city": {"type": "string"}},
    "required": ["city"]
  }
}]
# 2. Model replies with a tool-use request (not prose)
{ "type": "tool_use", "id": "t_01", "name": "get_weather",
  "input": { "city": "Paris" } }

# 3. YOUR code runs get_weather("Paris") and returns a tool_result
{ "type": "tool_result", "tool_use_id": "t_01",
  "content": "12C, light rain" }

# 4. Model now answers in prose, grounded in the result
"It's 12C with light rain in Paris right now."

One call is rarely the end. The result may prompt another, until the model has enough to answer. That cycle — model → tool request → execute → result → model — is the agent loop. Two consequences follow that no mechanism removes. You own the loop: the API does not auto-execute; you write the code that detects a tool-use message, runs the function, appends the result and calls the model again. And you need a stop condition: a tool that always errors can make the model retry until your budget is gone, so cap iterations (planning and termination).

Step 3Four mechanisms, and the exclusivity nobody reads.

As of November 2026 the major APIs offer these as separate, documented features, and the constraints between them are stated plainly in the docs rather than discovered in production — if you read that far.

  • Plain JSON tool use. The default. The model emits a tool-use block; nothing guarantees the arguments match your schema, so you validate. Several calls can arrive in one response (parallel tool calls) — on Anthropic's API you must return every result in a single following message, one tool_result block per call, or the request errors.
  • Strict / schema-constrained tool use. Setting strict: true on a tool definition switches the sampler to grammar-constrained decoding, so the arguments are schema-valid by construction. OpenAI exposes the same flag per function tool. It is a real guarantee, and it costs you flexibility: Anthropic requires additionalProperties: false and a complete required list, and only a subset of JSON Schema survives per vendor (JSON Schema subsets per vendor).
  • Tool search. Instead of putting every definition in the prompt, the model searches a catalogue and gets back references that expand into full definitions on demand. This exists because a large catalogue is a token problem before it is a selection problem — see tool docs and discoverability and MCP context overload.
  • Programmatic tool calling. The model writes code that invokes your tools inside an execution container, so a five-step chain costs one model turn instead of five. Anthropic's docs are explicit that strict: true is not supported with programmatic calling, and neither is disable_parallel_tool_use: true. See code as action.

The pattern across vendors is the same and it is the thing to remember: you can have a schema guarantee or you can have the mechanisms that reduce round trips, not both. OpenAI's Structured Outputs announcement states that structured outputs are not compatible with parallel function calls, and recommends setting parallel_tool_calls: false to get the guarantee — note that this flag defaults to true, and setting it false means exactly zero or one tool call per turn. Anthropic's exclusion list for programmatic calling is the same trade in a different place. Decide which guarantee you are buying, then write tools that assume it.

Step 4What the measurements say, and what they do not.

The temptation is to read "fewer round trips" as "worse accuracy, cheaper bill". An August 2026 arXiv study (The Bitter Lesson of Tool Calling, arXiv:2608.06370) measured a 309-entry subset of BFCL v4 across 14 models with typed Python stubs as the tool interface, and found programmatic calling matched or beat native JSON calling in 11 of 14 models, with the GPT-5.6 family 10.6% above the JSON baseline, and 13 of 14 under parallel fan-out.

Read the caveats, because they are the useful part. The authors note that BFCL v4's echo-return stubs mean the benchmark measures argument serialisation rather than end-to-end tool-use correctness; programmatic calling used 1.5× the input tokens of JSON in the chaining ablation; and the ablation samples were 31–52 entries each, so only the cross-model pattern is reliable. So: code mode is not a free win on either axis, and a benchmark score here is not a claim about your tools.

What you can take from it is the shape of the decision. Round trips dominate latency and transcript growth; the transcript is re-sent every step, so a chain collapsed into one turn is a quadratic cost saving on the steps it removes and an input-token premium on the turn it keeps. Three rules of thumb:

  • Few tools, high stakes, strict schema. A payments or database tool where a malformed argument is an incident: take the grammar guarantee and the extra round trips.
  • Long deterministic chains, bulk data. Fetch-filter-aggregate over hundreds of records: programmatic calling, because the intermediate results never need to enter the transcript at all.
  • Large catalogue, sparse use. Fifty tools of which any run touches three: tool search, and measure the prompt before and after rather than assuming.

Step 5The safety rules are mechanism-independent — and one of them gets harder.

A tool argument produced by the model is untrusted input. The model can be wrong, or steered by prompt injection in retrieved or tool-returned content. Treat every tool call as if a stranger on the internet sent that request to your function — because via injected content, one effectively did.

  • Validate before executing. Re-check arguments against the schema and your business rules in code. Never pass model output straight into a shell, SQL string or filesystem path. Note that strict: true narrows this job without removing it: a schema-valid order_id can still belong to another tenant.
  • Least privilege. Expose the narrowest tool that does the job: refund_order(order_id, amount) with server-side limits, not run_arbitrary_sql.
  • Gate irreversible actions. Deletes, payments, sends — require explicit confirmation. Capability limits hold when the prompt is subverted; a sentence telling the model "be careful" does not (blast radius).
  • Return errors as data. Send a structured error back as the tool result so the model can react ("that user does not exist") rather than crashing the loop.
  • Make tools idempotent. The loop may retry; a retried charge_card must not double-charge.

The rule that gets harder is review. With plain JSON calls your approval gate sees a tool name and its arguments — a reviewable artefact. With programmatic calling it sees a program, and "approve this code" is a strictly weaker control than "approve this payment of $4,000", because the interesting behaviour is a branch three lines down. If you adopt code mode on a surface with human approval, keep the gate on the effects inside the container rather than on the generated source.

Start with plain JSON tool use and a validation layer; it is the only mechanism every model supports and the only one whose failures you can read. Then make exactly one change based on a measurement you took: add strict: true to the tools where a malformed argument is an incident, turn on tool search when tool definitions are over roughly a tenth of your prompt, and reach for programmatic calling only when a single chain is costing you five round trips. Changing the mechanism changes your guarantees — so change one at a time, and keep the eval that told you to.

Related: structured outputs for the same JSON-shaping machinery pointed at the final answer, the tool-calling vendor matrix for feature-by-feature differences, shaping tool results for what to return, and MCP for making the whole thing portable.