This page is for readers who know some programming.

Under the hood

How an agent actually works, byte by byte

On the first page an agent was a loop with model.infer(context, tools) where the if/else used to be. This page opens up that one line. You'll see the exact JSON that goes over the wire, what comes back, how a “tool call” really happens, and what the model itself reads. At the end is a complete agent you can run: 112 lines of Python.

One request, one reply: the round trip in one minute.
Read the transcript

Let’s open up that one line: model.infer. It’s a single web request. Your program sends JSON: the rules, the tool descriptions, and every message so far. The API remembers nothing, so all of it goes, every time. Back comes JSON, with a stop reason. “Tool use” means: the model wants lookup_order, for order 4471. Here’s the call, with an ID. Your code runs it, and sends the result back, tagged with that same ID. Next, the model tries to refund $76. Your check refuses: that’s more than the charge. The error goes back as a result, and the model corrects itself: $38. When the stop reason says “end turn”, the loop ends. Five calls. One list. One loop.

  1. 1 Your harness

    Ordinary code you write. It keeps the list of messages, calls the API, checks and runs tools, and decides when to stop. The code →

  2. 2 The API

    One HTTPS endpoint: POST /v1/messages. JSON in, JSON out. It is stateless: it remembers nothing between calls. The request →

  3. 3 The model

    Reads one long sequence of tokens and writes more tokens, one at a time. It knows nothing about JSON, HTTP, or your functions. The tokens →

Running example · the same ticket as the first page

“Hi, I got charged twice for order #4471. Can you sort it out?”

What's real here. Every request body on this page is exactly what the Anthropic Python SDK sent when we ran the agent below. The responses were scripted in the documented format, so IDs, token counts, wording and the thinking signature are illustrative (the bodies won't replay against the real API). They include one deliberate model mistake so you can see the error path. The first page simplified this run to four calls; the full version here also leaves an internal note. The token view uses a real open-weights model's published template and tokenizer.

What you send: one HTTP request

One inference is one HTTPS POST. Here is the first call of our run, complete. The SDK builds exactly this; you could send it with curl.

POST https://api.anthropic.com/v1/messages x-api-key: $ANTHROPIC_API_KEY # who is paying anthropic-version: 2023-06-01 # API contract version content-type: application/json
{ "model": "claude-opus-5", 1 "max_tokens": 16000, 2 "system": "You are the support agent for Tidy Home, an online store.\nResolve the customer's ticket using the tools. Look up an order before acting on it.\nRefunds: only for duplicate or failed charges, and never more than the charge being refunded.\nWhen the issue is resolved, send the customer one short, friendly reply, add an internal note, and stop.\nIf you cannot resolve it, escalate to a human.", 3 "tools": [ 4 { "name": "lookup_order", "description": "Get an order's total and every charge made for it. Call this before acting on any order.", "input_schema": { "type": "object", "properties": {"order_id": {"type": "string", "description": "Digits only, e.g. 4471"}}, "required": ["order_id"] } }, { "name": "issue_refund", "description": "Refund one charge, fully or partly. Amount must not exceed that charge.", "input_schema": { "type": "object", "properties": {"charge_id": {"type": "string"}, "amount": {"type": "number"}}, "required": ["charge_id", "amount"] } }, { "name": "send_reply", "description": "Email the customer. Use once, when the ticket is resolved.", "input_schema": { "type": "object", "properties": {"text": {"type": "string"}}, "required": ["text"] } }, { "name": "add_internal_note", "description": "Leave a note on the ticket for the support team. The customer never sees it.", "input_schema": { "type": "object", "properties": {"text": {"type": "string"}}, "required": ["text"] } }, { "name": "escalate_to_human", "description": "Hand the ticket to a person. Use when the tools cannot resolve it.", "input_schema": { "type": "object", "properties": {"reason": {"type": "string"}}, "required": ["reason"] } } ], "messages": [ 5 { "role": "user", "content": "Hi, I got charged twice for order #4471. Can you sort it out?" } ] }
  1. model: which model runs the inference.
  2. max_tokens: a hard cap on the length of this one response, not the whole run. If it's hit, generation stops mid-sentence and stop_reason is "max_tokens".
  3. system: the system prompt. Explained below.
  4. tools: the menu of functions the model may ask for. Explained below.
  5. messages: the conversation so far. On call 1 that's just the ticket. Explained below.

Key order doesn't matter in JSON. We reordered keys for reading (the SDK sends them alphabetically) and pretty-printed the body. Nothing else changed.

The system prompt

system is a top-level field: a string, or a list of text blocks if you want to mark cache breakpoints. It is not the first entry of messages; it sits beside it. It holds your instructions as the operator: who the model is, what it's for, what it may do, and when it's finished. Ours has one line per job:

The system prompt, line by line, with the job each line does
LineJob
You are the support agent for Tidy Home, an online store.Role and setting
Resolve the customer's ticket using the tools. Look up an order before acting on it.Goal and method
Refunds: only for duplicate or failed charges, and never more than the charge being refunded.Policy (and your code enforces it too, as call 2 shows)
When the issue is resolved, send the customer one short, friendly reply, add an internal note, and stop.Done criteria: when to end the turn
If you cannot resolve it, escalate to a human.Fallback
  • A prompt is a request, not a guarantee. Hard limits, like never refunding more than is left on a charge, are also checked in code.
  • Keep out: the customer's words (those go in messages), secrets (the model can repeat anything it reads), and per-tool usage details (those go in each tool's description).

Tool definitions

Each tool is a name, a description, and an input_schema written in JSON Schema. The model never sees your Python function, only this JSON. So the description is prompt text: say when to call the tool, not just what it does (“Call this before acting on any order”).

  • Tools cost tokens. Definitions are fed to the model as input, and sending tools also adds a hidden tool-use system prompt: 286 tokens on Claude Opus 5, per Anthropic's pricing docs. You'll see what that looks like.
  • Want guaranteed-valid arguments? Add "strict": true to a tool (its schema also needs "additionalProperties": false) and the model's input is guaranteed to match the schema.

The user input and the messages list

messages is an array of {"role", "content"} objects. The rules are few and strict:

  • Conversation turns are "user" and "assistant", alternating, starting with user. (Consecutive same-role turns get merged into one. Newer models such as Claude Opus 5 also accept an occasional mid-conversation "system" entry, but the top-level field is the normal place.)
  • content is a string, or a list of typed blocks. "content": "Hi…" is shorthand for [{"type": "text", "text": "Hi…"}].
  • You send text, image, document and tool_result blocks. The model writes text, thinking and tool_use blocks.
  • Tool results travel in a user message. From the API's point of view, your harness is the “user” reporting back.
  • The API is stateless. Every call re-sends the entire list. The agent's “memory” is this array in your program, nothing more.

What comes back: content blocks and a stop reason

The response to call 1. The model has decided it needs data, so instead of answering it asks for a tool. It doesn't run anything; it returns a description of the call it wants.

{ "id": "msg_01XFDUDYJgAACzvnptvVoYEL", 1 "type": "message", "role": "assistant", "model": "claude-opus-5", "content": [ {"type": "thinking", "thinking": "", "signature": "EuYBCkQIBxgCIkBd3…"}, 2 {"type": "text", "text": "I'll pull up order #4471 first."}, 3 { "type": "tool_use", 4 "id": "toolu_01A09q90qw90lq917835lq9", "name": "lookup_order", "input": {"order_id": "4471"} } ], "stop_reason": "tool_use", 5 "stop_sequence": null, "usage": { 6 "input_tokens": 1165, "output_tokens": 96, "cache_creation_input_tokens": 0, "cache_read_input_tokens": 0 } }
  1. id, type, role: this is an assistant message. Append its content to messages unchanged.
  2. thinking: Claude Opus 5 decides for itself whether to think before answering. The reasoning text is omitted by default (empty string), but the signed block still comes back. Pass it back untouched.
  3. text: ordinary prose the model wrote. It can appear before, between, or instead of tool calls.
  4. tool_use: the tool call. It's just data: a unique id, the tool name, and input, already parsed JSON that normally follows your schema (guaranteed only with strict). Your code decides whether to run it.
  5. stop_reason: why the model stopped writing. This is the one field your loop branches on (table below).
  6. usage: tokens read and written, which is what you pay for. input_tokens counts everything re-sent (minus any prompt-cache hits, reported separately), so it grows every call.

Every stop_reason, and what your loop does

Stop reasons and how the harness should handle each
stop_reasonMeaningYour loop
tool_useThe model wants one or more tools run.Run every tool_use block, send all results back in one user message, call again.
end_turnIt finished naturally.break: use the text.
max_tokensIt hit your max_tokens cap.Output is cut off, maybe mid tool call. Don't run it; raise the cap or continue.
stop_sequenceIt wrote one of your stop_sequences.Read stop_sequence to see which, and handle it.
pause_turnA server-side tool loop (e.g. web search) paused.Send the assistant content back to continue. Never happens with only your own tools.
refusalClaude declined to respond.Run nothing. Read stop_details; stop, or retry on a fallback model.
model_context_window_exceededThe response filled the context window.Treat as truncated; shrink the history.

One run, five calls, every byte

The complete run. For each call you see what the harness appended to messages before sending, and the full response. Watch each tool_use.id come back as a tool_result.tool_use_id, and watch input_tokens climb as the history is re-sent.

All 5 calls shown · messages sent: 9 · input_tokens: 1,823

messages in your program

  1. user Hi, I got charged twice for order #4471…
  2. assistant thinking · text · tool_use lookup_order
  3. user tool_result → order JSON
  4. assistant tool_use issue_refund(76.00)
  5. user tool_result is_error: true
  6. assistant text · tool_use issue_refund(38.00)
  7. user tool_result → refund succeeded
  8. assistant tool_use send_reply + tool_use add_internal_note
  9. user 2 × tool_result in one message
  10. program stop_reason == "end_turn" → return

Call 1 · the ticket goes in

Appended to messages

[ { "role": "user", "content": "Hi, I got charged twice for order #4471. Can you sort it out?" } ]

Response

{ "id": "msg_01XFDUDYJgAACzvnptvVoYEL", "type": "message", "role": "assistant", "model": "claude-opus-5", "content": [ {"type": "thinking", "thinking": "", "signature": "EuYBCkQIBxgCIkBd3…"}, {"type": "text", "text": "I'll pull up order #4471 first."}, { "type": "tool_use", "id": "toolu_01A09q90qw90lq917835lq9", "name": "lookup_order", "input": {"order_id": "4471"} } ], "stop_reason": "tool_use", "stop_sequence": null, "usage": { "input_tokens": 1165, "output_tokens": 96, "cache_creation_input_tokens": 0, "cache_read_input_tokens": 0 } }

The model can't reach your order database. It asks. Your harness runs lookup_order("4471") and gets back the order: the customer's name, a $38 total, and two captured $38 charges, ch_8812 and ch_8813.

Call 2 · the result goes back, and the model makes a mistake

Appended to messages

[ { "role": "assistant", "content": [ {"type": "thinking", "thinking": "", "signature": "EuYBCkQIBxgCIkBd3…"}, {"type": "text", "text": "I'll pull up order #4471 first."}, { "type": "tool_use", "id": "toolu_01A09q90qw90lq917835lq9", "name": "lookup_order", "input": {"order_id": "4471"} } ] }, { "role": "user", "content": [ { "type": "tool_result", "tool_use_id": "toolu_01A09q90qw90lq917835lq9", "content": "{\"order_id\": \"4471\", \"customer\": \"Priya Shah\", \"total\": 38.0, \"charges\": [{\"id\": \"ch_8812\", \"amount\": 38.0, \"status\": \"captured\"}, {\"id\": \"ch_8813\", \"amount\": 38.0, \"status\": \"captured\"}]}" } ] } ]

Response

{ "id": "msg_01Mx7sLkQ2c9QbJ6Vr1uTzHn", "type": "message", "role": "assistant", "model": "claude-opus-5", "content": [ { "type": "tool_use", "id": "toolu_01BqKz3UvF8nW2xYc5dRtE7m", "name": "issue_refund", "input": {"charge_id": "ch_8813", "amount": 76.0} } ], "stop_reason": "tool_use", "stop_sequence": null, "usage": { "input_tokens": 1341, "output_tokens": 71, "cache_creation_input_tokens": 0, "cache_read_input_tokens": 0 } }

Two messages were appended: the model's own previous turn, verbatim, then your tool_result, whose tool_use_id matches the request. Now the model asks to refund $76, both charges added up. The system prompt said not to, but a prompt isn't a guarantee. Your run_checked compares the amount with the charge and refuses. Nothing is refunded.

Call 3 · the error goes back as data

Appended to messages

[ { "role": "assistant", "content": [ { "type": "tool_use", "id": "toolu_01BqKz3UvF8nW2xYc5dRtE7m", "name": "issue_refund", "input": {"charge_id": "ch_8813", "amount": 76.0} } ] }, { "role": "user", "content": [ { "type": "tool_result", "tool_use_id": "toolu_01BqKz3UvF8nW2xYc5dRtE7m", "is_error": true, "content": "Refund of 76.00 rejected: charge ch_8813 has 38.00 left to refund. Nothing was refunded." } ] } ]

Response

{ "id": "msg_01Hc4PnVw8XyRt2LmQs6JdKa", "type": "message", "role": "assistant", "model": "claude-opus-5", "content": [ { "type": "text", "text": "Only the duplicate charge should be refunded, so the amount is 38.00." }, { "type": "tool_use", "id": "toolu_01CwR5tYh9LmN3bVx7ZqPs2D", "name": "issue_refund", "input": {"charge_id": "ch_8813", "amount": 38.0} } ], "stop_reason": "tool_use", "stop_sequence": null, "usage": { "input_tokens": 1468, "output_tokens": 88, "cache_creation_input_tokens": 0, "cache_read_input_tokens": 0 } }

"is_error": true tells the model the call failed, and the content says why. No exception reaches the top of your program; the model reads the error and corrects itself. The refund of $38 passes the check and runs.

Call 4 · two tool calls at once

Appended to messages

[ { "role": "assistant", "content": [ { "type": "text", "text": "Only the duplicate charge should be refunded, so the amount is 38.00." }, { "type": "tool_use", "id": "toolu_01CwR5tYh9LmN3bVx7ZqPs2D", "name": "issue_refund", "input": {"charge_id": "ch_8813", "amount": 38.0} } ] }, { "role": "user", "content": [ { "type": "tool_result", "tool_use_id": "toolu_01CwR5tYh9LmN3bVx7ZqPs2D", "content": "{\"refund_id\": \"re_5520\", \"charge_id\": \"ch_8813\", \"amount\": 38.0, \"status\": \"succeeded\"}" } ] } ]

Response

{ "id": "msg_01Tq9WeR3yUi5OpAs7DfGh1J", "type": "message", "role": "assistant", "model": "claude-opus-5", "content": [ { "type": "tool_use", "id": "toolu_01DkL8mN2bV4cX6zQw9ErT5y", "name": "send_reply", "input": { "text": "Hi Priya, sorry about that! You were charged twice for order #4471, so I've refunded the duplicate $38.00 charge." } }, { "type": "tool_use", "id": "toolu_01EpO3iU7yT1rE5wQ9aSdF2g", "name": "add_internal_note", "input": {"text": "Duplicate charge ch_8813 refunded in full (re_5520)."} } ], "stop_reason": "tool_use", "stop_sequence": null, "usage": { "input_tokens": 1610, "output_tokens": 142, "cache_creation_input_tokens": 0, "cache_read_input_tokens": 0 } }

Replying and leaving a note don't depend on each other, so the model asks for both in one response: parallel tool calls. Your harness runs both (in any order, or concurrently).

Call 5 · both results in one message, then done

Appended to messages

[ { "role": "assistant", "content": [ { "type": "tool_use", "id": "toolu_01DkL8mN2bV4cX6zQw9ErT5y", "name": "send_reply", "input": { "text": "Hi Priya, sorry about that! You were charged twice for order #4471, so I've refunded the duplicate $38.00 charge." } }, { "type": "tool_use", "id": "toolu_01EpO3iU7yT1rE5wQ9aSdF2g", "name": "add_internal_note", "input": {"text": "Duplicate charge ch_8813 refunded in full (re_5520)."} } ] }, { "role": "user", "content": [ { "type": "tool_result", "tool_use_id": "toolu_01DkL8mN2bV4cX6zQw9ErT5y", "content": "{\"status\": \"sent\"}" }, { "type": "tool_result", "tool_use_id": "toolu_01EpO3iU7yT1rE5wQ9aSdF2g", "content": "{\"status\": \"saved\"}" } ] } ]

Response

{ "id": "msg_01Zx6CvB8nM1qW3eR5tY7uIo", "type": "message", "role": "assistant", "model": "claude-opus-5", "content": [ { "type": "text", "text": "Refunded the duplicate $38.00 charge (ch_8813), emailed Priya, and noted the ticket." } ], "stop_reason": "end_turn", "stop_sequence": null, "usage": { "input_tokens": 1823, "output_tokens": 29, "cache_creation_input_tokens": 0, "cache_read_input_tokens": 0 } }

Both results go back in a single user message, one tool_result per tool_use. (Anthropic warns that splitting them across messages makes the model less likely to call tools in parallel.) This time the response has no tool_use and stop_reason is "end_turn", so the loop breaks.

The tally

  • 5 inference calls. Each one was a full HTTP request carrying the whole history.
  • 5 tool calls requested, 4 run. One was blocked by your code, and the block became information the model used.
  • input_tokens went 1,165 → 1,823. You pay for the history every call, which is why long agent runs get expensive and why prompt caching exists.
  • The model never touched your systems. It wrote JSON; your harness decided what ran.

What the model actually reads: tokens

What the model actually reads, in one minute.
Read the transcript

Here’s the twist… the model never sees your JSON. Before inference, the server flattens everything (system prompt, tools, messages) into one long piece of text, using a chat template. Little markers say who’s talking: system, user, assistant. Then that text is chopped into tokens: little numbered pieces of words. That’s all the model ever reads… a long row of numbers. And all it does is predict the next token. Then the next. Then the next. When the tokens it writes spell out a tool call, a parser turns them back into JSON, and hands it to your program. So a “tool call” is just text the model wrote. Your code decides what it means. (The markers and token ids shown are Qwen3-8B’s, for example; Claude’s own format isn’t published.)

The model never sees your request as a JSON object. Before inference, the request is flattened: the server turns system, tools and messages into one long string using a chat template, splits it into tokens (integers), and the model predicts the next token, over and over.

Why a different model here? Claude's internal format isn't published. So this is the conversation as of call 3, rendered by the published template of an open-weights model, Qwen3-8B, and split by its real tokenizer. The markers differ between models; the mechanism is the same.

1 · The JSON you send (call 3)

{ "model": "claude-opus-5", "max_tokens": 16000, "tools": [… the same 5 tool definitions as call 1 …], "system": "You are the support agent for Tidy Home, an online store.\nResolve the customer's ticket using the tools. Look up an order before acting on it.\nRefunds: only for duplicate or failed charges, and never more than the charge being refunded.\nWhen the issue is resolved, send the customer one short, friendly reply, add an internal note, and stop.\nIf you cannot resolve it, escalate to a human.", "messages": [ { "role": "user", "content": "Hi, I got charged twice for order #4471. Can you sort it out?" }, { "role": "assistant", "content": [ {"type": "thinking", "thinking": "", "signature": "EuYBCkQIBxgCIkBd3…"}, {"type": "text", "text": "I'll pull up order #4471 first."}, { "type": "tool_use", "id": "toolu_01A09q90qw90lq917835lq9", "name": "lookup_order", "input": {"order_id": "4471"} } ] }, { "role": "user", "content": [ { "type": "tool_result", "tool_use_id": "toolu_01A09q90qw90lq917835lq9", "content": "{\"order_id\": \"4471\", \"customer\": \"Priya Shah\", \"total\": 38.0, \"charges\": [{\"id\": \"ch_8812\", \"amount\": 38.0, \"status\": \"captured\"}, {\"id\": \"ch_8813\", \"amount\": 38.0, \"status\": \"captured\"}]}" } ] }, { "role": "assistant", "content": [ { "type": "tool_use", "id": "toolu_01BqKz3UvF8nW2xYc5dRtE7m", "name": "issue_refund", "input": {"charge_id": "ch_8813", "amount": 76.0} } ] }, { "role": "user", "content": [ { "type": "tool_result", "tool_use_id": "toolu_01BqKz3UvF8nW2xYc5dRtE7m", "is_error": true, "content": "Refund of 76.00 rejected: charge ch_8813 has 38.00 left to refund. Nothing was refunded." } ] } ] }

2 · The same conversation after the chat template. Special tokens highlighted.

<|im_start|>system
You are the support agent for Tidy Home, an online store.
Resolve the customer's ticket using the tools. Look up an order before acting on it.
Refunds: only for duplicate or failed charges, and never more than the charge being refunded.
When the issue is resolved, send the customer one short, friendly reply, add an internal note, and stop.
If you cannot resolve it, escalate to a human.

# Tools

You may call one or more functions to assist with the user query.

You are provided with function signatures within <tools></tools> XML tags:
<tools>
{"type": "function", "function": {"name": "lookup_order", "description": "Get an order's total and every charge made for it. Call this before acting on any order.", "parameters": {"type": "object", "properties": {"order_id": {"type": "string", "description": "Digits only, e.g. 4471"}}, "required": ["order_id"]}}}
{"type": "function", "function": {"name": "issue_refund", "description": "Refund one charge, fully or partly. Amount must not exceed that charge.", "parameters": {"type": "object", "properties": {"charge_id": {"type": "string"}, "amount": {"type": "number"}}, "required": ["charge_id", "amount"]}}}
{"type": "function", "function": {"name": "send_reply", "description": "Email the customer. Use once, when the ticket is resolved.", "parameters": {"type": "object", "properties": {"text": {"type": "string"}}, "required": ["text"]}}}
{"type": "function", "function": {"name": "add_internal_note", "description": "Leave a note on the ticket for the support team. The customer never sees it.", "parameters": {"type": "object", "properties": {"text": {"type": "string"}}, "required": ["text"]}}}
{"type": "function", "function": {"name": "escalate_to_human", "description": "Hand the ticket to a person. Use when the tools cannot resolve it.", "parameters": {"type": "object", "properties": {"reason": {"type": "string"}}, "required": ["reason"]}}}
</tools>

For each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:
<tool_call>
{"name": <function-name>, "arguments": <args-json-object>}
</tool_call><|im_end|>
<|im_start|>user
Hi, I got charged twice for order #4471. Can you sort it out?<|im_end|>
<|im_start|>assistant
I'll pull up order #4471 first.
<tool_call>
{"name": "lookup_order", "arguments": {"order_id": "4471"}}
</tool_call><|im_end|>
<|im_start|>user
<tool_response>
{"order_id": "4471", "customer": "Priya Shah", "total": 38.0, "charges": [{"id": "ch_8812", "amount": 38.0, "status": "captured"}, {"id": "ch_8813", "amount": 38.0, "status": "captured"}]}
</tool_response><|im_end|>
<|im_start|>assistant
<tool_call>
{"name": "issue_refund", "arguments": {"charge_id": "ch_8813", "amount": 76.0}}
</tool_call><|im_end|>
<|im_start|>user
<tool_response>
Refund of 76.00 rejected: charge ch_8813 has 38.00 left to refund. Nothing was refunded.
</tool_response><|im_end|>
<|im_start|>assistant
<think>

</think>

3 · 780 tokens. The model receives only the integers: 151644, 8948, 198, 2610, 525, 279, 1824, 8315, 369, 350, 42586, 5379, … On a desktop, hover a token to see its id.

<|im_start|>system↵
You are the support agent for Tidy Home, an online store.↵
Resolve the customer's ticket using the tools. Look up an order before acting on it.↵
Refunds: only for duplicate or failed charges, and never more than the charge being refunded.↵
When the issue is resolved, send the customer one short, friendly reply, add an internal note, and stop.↵
If you cannot resolve it, escalate to a human.↵
↵
# Tools↵↵
You may call one or more functions to assist with the user query.↵
↵
You are provided with function signatures within <tools></tools> XML tags:↵
<tools>↵
{"type": "function", "function": {"name": "lookup_order", "description": "Get an order's total and every charge made for it. Call this before acting on any order.", "parameters": {"type": "object", "properties": {"order_id": {"type": "string", "description": "Digits only, e.g. 4471"}}, "required": ["order_id"]}}}↵
{"type": "function", "function": {"name": "issue_refund", "description": "Refund one charge, fully or partly. Amount must not exceed that charge.", "parameters": {"type": "object", "properties": {"charge_id": {"type": "string"}, "amount": {"type": "number"}}, "required": ["charge_id", "amount"]}}}↵
{"type": "function", "function": {"name": "send_reply", "description": "Email the customer. Use once, when the ticket is resolved.", "parameters": {"type": "object", "properties": {"text": {"type": "string"}}, "required": ["text"]}}}↵
{"type": "function", "function": {"name": "add_internal_note", "description": "Leave a note on the ticket for the support team. The customer never sees it.", "parameters": {"type": "object", "properties": {"text": {"type": "string"}}, "required": ["text"]}}}↵
{"type": "function", "function": {"name": "escalate_to_human", "description": "Hand the ticket to a person. Use when the tools cannot resolve it.", "parameters": {"type": "object", "properties": {"reason": {"type": "string"}}, "required": ["reason"]}}}↵
</tools>↵
↵
For each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:↵
<tool_call>↵
{"name": <function-name>, "arguments": <args-json-object>}↵
</tool_call><|im_end|>↵
<|im_start|>user↵
Hi, I got charged twice for order #4471. Can you sort it out?<|im_end|>↵
<|im_start|>assistant↵
I'll pull up order #4471 first.↵
<tool_call>↵
{"name": "lookup_order", "arguments": {"order_id": "4471"}}↵
</tool_call><|im_end|>↵
<|im_start|>user↵
<tool_response>↵
{"order_id": "4471", "customer": "Priya Shah", "total": 38.0, "charges": [{"id": "ch_8812", "amount": 38.0, "status": "captured"}, {"id": "ch_8813", "amount": 38.0, "status": "captured"}]}↵
</tool_response><|im_end|>↵
<|im_start|>assistant↵
<tool_call>↵
{"name": "issue_refund", "arguments": {"charge_id": "ch_8813", "amount": 76.0}}↵
</tool_call><|im_end|>↵
<|im_start|>user↵
<tool_response>↵
Refund of 76.00 rejected: charge ch_8813 has 38.00 left to refund. Nothing was refunded.↵
</tool_response><|im_end|>↵
<|im_start|>assistant↵
<think>↵↵
</think>↵↵

What to notice

  1. Roles are just tokens. Each turn is wrapped in <|im_start|>role … <|im_end|>. The model was trained on huge numbers of conversations in exactly this shape, which is why “system” text carries weight and why APIs keep turns in a strict user/assistant rhythm.
  2. Tools are just text. Your tool list is pasted into the system turn as JSON lines, with instructions on how to call them. That's the equivalent of Claude's hidden 286-token tool-use prompt. Note that <tools> here isn't even a special token, just ordinary characters.
  3. A tool call is text the model wrote. Earlier assistant turns contain <tool_call>{"name": …, "arguments": …}</tool_call>. The API's tool_use blocks are this, parsed.
  4. Results come back as a user turn wrapped in <tool_response>. There's no is_error field at this level: the error is just words the model reads.
  5. The prompt ends with an open slot: <|im_start|>assistant. (The empty <think></think> is there because thinking was switched off for this render.) Whatever the model writes next is its turn.

Generation: one token at a time

Here is call 3's (scripted) answer, split into this model's real tokens. Qwen didn't generate it; the demo shows how any model emits text. Each step, the model scores every token in its vocabulary (about 150,000), one is picked, it's appended, and the model runs again on the longer sequence.

  1. Only7308
  2. the279
  3. duplicate22513
  4. charge6757
  5. should1265
  6. be387
  7. refunded93452
  8. ,11
  9. so773
  10. the279
  11. amount3311
  12. is374
  13. ␣220
  14. 318
  15. 823
  16. .13
  17. 015
  18. 015
  19. .↵624
  20. <tool_call>151657
  21. ↵198
  22. {"4913
  23. name606
  24. ":788
  25. "330
  26. issue11159
  27. _ref7793
  28. und1241
  29. ",497
  30. "330
  31. arguments16370
  32. ":788
  33. {"5212
  34. charge13891
  35. _id842
  36. ":788
  37. "330
  38. ch331
  39. _62
  40. 823
  41. 823
  42. 116
  43. 318
  44. ",497
  45. "330
  46. amount6045
  47. ":788
  48. ␣220
  49. 318
  50. 823
  51. .13
  52. 015
  53. }}↵11248
  54. </tool_call>151658
  55. <|im_end|>151645

Then the server parses the text back into JSON

# text before <tool_call> → a text block {"type": "text", "text": "Only the duplicate charge should be refunded, so the amount is 38.00."} # JSON inside <tool_call>…</tool_call> → a tool_use block with a fresh id {"type": "tool_use", "id": "toolu_01CwR5tYh9LmN3bVx7ZqPs2D", "name": "issue_refund", "input": {"charge_id": "ch_8813", "amount": 38.0}} # <|im_end|> after a tool call → stop_reason "tool_use" (else "end_turn")
  • <tool_call> is one token (id 151657), added to the vocabulary for this job, so the start of a call is unambiguous to both model and parser.
  • Numbers are written digit by digit. “38.00” is five tokens. The model doesn't calculate a refund like a calculator; it writes plausible digits. That's one more reason for run_checked.
  • <|im_end|> (id 151645) is the stop token. When it's picked, generation ends. If max_tokens runs out first, you get half a turn, maybe half a tool call. That's why you check stop_reason before running anything.
  • Malformed JSON is just bad text. The parser fails on it. Strict tool use ("strict": true) prevents this by constraining sampling so only tokens that keep the output schema-valid can be picked.

The same ideas in Claude's API

How the open-model token format maps to Claude API fields
In the token stream (Qwen3)In Claude's API
<|im_start|>user … <|im_end|>{"role": "user", "content": …}
Tool JSON pasted into the system turntools, plus a hidden tool-use system prompt (286 tokens on Opus 5)
<think> … </think>thinking block
<tool_call> … </tool_call>tool_use block with an id
<tool_response> … </tool_response>tool_result block with tool_use_id and optional is_error
End-of-turn tokenstop_reason: "tool_use" or "end_turn"

The whole agent, in one file

Everything above in one runnable file. Run pip install anthropic, set ANTHROPIC_API_KEY, then python agent.py. The orders are fake, so it's safe to run; swap the function bodies for calls to your real systems.

import json import anthropic client = anthropic.Anthropic() # reads ANTHROPIC_API_KEY from the environment MODEL = "claude-opus-5" MAX_STEPS = 10 SYSTEM_PROMPT = """You are the support agent for Tidy Home, an online store. Resolve the customer's ticket using the tools. Look up an order before acting on it. Refunds: only for duplicate or failed charges, and never more than the charge being refunded. When the issue is resolved, send the customer one short, friendly reply, add an internal note, and stop. If you cannot resolve it, escalate to a human.""" TOOLS = [ {"name": "lookup_order", "description": "Get an order's total and every charge made for it. Call this before acting on any order.", "input_schema": {"type": "object", "properties": {"order_id": {"type": "string", "description": "Digits only, e.g. 4471"}}, "required": ["order_id"]}}, {"name": "issue_refund", "description": "Refund one charge, fully or partly. Amount must not exceed that charge.", "input_schema": {"type": "object", "properties": {"charge_id": {"type": "string"}, "amount": {"type": "number"}}, "required": ["charge_id", "amount"]}}, {"name": "send_reply", "description": "Email the customer. Use once, when the ticket is resolved.", "input_schema": {"type": "object", "properties": {"text": {"type": "string"}}, "required": ["text"]}}, {"name": "add_internal_note", "description": "Leave a note on the ticket for the support team. The customer never sees it.", "input_schema": {"type": "object", "properties": {"text": {"type": "string"}}, "required": ["text"]}}, {"name": "escalate_to_human", "description": "Hand the ticket to a person. Use when the tools cannot resolve it.", "input_schema": {"type": "object", "properties": {"reason": {"type": "string"}}, "required": ["reason"]}}, ] # ---- Your functions. Fake data here so the file runs as-is; in real life these call your APIs. ---- ORDERS = {"4471": {"order_id": "4471", "customer": "Priya Shah", "total": 38.00, "charges": [ {"id": "ch_8812", "amount": 38.00, "status": "captured"}, {"id": "ch_8813", "amount": 38.00, "status": "captured"}]}} REFUNDED = {} # charge_id -> amount refunded so far (your payments system knows this) def find_charge(charge_id): charge = next((c for o in ORDERS.values() for c in o["charges"] if c["id"] == charge_id), None) if charge is None: raise ValueError(f"No charge with id {charge_id!r}.") return charge def lookup_order(order_id): if order_id not in ORDERS: raise ValueError(f"No order with id {order_id!r}.") return ORDERS[order_id] def issue_refund(charge_id, amount): REFUNDED[charge_id] = REFUNDED.get(charge_id, 0) + amount return {"refund_id": "re_5520", "charge_id": charge_id, "amount": amount, "status": "succeeded"} def send_reply(text): print("REPLY TO CUSTOMER:", text) return {"status": "sent"} def add_internal_note(text): return {"status": "saved"} def escalate_to_human(reason): return {"status": "escalated", "reason": reason} FUNCTIONS = {f.__name__: f for f in (lookup_order, issue_refund, send_reply, add_internal_note, escalate_to_human)} # ---- The harness: the model suggests, this code decides what actually runs. ---- def run_checked(name, args): if name not in FUNCTIONS: raise ValueError(f"Unknown tool {name!r}.") if name == "issue_refund": charge = find_charge(args["charge_id"]) left = charge["amount"] - REFUNDED.get(charge["id"], 0) if not 0 < args["amount"] <= left: raise ValueError(f"Refund of {args['amount']:.2f} rejected: charge {charge['id']} " f"has {left:.2f} left to refund. Nothing was refunded.") return FUNCTIONS[name](**args) def run_agent(ticket): messages = [{"role": "user", "content": ticket}] for step in range(MAX_STEPS): response = client.messages.create( model=MODEL, max_tokens=16000, system=SYSTEM_PROMPT, tools=TOOLS, messages=messages) messages.append({"role": "assistant", "content": response.content}) if response.stop_reason == "end_turn": # the model says it's done return "".join(b.text for b in response.content if b.type == "text") if response.stop_reason != "tool_use": # max_tokens, refusal, ... raise RuntimeError(f"Stopped early: {response.stop_reason}") results = [] # every tool_use gets exactly one tool_result for block in response.content: if block.type != "tool_use": continue try: output = run_checked(block.name, block.input) results.append({"type": "tool_result", "tool_use_id": block.id, "content": json.dumps(output)}) except Exception as err: # errors go back to the model, not up the stack results.append({"type": "tool_result", "tool_use_id": block.id, "content": str(err), "is_error": True}) messages.append({"role": "user", "content": results}) # all results in ONE message escalate_to_human(f"Agent hit MAX_STEPS on: {ticket}") return None if __name__ == "__main__": print(run_agent("Hi, I got charged twice for order #4471. Can you sort it out?"))

Mapped to the first page's pseudocode

Pseudocode from the first page and the real code that implements it
First pageThis file
context = [SYSTEM_PROMPT, ticket.body]system=SYSTEM_PROMPT and messages = [{"role": "user", "content": ticket}]
move = model.infer(context, tools)response = client.messages.create(…)
if move.done: breakif response.stop_reason == "end_turn": return …
check(move)run_checked(block.name, block.input)
context.append(result)messages.append({"role": "user", "content": results})
for step in range(MAX_STEPS)Same, then escalate_to_human if it runs out

We wrote the loop by hand because showing it is the point. In production you can let the SDK's Tool Runner (client.beta.messages.tool_runner) run this loop over your functions, or use the Claude Agent SDK for a full harness with built-in file, shell and web tools.

What production harnesses add

The loop above is the whole idea. Real harnesses wrap it with engineering, none of which changes the mechanism:

Streaming
Show tokens as they're generated instead of waiting for the full response. Needed for long outputs and large max_tokens.
Retries
Rate limits and server errors happen. The SDK retries 429 and 5xx responses for you (twice by default).
Context management
Re-sending everything gets expensive. Prompt caching reuses the unchanged prefix; compaction summarizes old turns.
Approval gates
Pause before side-effecting tools, like refunds, and ask a person. It's the same spot as run_checked.
Budgets
Step caps, token budgets, timeouts, and a spending limit per run.
Fallbacks and sub-agents
Retry a declined request on another model; let one agent be a tool of another. It's the same loop, nested.

An agent is a list of messages, one POST request, and a loop. The model writes text; a parser turns some of that text into tool calls; your code decides what runs and when to stop.

← Back to the idea Play Loop Lab

Primary sources