Agents and tools: when a loop is worth it

Pattern (ReAct and cousins): the model emits a thought and a tool call, your runtime executes the tool, the result comes back as an observation, the model continues until it emits a final answer or you cut it off. That is all. There is no inner homunculus. There is no 'the agent decided to be helpful.' There is a sampled next token that happens to be shaped like a function call because you offered tools and a prompt that invites them.

'Autonomous' just means you were sloppy about the cutoff. Marketing uses the word to mean 'fewer clicks.' Security hears 'unbounded side effects.' Your job is to retire the word in working conversations. Say 'a model with these three tools, max five steps, writes confirm in the UI.' If that sentence is less exciting than the keynote, good. Excitement is not a control.

Each turn is a model call: cost, latency, a chance to go stupid. A six-step loop is six chances to pick the wrong tool, hallucinate an argument, ignore the observation, or start searching forever. Unbounded loops are unbounded invoices and unbounded incident surfaces. The cap is the product. A loop without a cap is a job you would never approve for a cron.

Thought traces are optional to show the user and mandatory to log (redacted). They are how you debug 'it went crazy.' They are also how the model talks itself into a bad action if you feed the whole ramble back in. Keep the runtime's trace. Do not assume the model's 'thought' field is truthful. It is more tokens, not a confession.

Multi-agent, swarms, managers-and-workers: extra loops. You now have several unbounded invoices and a coordination problem. v1 does not need this. If a vendor leads with a swarm diagram for a wiki Q&A, they are selling a keynote. A single model, three tools, a cap, a human gate on writes — that is an agent you can run.

The wiki copilot from days 10–12 should not be an agent yet. Retrieval plus generate plus cite is a pipeline, not a loop. Adding a 'search again if unsure' loop is the first agent-ish temptation. It can wait until recall@5 is honest. Query rewrite is a retrieve trick you can do once, without a free-form loop. Do not gift the model a while(true) because the answer was empty. Empty is allowed.

When someone says 'we need an agent,' translate: which tools, which writes, which cap, which human gate, which eval (did the tool get the right args, did we cap, did we confirm). If they cannot answer, they need a copilot or a form. Your four sentences in the practice exist for this moment.

An agent is a model allowed to call tools in a loop until a final answer or a cap. Autonomous means you were sloppy about the cutoff. Each turn is a model call — cost, latency, a chance to go stupid. Thought traces are logged (redacted) and not trusted as confession. Multi-agent swarms are extra unbounded invoices; v1 does not need them. The wiki copilot is not an agent yet. Retrieval plus generate plus cite is a pipeline. Do not gift the model a while-true because the answer was empty. Empty is allowed. When someone says they need an agent, translate: which tools, which writes, which cap, which human gate, which eval. If they cannot answer, they need a copilot or a form. Your four sentences exist for that moment. Use them without apology.

A tool has a name, a JSON schema for arguments, a description the model reads, and your implementation. Bad descriptions → wrong calls. Wide arguments ('query: string' that can be any SQL) → a security incident. Tight tools ('get_ticket(id)') beat clever ones. The model is not a DBA. Do not hand it run_sql.

The description is a prompt. 'search_kb: find relevant SOP chunks for a product. Args: query (string), product (enum: Atlas, Orion). Returns: up to 5 chunks with source and heading. Does not search HR or finance.' That is a contract. 'search: look stuff up' is how you get the model querying salary. Spend time on descriptions. They are cheaper than incidents.

Read tools vs write tools. Reads: search, get_status, lookup_account. Writes: create_ticket, send_email, refund, change_group. Writes need authz of the user (not a god token), idempotency keys, audit, and usually a confirmation step in v1. The confirm UI is the human gate. The model drafts. The human sends. If you skip confirm because 'the demo was smoother,' you shipped an intern with prod credentials.

Authz is the user's identity, not the agent's. The tool executes as the user, through the same SSO groups as a normal UI. A support copilot that looks up accounts the user cannot see in the CRM is an ACL leak with a nicer name. God tokens behind tools are the day-12 leak, moved to the tool layer. Ban them in the spec.

Failure behavior is part of the contract. lookup_account on an unknown id returns a structured not_found, not a 500 the model then invents around. Timeouts return timeout. The model may retry a read; the runtime, not the model, retries writes, and only with the same idempotency key. If the model is allowed to invent a second key, you get two tickets. The runtime holds the key.

Do not expose tools that are 'the entire API surface.' Three tools for v1 is a feature. Twenty tools is a model that will pick the wrong one and a security review that will never end. Add tools when an eval shows a missing verb, not when a roadmap slide has empty boxes.

Side effects belong in the schema comments and in the audit log: who (user), what (tool + args), when, request id, whether confirmed, whether retried. You already wanted this for integrations. The new part is that a probabilistic caller is choosing the args. That makes the audit more important, not less.

Tools are APIs with a name, a tight JSON schema, a description that is a prompt, user-scoped authz, structured failures, and an audit line. Wide arguments are incidents. God tokens are the day-12 leak moved to the tool layer. Reads versus writes: writes need a human gate, an idempotency key minted by the runtime not the model, and no send tool in the list if you only meant draft. Three tools for v1 is a feature. Twenty is a model that will pick the wrong one and a security review that will never end. The description is cheaper than the incident. Spend time on it. 'search: look stuff up' is how you get salary queries. 'search_kb: SOP chunks for a product enum, k=5, no HR' is a contract. Write contracts.

Max steps (e.g. 6). Max wall-clock (e.g. 20s for interactive). Max tokens / spend per session (e.g. $0.20). A watchdog that stops 'I will try another search' forever. On cap: return what you have, show the trace, ask the human. This is identical in spirit to a batch job with a timeout — you already run those. The new part is putting the numbers on a steering slide so 'autonomous' has a ceiling.

Steps vs time vs spend catch different failure modes. A loop can take 3 steps and 40 seconds if a tool hangs — time cap. It can take 20 steps in 8 seconds if the model is spinning on search — step cap. It can take 4 steps and $3 if it stuffed the history — spend cap. You want all three. One cap is a hole the other two exist to cover.

Watchdog is the runtime, not the prompt. 'Please stop after five tools' in the system prompt is a wish. The runtime counts and cuts. The prompt can mention the cap so the model tries to wrap up. The cut happens whether it wraps up or not. Wishes are not controls. You have been told this every day this week. It is still the failure.

On cap, the UX is a feature: here is what I found, here is the trace, hand to human. Not a blank bubble. Not a silent retry from step 0 (that retries the bill). Partial useful output is allowed. Inventing the rest to look finished is not. The generator-as-prisoner rule still applies to the final message.

Observability: log the trace (tool, args, latency, success, who confirmed) so you can debug with a timeline, not a shrug. request id joins the UI to the trace to the tool audit. If you cannot replay the loop, you cannot tell a sponsor why it refunded twice. Replay is a requirement on write-capable agents. It is a strong want on read-only loops.

Kill switch from day 9 still applies: error rate, spend, or a vendor incident trips it. Degrade to the non-agent path (search, form, human queue). An agent that cannot be turned off without a deploy will be turned off by finance after the bill, which is worse.

Caps belong in the same spec block as the tools. They are not an appendix. A tool table without caps is a menu without a budget. Sponsors will order everything.

Caps are the product: max steps, max wall-clock, max spend, a watchdog in the runtime not in the prompt, a kill switch, and a UX on cut that returns what you have plus a hand-to-human. One cap is a hole the other two exist to cover. Prompt wishes do not cut. Observability is the trace you can replay, especially on write-capable loops. Put caps in the same spec block as the tools. A tool table without caps is a menu without a budget. Sponsors will order everything. You already run batch jobs with timeouts. This is that instinct with a probabilistic contractor in the loop. Write the numbers on the steering slide so 'autonomous' has a ceiling someone other than you can see.

Multi-hop questions ('compare the Q2 decision to the current SOP and open a follow-up if they conflict') may need a search, a second search, then a draft. That is a small agent. A refund flow with four systems may need tools — or it may need a BPM engine with a model only on the email text. Prefer the boring orchestrator when the path is known.

If you can flowchart it without a model, maybe do not put a model in the loop. Use the model where the input is messy language (the email, the chat, the SOP exception) and use code where the path is known (if status == paid, then refund via this API). Mixing them by letting the model choose every hop is how you get a different flowchart every Tuesday.

RAG without tools is the default for 'what do our documents say.' A single read tool (ticket lookup, account lookup) is worth it when the answer lives in a system of record, not in a PDF. That is not a loop yet. It is RAG plus one fetch. Do the fetch, stuff the packet, generate, cite. Only start looping when you must retrieve, then fetch, then retrieve again, or when a write is in scope.

A form is still allowed. 'Create a ticket' with fields the user fills is more reliable than a write-tool that might invent the priority. Use the model to draft the body from the conversation, then dump the draft into the form. That is a copilot, not an agent, and it will pass security faster. Do not be ashamed of forms. Forms are how money moves today.

When the loop is not worth it: v1 wiki copilot; any corpus without evals; any write without a confirm UI; any path you can BPM; any sponsor request that cannot name the tools; any team that cannot staff the trace review. Write that list into the design pack as non-goals. Day 14 will need it.

Eval for agents is extra. Retrieval eval still exists if you search. Plus: tool-arg correctness (did it pass the right id), gate compliance (did a write ever fire without confirm), cap compliance (did we cut), and task success on a frozen set of messy tickets. If you only eval the final prose, you will ship a polite loop that files the wrong account.

The decision in one paragraph, which you should be able to say out loud: 'If the job is find-and-cite in a corpus, it is RAG. If the job is fill a known path, it is a form or an orchestrator, maybe with a model on the messy text. If the job is a short unknown hop-count with named tight tools and a cap and a human gate on writes, it can be an agent. If you cannot name the tools and the cap, it is not an agent yet.'

The loop earns its keep on short unknown hops with named tight tools. If you can flowchart it without a model, do not put a model in the loop. Prefer BPM when the path is known. Prefer RAG when the job is find-and-cite. Prefer a form when money moves. A single read tool stuffed into a packet is not yet a loop — do that before you start ReAct. Eval for agents is extra: tool-arg correctness, gate compliance, cap compliance, task success. Final prose is not enough. The decision in one paragraph is the one you should be able to say out loud in a steering meeting without notes. If you cannot, you are not ready to pick the rightmost column, and defaulting left is the grown-up move.

Use case: a support copilot for an account manager. You will not give it a god token. Tool 1 read: lookup_account(id). Tool 2 read: search_kb(query). Tool 3 write: draft_ticket(...) that does not send — it opens a confirm UI. That set is enough to be useful and small enough to review. Resist the fourth tool until these three have traces you can stand.

lookup_account(id): schema is an account id string with a format (not 'any string that might be a name'). Authz: the calling user's CRM visibility. Failure: not_found, timeout, forbidden (user cannot see it — do not leak existence beyond what CRM would). Returns a small object: name, tier, region, open_tickets count — not the whole account dump. Wide returns become wide context and accidental PII in logs.

search_kb(query): this is yesterday's retrieve, exposed as a tool so the model can hop. Constrain with product enum if you have one. k=5. Empty is allowed. Do not let this tool search tickets or CRM. One verb per tool. Hybrid search underneath, ACL of the user, same as the copilot. The model does not get a second, wider search 'just in case.'

draft_ticket(account_id, title, body): runtime checks the account is one the user can see, generates idempotency key, stores a draft, returns draft_id and a confirm URL. The model never calls send. A separate confirm action is a button the user hits, not a tool the model has. If you expose send_ticket as a tool, the model will eventually call it. Do not put it in the list.

Caps for this use: max 5 model steps, 20s wall-clock, $0.20/session. On cap: show the trace and a 'hand to human' button, including any draft that exists so the user is not starting from zero. Kill switch named. Logs: tool names, args redacted for PII, latency, confirm events. Retention as Legal said on day 9.

Failure of the write: if confirm is abandoned, the draft expires (put a TTL). If confirm is hit twice, idempotency on the actual create. If the CRM is down at confirm, the UI says so; the model is out of the picture. Once the human is in the gate, stop looping.

Four sentences for the sponsor: This is not autonomous support. It can look up an account the user already can see, search the KB, and draft a ticket. A human sends the ticket. Loops stop at five steps or twenty seconds. If you want it to send mail or refund, that is a different design with a different review, not a flag.

The three-tool spec is lookup_account, search_kb, draft_ticket-that-does-not-send. Tight schemas, user CRM/KB visibility, structured not_found, runtime idempotency, confirm UI, TTL on abandoned drafts. Caps: five steps, 20 seconds, 20 cents, trace on cut, kill switch. Four sentences for the sponsor: not autonomous; can look up what the user can already see; a human sends; loops stop. If you want mail or refund, that is a different design with a different review, not a flag. Resist the fourth tool until these three have traces you can stand. Small and reviewable beats a menu that security will never sign.

Day 14 is a wiki copilot, which you will argue should not be an agent yet. That argument is this day applied. v1: RAG, citations, refuse, ACL, evals, no writes, no loop. Later: maybe lookup_ticket as a read. Much later: draft a RAID row with a confirm. Anyone who wants an 'Atlas agent' on the first ship is asking for the keynote. You have the sentences to shrink it.

Bring forward: the day 10 retrieval table, the day 11 one-pager and generator contract, the day 12 RAID and no-go, today's tool table as a non-goal (or as a v1.5 appendix). The design pack is an assembly of objects you already wrote. If one of them is thin, thicken it tonight, not during the 70-minute write block.

The talk track distinction: 'we could loop' vs 'we will loop.' Interviews like people who can draw ReAct and then refuse to use it. That is judgment. Building a swarm on a wiki is a tell that you learned the words and not the controls.

If your primary role is FDE, you may want a 20-line pseudo-loop in the optional script: for step in range(MAX): think/act/observe, break on final or cap. Keep it rude. No swarms. If your primary role is delivery lead, the spec is enough. Do not fake a runtime.

Security will ask: user identity on tools, no god token, confirm on writes, audit, caps, kill switch. You can now answer without waiting for an engineer to translate. That is the point of the week.

Sponsors will ask: when do we get the agent. Answer with the shrink path: copilot, one read, one gated write. Put dates only on the copilot until evals exist. Dates on v2 without numbers are how you inherit a demo.

Stop treating 'agent' as a grade of AI. It is an architecture with a blast radius. You pick it like you pick 'we'll call the billing API from a cron' — with a runbook, or not at all.

Tomorrow's lab is a wiki copilot you will argue should not be an agent. Bring the shrink path: copilot, one read later, one gated write much later. Dates only on the copilot until evals exist. Interviews like people who can draw ReAct and then refuse to use it. That is judgment. Building a swarm on a wiki is a tell that you learned the words and not the controls. Security will ask identity, god token, confirm, audit, caps, kill switch. You can answer now. Sponsors will ask when they get the agent. Answer with the path, not with a date on v2 without numbers. Stop treating agent as a grade of AI. It is an architecture with a blast radius. You pick it like you pick 'we'll call the billing API from a cron' — with a runbook, or not at all.