Augmented LLM

Tools in context

Implement the Augmented LLM pattern in this codebase, using the LLM client and language we already use.

## Goal

Answer one request with a single model that can call tools (document search, a calculator, a note store) whenever it needs them, and return a plain-language answer once it needs no more.

## How it works

1. Load the user's saved notes from the memory store and fill them into the System prompt with tools; send it with the tool definitions and the user's message.
2. Read the model's reply. For each tool call in it, check the arguments against that tool's schema, run the tool, and append the result to the messages as a tool result.
3. If a tool call has bad arguments, fails or times out, append the error text as its tool result instead of raising, so the model can correct itself.
4. Write any fact passed to save_note to the memory store, keyed by user, so later conversations can recall it.
5. Call the model again with the extended messages, and repeat while its replies contain tool calls.
6. Return the text of the first reply that contains no tool calls as the answer.

## Prompts

### System prompt with tools

```markdown
You are a support assistant for {{company}}.

You can call these tools. Call one only when you need it:
- search_docs(query): search the {{company}} help center
- calculator(expression): exact arithmetic
- save_note(text): store a durable fact about this customer

Rules:
- Never guess a policy. If you are unsure, call search_docs first.
- Quote the document title when you rely on a search result.
- When you have enough information, answer in plain language.

Relevant notes from earlier conversations:
{{memory_notes}}
```

## Stop rule

Stop at the first reply with no tool calls, or after 5 tool rounds. At the cap, make one last call with tools disabled and return that answer, flagged as having hit the tool limit.

## Requirements

- Make the tool-round cap (default 5) and the per-tool timeout (default 10 seconds) parameters.
- Give every tool a one-line description that says when to use it; the model picks tools from these descriptions alone.
- Keep tools that change state (save_note) separate from read-only ones in configuration, so they can be audited or disabled.
- Log every model call and tool call with its arguments, result and token counts.
- Add a test with a stub LLM that first calls calculator("240*29/30") and then replies in plain text; assert the calculator ran exactly once, its result 232 is in the second call's messages, and the plain text is returned.

Deferred tool search

Implement the Augmented LLM pattern (Deferred tool search variant) in this codebase, using the LLM client and language we already use.

## Goal

Let one model use a large tool library without putting every tool definition in its prompt: it searches a catalog, and only the tools it finds are loaded for the rest of the request.

## How it works

1. Store every tool's name, one-line description and full schema in a catalog, indexed for keyword or embedding search.
2. Start each request with one tool defined, tool_search(query), and a system-prompt line telling the model to search for a tool before assuming none exists.
3. When the model calls tool_search, return the names and schemas of the top 3 matches, and add those schemas to the tool list for every later call in this request.
4. Run any loaded tool the model calls and append its result, or its error text, to the messages as a tool result.
5. Call the model again, and return the text of the first reply that contains no tool calls.

## Prompts

### System prompt with tools

```markdown
You are a support assistant for {{company}}.

You can call these tools. Call one only when you need it:
- search_docs(query): search the {{company}} help center
- calculator(expression): exact arithmetic
- save_note(text): store a durable fact about this customer

Rules:
- Never guess a policy. If you are unsure, call search_docs first.
- Quote the document title when you rely on a search result.
- When you have enough information, answer in plain language.

Relevant notes from earlier conversations:
{{memory_notes}}
```

## Stop rule

Stop at the first reply with no tool calls, or after 6 tool rounds, searches included. At the cap, make one last call with tools disabled and return that answer, flagged as having hit the limit.

## Requirements

- Make the schemas returned per search (default 3) and the round cap (default 6) parameters.
- If the model calls a tool that has not been loaded, return an error naming tool_search as the way to load it, and do not run it.
- Allow a short, configurable list of always-loaded tools for the ones most requests use (default: none).
- Log every search query with the tools it returned, so weak catalog descriptions show up.
- Add a test with a stub LLM, and the always-loaded list left empty, that calls tool_search("refund") and then refund_quote; assert the first call's tool list holds only tool_search and the second call's includes refund_quote.

Forced output tool

Implement the Augmented LLM pattern (Forced output tool variant) in this codebase, using the LLM client and language we already use.

## Goal

Turn free text into one record that matches a fixed schema by forcing the model to call a single tool whose parameters are that schema.

## How it works

1. Define one tool, record_request, whose parameter schema is the record you want: field names, types, allowed values and which fields are required.
2. Call the model once with the input text and that single tool, with tool choice forced to record_request, so it cannot reply in prose.
3. Take the arguments of the tool call as the record. Never execute the tool itself.
4. Validate the arguments against the schema. If they pass, return the record.
5. If they fail, return a rejection holding the validation errors and the raw arguments.

## Prompts

### System prompt with tools

```markdown
You are a support assistant for {{company}}.

You can call these tools. Call one only when you need it:
- search_docs(query): search the {{company}} help center
- calculator(expression): exact arithmetic
- save_note(text): store a durable fact about this customer

Rules:
- Never guess a policy. If you are unsure, call search_docs first.
- Quote the document title when you rely on a search result.
- When you have enough information, answer in plain language.

Relevant notes from earlier conversations:
{{memory_notes}}
```

## Stop rule

Stop after one model call: one forced tool call, one validation, then either a record or a rejection.

## Requirements

- Keep the schema in one place and generate both the tool definition and the validator from it.
- Treat a reply with no tool call, or with more than one, as a rejection with the reason 'no single tool call'.
- Describe every field in the schema; the model fills each field from its description.
- Leave repair out of this step: if rejections need fixing, add a separate repair step after it.
- Add a test with a stub LLM whose forced call sets plan_price to the string "$240"; assert the result is a rejection whose errors name plan_price and that the model was called exactly once.