Plan-and-Execute

With replanner

Implement the Plan-and-Execute pattern in this codebase, using the LLM client and language we already use.

## Goal

Turn an open-ended goal into a written step list, run the steps one at a time with a tool-using executor, and revise the remaining steps after each result until the goal is answered.

## How it works

1. Call the planner model once with the Planner prompt, passing the goal and the tool names, and parse its numbered list into a plan: an ordered list of step strings.
2. Pop the first step and hand it, with the goal, to the executor: a small tool-using agent loop that sees only that step and returns a short result string.
3. Append the step and its result to past_steps, a list of (step, result) pairs.
4. Call the replanner model with the Replanner prompt, passing the goal, the remaining plan and all of past_steps, and parse its JSON reply.
5. If the reply has final_answer, return it. If it has remaining_steps, replace the plan with them and go back to the executor step.

## Prompts

### Planner prompt

```markdown
For the goal below, write a step-by-step plan. Each step must be a
self-contained task that one worker with {{tool_names}} can complete.
Use as few steps as possible. Do not add a step that only restates
the goal.

<goal>{{goal}}</goal>

Respond as a numbered list. The last step should produce the final answer.
```

### Replanner prompt

```markdown
Goal: {{goal}}

Current plan:
{{plan}}

Steps already completed, with results:
{{past_steps}}

Update the plan. Remove steps that are done or no longer needed, and
add or change steps if the results require it. Do not repeat a
completed step.

If you can already answer the goal, respond with:
{"final_answer": "..."}
Otherwise respond with:
{"remaining_steps": ["...", "..."]}
```

## Stop rule

Stop when the replanner returns final_answer, or after 10 executed steps. On the cap, return past_steps and the unfinished plan with a status saying the goal was not reached.

## Requirements

- Make the step cap (default 10), the plan length cap (default 7) and the executor's own tool-call cap per step (default 5) parameters.
- If the planner's list or the replanner's JSON cannot be parsed, or remaining_steps is empty with no final_answer, retry once with the error appended, then stop with a failed status. Truncate a plan over the cap, and log it.
- When a tool errors or the executor hits its cap, record the error text as that step's result so the replanner can route around it; do not abort the run.
- Let the planner, executor and replanner use separately configured models, so a cheaper model can execute the steps.
- Log the plan before and after every replan, next to the step result that caused the change.
- Add a test with a stub LLM whose planner returns 3 steps and whose replanner returns 1 remaining step, then a final_answer; assert the executor ran 2 steps and the final_answer is returned.

Plan-and-Solve

Implement the Plan-and-Execute pattern (Plan-and-Solve variant) in this codebase, using the LLM client and language we already use.

## Goal

Answer a multi-step reasoning question by having one model call write a plan and carry it out in the same reply, then reading the final answer off that reasoning.

## How it works

1. Call the model with the Plan-and-Solve prompt, passing the question; the reply is reasoning text that states a plan and then works through each step.
2. Call the model again with the Answer extraction prompt, passing the question and that reasoning text, to get the bare final answer.
3. Validate the answer against the expected format (a number, a date, one option letter) and return it together with the reasoning.

## Prompts

### Plan-and-Solve prompt

```markdown
Q: {{question}}

A: Let's first understand the problem, extract the relevant variables
and their values, and devise a plan. Then, let's carry out the plan,
calculate the intermediate values (paying attention to calculation
and common sense), solve the problem step by step, and show the answer.
```

### Answer extraction prompt

```markdown
Q: {{question}}

{{reasoning}}

Therefore, the answer (a single {{answer_format}}) is:
```

## Stop rule

Stop after the extraction call; there is no loop. If the extracted answer fails format validation twice, return the reasoning text with a status saying no answer could be extracted.

## Requirements

- Use no tools: every fact the plan needs must be in the question, so route questions that need lookups to the tool-using pattern instead.
- Cap the reasoning reply length (default 1,000 tokens). If a reply is truncated, retry once; if it is truncated again, stop with a status saying the reasoning did not finish.
- If the extracted answer does not match the expected format, retry the extraction call once with the format error appended.
- Keep the reasoning text alongside the answer so a wrong answer can be traced to the step that went wrong.
- Add a test with a stub LLM that returns a two-step plan with a worked result, then '960' from extraction; assert exactly two model calls and that the answer is 960.

As-needed (ADaPT)

Implement the Plan-and-Execute pattern (As-needed (ADaPT) variant) in this codebase, using the LLM client and language we already use.

## Goal

Complete a task by trying it directly with a tool-using executor, and decompose only the parts that fail into sub-steps, recursively, up to a depth limit.

## How it works

1. Hand the task to the executor with the Executor prompt; it works with its tools and must end by reporting success with a result, or failure with a reason.
2. If the executor reports success, return its result for this task.
3. If it reports failure and the depth limit is not reached, call the planner with the Decompose-on-failure prompt, passing the task and the failure reason, and parse its JSON into sub_steps plus a logic of and or or.
4. Run each sub-step through this same procedure, one level deeper. With and, stop at the first sub-step that fails; with or, stop at the first that succeeds.
5. Return the result upwards. For and, the parent's result is the list of sub-step results in order (the last is usually the deliverable); for or, it is the result of the sub-step that succeeded.

## Prompts

### Executor prompt

```markdown
Complete the task below using the tools. Work step by step.

<task>{{task}}</task>
<tools>{{tool_descriptions}}</tools>

When you finish, end with exactly one line:
SUCCESS: <the result>
or, if you cannot complete the task as a whole,
FAILURE: <what blocked you>
```

### Decompose-on-failure prompt

```markdown
An executor tried this task and could not complete it.

<task>{{task}}</task>
<failure>{{reason}}</failure>

Split the task into 2 to {{max_sub_steps}} smaller sub-steps that avoid
the blocker. Say whether all of them must succeed (and) or any one
is enough (or).

Respond in JSON:
{"logic": "and" | "or", "sub_steps": ["...", "..."]}
```

## Stop rule

Stop when the top-level task succeeds, or when a task fails at the depth limit (default 3). On failure, return the tree of attempted sub-steps with each one's result or reason.

## Requirements

- Make the depth limit (default 3), the sub-steps per decomposition (default 4) and the executor's tool calls per attempt (default 8) parameters.
- Require the executor to end with an explicit success or failure line; treat a missing or unparseable verdict as failure with the raw reply as the reason.
- If the planner's JSON cannot be parsed, retry once with the parse error appended, then report the task as failed.
- Pass a tool error to the executor as an observation; only the executor's final verdict decides whether a task failed.
- Add a test with a stub LLM whose executor fails the top task, then succeeds on both of 2 and-sub-steps; assert one planner call, three executor calls and a success result.