Reflexion

Unit tests

Implement the Reflexion pattern in this codebase, using the LLM client and language we already use.

## Goal

Solve a task that has an automatic pass or fail check by retrying it in fresh trials, each carrying the plain-language lessons the model wrote after earlier failures.

## How it works

1. Run trial 1: call the actor model with the Actor prompt, filling in the task and an empty lessons list.
2. Run the attempt against the unit tests in a sandbox and capture pass or fail, plus the failing test output.
3. If every test passes, return the attempt.
4. Otherwise call the reflector with the Reflection prompt, passing the task, the attempt and the test output, and append its lesson to the lessons list.
5. Start the next trial with a fresh actor call that includes the lessons list, but not the earlier attempts or their transcripts.

## Prompts

### Actor prompt

```markdown
Solve the task below. Your code will be checked by unit tests.

<task>{{task}}</task>

{{#if lessons}}
You have attempted this before. Lessons you wrote for yourself:
<lessons>{{lessons}}</lessons>
Do not repeat a mistake listed here.
{{/if}}

Return only the code.
```

### Reflection prompt

```markdown
Your attempt failed. Work out why, and write a lesson for your next try.

<task>{{task}}</task>
<attempt>{{attempt}}</attempt>
<test_output>{{trace}}</test_output>

In two or three sentences, state:
1. the specific mistake (not "the code was wrong")
2. what to do differently next time

Write in the first person. Do not write code.
```

## Stop rule

Stop when an attempt passes every test, or after 4 trials. At the cap, return the attempt that passed the most tests, with every lesson and a status saying it did not pass.

## Requirements

- Make the trial cap (default 4) and the number of lessons kept in the prompt (default the 3 most recent) parameters.
- Run the tests with a timeout (default 30 seconds), and pass a timeout or crash to the reflector as the failure trace.
- If the reflector's reply is empty or contains code, retry it once; if it fails again, use the first 500 characters of the test output as the lesson.
- Store the lessons with the task id, so a rerun of the same task can start from them.
- Add a test with a stub LLM whose first attempt fails a fake test and whose second passes; assert the reflector ran once and the second actor prompt contains its lesson verbatim.

LLM judge

Implement the Reflexion pattern (LLM judge variant) in this codebase, using the LLM client and language we already use.

## Goal

Retry a task that has no automatic tests in fresh trials, using an LLM judge with a rubric to decide success and carrying forward the lessons the model writes after each failed trial.

## How it works

1. Call the actor with the Actor prompt, filling in the task and the lessons list (empty on trial 1).
2. Send the task, its rubric and the attempt to a separate judge call with the Judge prompt; parse its JSON reply into passed and reason.
3. If passed is true, return the attempt.
4. Otherwise call the reflector with the Reflection prompt, passing the task, the attempt and the judge's reason, and append its lesson to the lessons list.
5. Start the next trial with a fresh actor call that includes the updated lessons list.

## Prompts

### Actor prompt

```markdown
Complete the task below. A reviewer will check your work against a rubric.

<task>{{task}}</task>

{{#if lessons}}
You have attempted this before. Lessons you wrote for yourself:
<lessons>{{lessons}}</lessons>
Do not repeat a mistake listed here.
{{/if}}

Return only the result.
```

### Judge prompt

```markdown
You check one attempt against a rubric. Be strict and literal.

<task>{{task}}</task>
<rubric>{{rubric}}</rubric>
<attempt>{{attempt}}</attempt>

Check every rubric item. Respond in JSON:
{"passed": true|false, "reason": "each failed item and why; empty if passed"}
```

### Reflection prompt

```markdown
Your attempt failed review. Work out why, and write a lesson for your next try.

<task>{{task}}</task>
<attempt>{{attempt}}</attempt>
<review>{{reason}}</review>

In two or three sentences, state:
1. the specific mistake
2. what to do differently next time

Write in the first person. Do not rewrite the attempt.
```

## Stop rule

Stop when the judge passes an attempt, or after 4 trials. At the cap, return the last attempt with the judge's last reason and every lesson, flagged as not passed.

## Requirements

- Make the trial cap a parameter (default 4), and run the judge at temperature 0.
- If the judge's reply is not valid JSON, retry it once; if it fails again, count the trial as failed with the raw reply as the reason.
- Write the rubric as checkable items, such as a word limit or a required fact; a vague rubric lets weak attempts pass.
- Log each trial's attempt, verdict, reason and lesson together.
- Add a test with a stub LLM whose judge fails trial 1 with the reason 'no warranty' and passes trial 2; assert two actor calls, one reflector call, and that the second actor prompt contains the lesson.

Insight bank (ExpeL)

Implement the Reflexion pattern (Insight bank (ExpeL) variant) in this codebase, using the LLM client and language we already use.

## Goal

Make an agent improve across tasks by distilling insights from its past successes and failures into a shared bank that is recalled at the start of every new task.

## How it works

1. Run each task with the actor as in plain Reflexion, and save every finished trial (task id, steps, outcome) to a trajectory store, failures and successes alike.
2. After every 20 new trajectories, send the extractor pairs of a failed and a successful trial on the same task, together with the current insight list.
3. Parse the extractor's reply as a list of operations (ADD, EDIT, UPVOTE, DOWNVOTE) and apply them to the insight bank; remove any insight whose score falls to zero.
4. When a new task starts, put the 5 highest-scored insights and the 2 most similar successful trajectories into the actor's first prompt (ExpeL itself includes the whole insight list; the cap keeps prompts short).
5. Run the task and save its trajectories for the next round of extraction.

## Prompts

### Actor prompt

```markdown
Solve the task below. Your code will be checked by unit tests.

<task>{{task}}</task>

{{#if lessons}}
You have attempted this before. Lessons you wrote for yourself:
<lessons>{{lessons}}</lessons>
Do not repeat a mistake listed here.
{{/if}}

Return only the code.
```

### Reflection prompt

```markdown
Your attempt failed. Work out why, and write a lesson for your next try.

<task>{{task}}</task>
<attempt>{{attempt}}</attempt>
<test_output>{{trace}}</test_output>

In two or three sentences, state:
1. the specific mistake (not "the code was wrong")
2. what to do differently next time

Write in the first person. Do not write code.
```

## Stop rule

Each task stops as in Reflexion: on a pass or after 4 trials. Extraction runs in the background and never blocks a task.

## Requirements

- Make the extraction batch (default 20), the insights recalled (default 5) and the examples recalled (default 2) parameters.
- If a line of the extractor's reply does not parse as an operation, skip it and log it; apply the rest.
- Store each insight with its score, creation date and the trajectory ids it came from, so a bad insight can be traced and removed.
- Cap the bank (default 30 insights), dropping the lowest-scored one when it is full.
- Add a test with the extraction batch set to 1 and a stub extractor that returns 'ADD: check 29 February'; after one saved trajectory, assert the next new task's first actor prompt contains 'check 29 February'.