Multi-Agent Debate

Optional judge

Implement the Multi-Agent Debate pattern in this codebase, using the LLM client and language we already use.

## Goal

Answer a hard question more reliably by having several debaters answer independently, read and critique each other's answers over a few rounds, and then settle on one answer by majority or an optional judge.

## How it works

1. Send the question to every debater (default 3) in parallel and collect their independent first answers, each ending in a one-line answer.
2. For each round, give every debater its own previous answer and all the other debaters' answers with the Debater revision prompt, and call all of them in parallel.
3. After the first answers and after each round, pull the one-line answer out of each reply; if every debater gives the same answer, end the rounds early.
4. After the last round, take the majority answer. If there is no majority, or you always want a ruling, send the final answers to the judge with the Judge prompt and parse its JSON into answer and why. If that is not valid JSON, retry once, then return the majority answer if there is one, otherwise all final answers with a not-converged status.
5. Return the answer together with every debater's final answer, so a reader can see where they disagreed.

## Prompts

### Debater revision prompt

```markdown
You are debater {{name}}. Question: {{question}}

<your_previous_answer>{{own}}</your_previous_answer>
<other_answers>{{others}}</other_answers>

Reread the other answers critically. If they expose a mistake in yours,
fix it. If you still think you are right, defend your answer and say why
the others are wrong. Do not change your answer just to agree.
End with your current answer in one line.
```

### Judge prompt

```markdown
You are an impartial judge. {{n}} debaters answered the question below
over {{rounds}} rounds.

<question>{{question}}</question>
<final_answers>{{answers}}</final_answers>

Pick the best-supported answer, or merge them if they are compatible.
Respond in JSON: {"answer": "...", "why": "two sentences on the deciding evidence"}
```

## Stop rule

Stop after 2 rounds, or earlier when every debater gives the same one-line answer. With no majority and no judge, return all the final answers with a status saying the debate did not converge.

## Requirements

- Make the number of debaters (default 3) and the round cap (default 2) parameters.
- Vary the debaters by model, temperature or persona so their mistakes are independent.
- Normalize the one-line answers (trim, lowercase, strip punctuation) before comparing them. If a reply has no answer line, retry that debater once, then leave it out of the vote.
- Shuffle the order of the other answers in each prompt, so no debater always reads the same peer first.
- Add a test with stub debaters whose first answers are yes, no, no and whose round-1 answers are all no; assert exactly 6 debater calls, no judge call, and the answer no.

Adversarial + judge

Implement the Multi-Agent Debate pattern (Adversarial + judge variant) in this codebase, using the LLM client and language we already use.

## Goal

Get past a model's first, tempting answer by making two debaters argue fixed opposite sides, while a judge ends the debate as soon as one side is right.

## How it works

1. Start an empty transcript. Call the affirmative debater with the Debater prompt (fixed side), side set to affirmative, and append its answer and argument.
2. Call the negative debater with the same prompt, side set to negative, and the transcript; it must find the flaw and argue for a different answer. Append its rebuttal.
3. At the end of each round, call the judge with the Judge prompt and the transcript, and parse its JSON into decided, answer and reason.
4. If decided is true, return the answer and the reason.
5. Otherwise run the next round: the affirmative replies to the latest rebuttal, then the negative replies.
6. If the round cap is hit, call the judge once more with final set to true, so it must pick the better-supported answer from the transcript.

## Prompts

### Debater prompt (fixed side)

```markdown
You are the {{side}} side in a debate. Question: {{question}}

<transcript>{{transcript}}</transcript>

{{#if affirmative}}
State your answer and the argument for it. If the other side has attacked
it, reply to each point, and concede only if the attack is correct.
{{else}}
You disagree with the affirmative side. Find the flaw in its argument and
argue for a different answer, step by step.
{{/if}}
End with your current answer in one line.
```

### Judge prompt

```markdown
You judge a debate between an affirmative and a negative side.

<question>{{question}}</question>
<transcript>{{transcript}}</transcript>

Decide whether one side has given a correct, well-supported answer.
{{#if final}}This is the last round: you must pick the better-supported answer.{{/if}}
Respond in JSON:
{"decided": true|false, "answer": "only when decided", "reason": "one or two sentences"}
```

## Stop rule

Stop when the judge decides, or after 3 rounds, when the judge must pick an answer. If its final reply is still unusable, return both sides' last answers with a status saying the debate was not settled.

## Requirements

- Make the round cap a parameter (default 3).
- Keep the sides fixed for the whole debate, and tell the negative to disagree even when the affirmative sounds plausible.
- If the judge's reply is not valid JSON, retry once, then treat the round as undecided.
- Let the judge use a different model from the debaters, at a low temperature.
- Add a test with stub debaters and a stub judge that is undecided after round 1 and decides after round 2; assert 4 debater calls, 2 judge calls, and that the judge's answer is returned.

Consensus, no judge

Implement the Multi-Agent Debate pattern (Consensus, no judge variant) in this codebase, using the LLM client and language we already use.

## Goal

Answer a hard question by letting several debaters revise their answers after reading each other's, with no judge: the rounds end when they agree, and otherwise the majority wins.

## How it works

1. Send the question to every debater (default 3) in parallel and collect their first answers, each ending in a one-line answer.
2. Pull out each one-line answer and compare them after normalizing (trim, lowercase, strip punctuation). If they all agree, return that answer.
3. Otherwise give each debater the other debaters' full answers with the Debater update prompt, and call all of them in parallel for the next round.
4. Repeat the agreement check after every round.
5. After the last round, return the majority answer with every debater's final answer attached.

## Prompts

### Debater update prompt

```markdown
Question: {{question}}

These are the answers to the question from the other agents:
{{#each others}}
<agent_answer>{{this}}</agent_answer>
{{/each}}

Using the other agents' reasoning as additional advice, give an updated
answer. Check each step yourself; do not just adopt the majority.
End with your answer in one line.
```

## Stop rule

Stop as soon as every debater agrees, or after 2 rounds. If there is no majority after the last round, return all the final answers with a status saying there was no consensus.

## Requirements

- Make the number of debaters (default 3) and the round cap (default 2) parameters; an odd number of debaters avoids most ties.
- Keep the round check in plain code: no model call decides the outcome.
- If a debater's reply has no answer line, retry it once, then leave it out of that round's check.
- Log every debater's answer per round, so you can see who changed their mind and when.
- Add a test with stub debaters whose first answers are yes, no, no and whose round-1 answers are all no; assert 6 debater calls and the answer no.