Group Chat

Problem

Some tasks need several specialists who react to each other. Planning “a 3-day Tokyo trip for under $1,500” takes someone to draft the days, someone who knows the city, and someone who keeps the total honest. Nobody can say in advance whose turn comes next: whether the planner speaks again depends on whether the budget checker just found an overrun. A fixed pipeline guesses the order wrong, and a supervisor that collects results privately means the specialists never hear each other.

How it works

All the agents share one conversation. The task is posted as the first message of the shared thread. Each turn, the chat manager reads the whole thread and picks one speaker. It is usually a cheap model call given the agents’ role descriptions, but it can be a simple rule. The chosen agent is called with the full thread, and its reply is appended for everyone to see.

In the example, the manager first calls on the planner, which drafts an itinerary. It then picks the budget checker, which prices the plan and posts “over by $200”. Because that message is in the shared thread, the manager sends the turn back to the planner, which swaps to a cheaper hotel without anyone relaying the problem. The local expert is there whenever the conversation needs it. After every message a termination check runs. When a participant ends with TERMINATE (here the planner, once its revised plan is under budget), or the turn cap is hit, the chat stops and the latest plan goes to the output. Some implementations also let the selector answer TERMINATE.

The manager never writes content. It only chooses who talks, so all the actual work lives in the agents’ messages.

When to use

• Several roles need to see and respond to each other’s contributions.
• The right speaking order depends on what was just said and cannot be fixed in advance.
• You want a conversation you can read afterwards: the thread is a complete, attributed record of who said what.

When to avoid

• The order is known. A fixed sequence such as an SOP-driven team or prompt chaining is cheaper and easier to test.
• You have many agents or long messages. Every agent rereads the whole thread every turn, so the context grows quickly.
• The work splits into independent pieces. Running them in parallel and combining the results is faster than taking turns.

Trade-offs

Each turn costs two model calls, one to select the speaker and one for the speaker itself, and every call carries the whole growing thread. The cost climbs faster than the number of turns. In exchange you get flexible coordination with very little orchestration code: adding a role means adding a participant and a one-line description for the selector. Behaviour is harder to predict than a fixed pipeline, because the selector can choose a different order on every run.

Failure modes

• Endless chatter: agents keep agreeing or politely re-litigating, and nobody says TERMINATE. Always cap the turns, and return the thread with a clear status when the cap is hit.
• Bad speaker choice: the selector picks the same agent repeatedly, or names one that does not exist. Validate the name, block back-to-back repeats, and fall back to a rotation.
• Impersonation: an agent writes lines for other participants. Tell each one to speak only for itself, and cut its reply at another speaker’s name.
• Context blow-up: the shared thread outgrows the context window. Trim the oldest replies or summarise them, but always keep the original task.

Implement the Group Chat pattern in this codebase, using the LLM client and language we already use.

## Goal

Let several role agents solve a task in one shared conversation, where a selector model chooses who speaks each turn, until the task is done or a turn cap is reached.

## How it works

1. Define each participant with a name, a one-line role description and its own system prompt. Start one shared thread and post the user's task as its first message.
2. Each turn, call the selector model with the Speaker selector prompt, passing the role descriptions and the thread. Parse its reply as one participant name or the word TERMINATE.
3. If the reply is not a known name, retry the selector once; if it is still invalid, fall back to the next participant in a fixed rotation.
4. Call the chosen participant with the Participant prompt and the whole thread, then append its reply to the thread, tagged with its name.
5. After every message, check the stop conditions: the selector said TERMINATE, a participant ended its message with TERMINATE, or the turn cap is hit.
6. Return the last substantive message as the answer, together with the full thread.

## Prompts

### Speaker selector prompt

```markdown
You are coordinating a group chat. Choose who speaks next.

<roles>
{{roles}}
</roles>

<conversation>
{{thread}}
</conversation>

Read the conversation, then pick the one participant from {{participants}}
whose role best fits what the conversation needs next.
If the latest messages fully answer the task, reply TERMINATE instead.
Reply with only the participant's name, or TERMINATE.
```

### Participant prompt

```markdown
You are {{name}}, the {{role}} in a group chat working on the task below.
The other participants are: {{others}}.

<task>{{task}}</task>
<conversation>{{thread}}</conversation>

Write your next message. Speak only for your own role: build on or correct
what others said, and name who you are answering. Keep it under 150 words.
Never write messages for other participants.
If the task is complete and nothing is left for anyone to do, end with TERMINATE.
```

## Stop rule

Stop when the selector or a participant says TERMINATE, or after 12 participant messages. On the cap, return the thread and the last message with a status of max_turns, so the caller knows no one declared the task done.

## Requirements

- Make the turn cap a parameter (default 12); the chat must never run without one.
- Tag every message with its speaker's name, and give every participant the whole thread; when it grows past the context budget, keep the task message and trim from the oldest replies.
- Validate the selector's choice against the participant list, and do not let the same agent speak twice in a row unless that is explicitly allowed.
- Log each turn: the selected speaker, the message, and the token counts for both the selector and the participant call.
- Add a test with a stub LLM whose selector returns planner, budget, planner, then TERMINATE; the participant stubs return fixed text without TERMINATE. Assert that the participants were called in exactly that order and the thread holds 4 messages.

You are coordinating a group chat. Choose who speaks next.

<roles>
{{roles}}
</roles>

<conversation>
{{thread}}
</conversation>

Read the conversation, then pick the one participant from {{participants}}
whose role best fits what the conversation needs next.
If the latest messages fully answer the task, reply TERMINATE instead.
Reply with only the participant's name, or TERMINATE.

You are {{name}}, the {{role}} in a group chat working on the task below.
The other participants are: {{others}}.

<task>{{task}}</task>
<conversation>{{thread}}</conversation>

Write your next message. Speak only for your own role: build on or correct
what others said, and name who you are answering. Keep it under 150 words.
Never write messages for other participants.
If the task is complete and nothing is left for anyone to do, end with TERMINATE.