Standard (default)

All steps

Parts of this pattern

Question Input / output: A question whose answer breaks into independent points: 'What are the most effective ways to improve sleep quality?'
Skeleton LLM call: One quick call that writes only the outline: a numbered list of short points, a few words each.
Point 1 LLM call: Expands point 1, 'Fixed sleep schedule', into a sentence or two, knowing the whole skeleton.
Point 2 LLM call: Expands point 2, 'Cool, dark bedroom', at the same time as the other points.
Point 3 LLM call: Expands point 3, 'Cut afternoon caffeine', in parallel with the others.
Join Aggregator: Plain code, not a model: places the expansions under their points in outline order.
Answer Input / output: The full answer, finished in roughly the time of the skeleton plus the slowest single point.
Question → Skeleton: The question goes to the skeleton prompt first.What are the most effective ways to improve sleep quality?
Skeleton → Point 1: Point 1 goes to its own call, with the question and the whole skeleton.Skeleton: 1. Fixed sleep schedule 2. Cool, dark bedroom 3. Cut afternoon caffeine. Write point 1 only.
Skeleton → Point 2: Point 2 goes out at the same moment, to a separate call.Skeleton: 1. Fixed sleep schedule 2. Cool, dark bedroom 3. Cut afternoon caffeine. Write point 2 only.
Skeleton → Point 3: Point 3 goes out at the same moment too.Skeleton: 1. Fixed sleep schedule 2. Cool, dark bedroom 3. Cut afternoon caffeine. Write point 3 only.
Point 1 → Join: Point 1's expansion is held until the others arrive.Go to bed and get up at the same times every day, weekends included, so your body clock stays steady.
Point 2 → Join: Point 2's expansion arrives whenever its call finishes.Keep the bedroom around 18 °C and fully dark; blackout curtains or an eye mask help.
Point 3 → Join: Point 3's expansion completes the set.Avoid coffee, tea and energy drinks after about 14:00; caffeine lingers for hours.
Join → Answer: The joined answer is returned: the skeleton's points, each with its expansion, in order.1. Fixed sleep schedule: go to bed and get up at the same times... 2. Cool, dark bedroom: keep it around 18 °C... 3. Cut afternoon caffeine: avoid coffee after 14:00...

With router (SoT-R)

A router first decides whether the question suits an outline; sequential questions such as maths or code get one ordinary call instead.

All steps

List question

Maths question

Parts of this pattern

Question Input / output: Any question: some split into independent points, others need one line of reasoning from start to finish.
Router Router: A cheap call that decides whether the answer can be written as independent points.
Skeleton LLM call: Writes the outline for questions the router sent down the parallel path.
Expand points LLM call: The point expansions, run as parallel calls (drawn as one node here), one per outline point.
Direct answer LLM call: One ordinary call, used for maths, code and other questions whose steps depend on each other.
Answer Input / output: Either the joined points or the direct answer.
Question → Router: Every question goes to the router first.What are the most effective ways to improve sleep quality?
Router → Skeletonyes: Questions whose answer splits into independent points take the skeleton path.router: yes (a list of independent tips)
Router → Direct answerno: Questions that need sequential reasoning go to one ordinary call.'A train leaves at 14:05 at 80 km/h... when does it arrive?' router: no (step-by-step arithmetic)
Skeleton → Expand points: Each outline point goes out to its own expansion call, all at once.1. Fixed sleep schedule 2. Cool, dark bedroom 3. Cut afternoon caffeine
Expand points → Answer: The expansions are joined in outline order by plain code and returned.1. Fixed sleep schedule: go to bed and get up at the same times... 2. Cool, dark bedroom: ...
Direct answer → Answer: The direct answer is returned as is.It arrives at 16:50: 220 km at 80 km/h takes 2 h 45 min.

← → neighboursesc atlas

Efficiency & Cost

Skeleton-of-Thought

Also known as SoT · Outline-then-expand · Parallel point expansion

Write a short outline first, then expand every outline point in parallel and join the results, cutting time to a complete answer.

Introduced by Ning et al. (2023) · Tsinghua University, Microsoft Research and others

Published at ICLR 2024. Framed as a latency technique: decoding is sequential, so generating the points in parallel finishes long answers sooner.

Measured over one run

Standard

Nodes: 7. Boxes in the diagram: models, tools, stores, people and plain code.
LLM calls: 4. Model invocations in the first scenario: each time a model or agent node starts thinking, plus each branch it decides.
Tool calls: 0. Messages sent to a tool node in the first scenario.
Steps: 4. Steps in the first scenario, with repeated steps counted every time they run.
Scenarios: 1. Distinct example runs you can switch between on the diagram.
Loop: no. Whether the flow cycles back on itself until a condition is met.
Parallel: yes. Whether two or more messages are in flight at the same time.
Branch: no. Whether the flow takes one path out of several, chosen at run time.
Human in the loop: no. Whether a person appears as a node, to approve, correct or answer.
Autonomy: low. How much the model decides the control flow itself, from fixed code paths (low) to open-ended (high).
Cost: medium. Relative token spend per request.
Latency: low. Relative wall-clock time per request.

With router (SoT-R)

Nodes: 6. Boxes in the diagram: models, tools, stores, people and plain code.
LLM calls: 3. Model invocations in the first scenario: each time a model or agent node starts thinking, plus each branch it decides.
Tool calls: 0. Messages sent to a tool node in the first scenario.
Steps: 2. Steps in the first scenario, with repeated steps counted every time they run.
Scenarios: 2. Distinct example runs you can switch between on the diagram.
Loop: no. Whether the flow cycles back on itself until a condition is met.
Parallel: no. Whether two or more messages are in flight at the same time.
Branch: yes. Whether the flow takes one path out of several, chosen at run time.
Human in the loop: no. Whether a person appears as a node, to approve, correct or answer.
Autonomy: low. How much the model decides the control flow itself, from fixed code paths (low) to open-ended (high).
Cost: medium. Relative token spend per request.
Latency: low. Relative wall-clock time per request.

Counted from the first scenario's timeline when the site is built.

Problem

Models write one token at a time, so a long answer takes a long time no matter how simple it is. Ask “What are the most effective ways to improve sleep quality?” and the user watches tip one, then tip two, then tip three appear, although no tip depends on another. A person writing the same answer would jot down an outline first and could hand each point to a different writer. A single model call cannot do that.

How it works

Skeleton-of-thought splits the answer into two stages. First, a skeleton call writes only an outline: a numbered list of short points, a few words each. That is quick, because the outline is short. In the example it is “1. Fixed sleep schedule 2. Cool, dark bedroom 3. Cut afternoon caffeine”.

Then every point is expanded at the same time, in separate calls: point 1, point 2 and point 3. Each expansion call sees the question and the whole skeleton, so it knows what the other points cover and does not repeat them, but it writes only its own point. The calls run side by side, so the answer waits only for the slowest one.

Finally, a join step places the expansions under their points in outline order. It is plain code, not another model call; adding a synthesis call would put back the wait the pattern removes. The full answer arrives in roughly the time of the skeleton plus one point, instead of all the points in a row. The paper reports considerable speed-ups across the 12 models it tested.

Variants

With router (SoT-R): a cheap router first decides whether the question suits an outline at all. Questions about maths, code or step-by-step reasoning go to one ordinary call; only list-shaped questions take the parallel path. The same paper proposes it.

When to use

Answers are long and split naturally into independent points: tips, pros and cons, comparisons, checklists.
Time to a complete answer matters more than token cost.
You can run several model calls concurrently.

When to avoid

Reasoning is sequential, as in maths, code or arguments where each step builds on the last. Points written in isolation will not connect.
The points must refer to each other, or the answer needs a flowing narrative.
Answers are short; the extra skeleton call adds latency rather than saving it.

Trade-offs

Skeleton-of-thought trades tokens for time. Every expansion call repeats the question and the skeleton in its input, and the skeleton is an extra call, so the total cost is higher than one long answer. The answer is also more list-like and less connected, which suits some questions and hurts others. The router variant limits both costs to the questions where the speed-up is worth it.

Failure modes

Wrong question shape: a maths problem split into independent points produces confident fragments that do not add up. Route sequential questions to an ordinary call.
Repetition: expansions overlap because each call writes in isolation. Pass the whole skeleton to every call and tell it to cover its own point only.
Uneven depth: one point gets a paragraph and another a line. Fix the expansion length in the prompt.
Slowest-call bound: one stalled expansion holds up the whole answer. Set a timeout and fall back to the skeleton line for that point.

Build it with your AI

Standard

Paste this into Claude Code, Cursor, ChatGPT or any AI coding assistant.

Implement the Skeleton-of-Thought pattern in this codebase, using the LLM client and language we already use.

## Goal

Answer a question with a long, list-shaped answer faster by generating a short outline, expanding each point in parallel calls, and joining the expansions in outline order.

## How it works

1. Call the model with the Skeleton prompt to get a numbered outline of 3 to 10 points, each only a few words.
2. Parse the outline into a list of points; if it cannot be parsed, answer the question with one ordinary call instead.
3. For every point at once, call the model with the Point-expanding prompt, passing the question, the whole skeleton and the point's number, so each expansion knows what the others cover.
4. Wait for all the expansions, then join them in outline order, each under its point's title. No model call is needed for this step.

## Prompts

### Skeleton prompt

```markdown
You are an organizer. Do not answer the question yet. Write only the skeleton
of a good answer: a numbered list of 3 to 10 points, each 3 to 5 words.

<question>{{question}}</question>

Skeleton:
1.
```

### Point-expanding prompt

```markdown
You are writing one part of an answer. The full skeleton is below so you
know what the other parts cover; do not repeat them.

<question>{{question}}</question>
<skeleton>{{skeleton}}</skeleton>

Write point {{point_index}} only, "{{point}}", in 1 to 2 concise sentences.
Do not restate the point's title.
```

## Stop rule

Stop when every expansion has returned. If one fails after a retry, keep its one-line skeleton point in its place and mark the answer as partial.

## Requirements

- Run the expansions concurrently, with a concurrency cap (default 10) and a per-call timeout.
- Cap the number of points (default 10) and the length of each expansion (default 2 sentences).
- Keep the join step as plain code; do not add a synthesis call, which would put back the latency the pattern removes.
- Log the skeleton, each expansion's latency, and the total time to the full answer.
- Add a test with a stub LLM returning a 3-point skeleton and a fixed expansion per point, with point 1 delayed longest; have the stub record its maximum concurrent calls and assert it is 3, and that the joined answer lists the points in order 1, 2, 3.

With router (SoT-R)

Paste this into Claude Code, Cursor, ChatGPT or any AI coding assistant.

Implement the Skeleton-of-Thought pattern (With router (SoT-R) variant) in this codebase, using the LLM client and language we already use.

## Goal

Use skeleton-of-thought only for questions whose answers split into independent points, and answer every other question with one ordinary call.

## How it works

1. Call a small model with the Router prompt to decide whether the question suits skeleton-of-thought (or use a small classifier trained on labelled questions).
2. If the router says no, answer the question with a single ordinary call and return that.
3. If it says yes, call the model with the Skeleton prompt to get a numbered outline.
4. Expand every point in parallel with the Point-expanding prompt, passing the question, the skeleton and the point number.
5. Join the expansions in outline order with plain code and return the result.

## Prompts

### Router prompt

```markdown
Can a good answer to this question be written as several independent points,
each of which could be written without seeing the others?

<question>{{question}}</question>

Answer "no" for maths, code, step-by-step reasoning, or very short answers.
Reply with exactly one word: yes or no.
```

### Skeleton prompt

```markdown
You are an organizer. Do not answer the question yet. Write only the skeleton
of a good answer: a numbered list of 3 to 10 points, each 3 to 5 words.

<question>{{question}}</question>

Skeleton:
1.
```

### Point-expanding prompt

```markdown
You are writing one part of an answer. The full skeleton is below so you
know what the other parts cover; do not repeat them.

<question>{{question}}</question>
<skeleton>{{skeleton}}</skeleton>

Write point {{point_index}} only, "{{point}}", in 1 to 2 concise sentences.
Do not restate the point's title.
```

## Stop rule

Stop when the single answer, or every expansion, has returned. If the router's reply cannot be parsed, take the ordinary path.

## Requirements

- Keep the router cheap: a short prompt on a small model, or a small trained classifier.
- Log the router's decision per question, and sample both paths for quality review.
- Cap the points (default 10) and run the expansions concurrently with a timeout.
- Add a test with a stub router that answers no for 'What is 17 × 23?' and yes for a list question; assert the first makes exactly one answer call and the second makes one skeleton call plus one call per point.

Prompt templates

Standard

You are an organizer. Do not answer the question yet. Write only the skeleton
of a good answer: a numbered list of 3 to 10 points, each 3 to 5 words.

<question>{{question}}</question>

Skeleton:
1.

You are writing one part of an answer. The full skeleton is below so you
know what the other parts cover; do not repeat them.

<question>{{question}}</question>
<skeleton>{{skeleton}}</skeleton>

Write point {{point_index}} only, "{{point}}", in 1 to 2 concise sentences.
Do not restate the point's title.

With router (SoT-R)

Can a good answer to this question be written as several independent points,
each of which could be written without seeing the others?

<question>{{question}}</question>

Answer "no" for maths, code, step-by-step reasoning, or very short answers.
Reply with exactly one word: yes or no.

You are an organizer. Do not answer the question yet. Write only the skeleton
of a good answer: a numbered list of 3 to 10 points, each 3 to 5 words.

<question>{{question}}</question>

Skeleton:
1.

You are writing one part of an answer. The full skeleton is below so you
know what the other parts cover; do not repeat them.

<question>{{question}}</question>
<skeleton>{{skeleton}}</skeleton>

Write point {{point_index}} only, "{{point}}", in 1 to 2 concise sentences.
Do not restate the point's title.

Related patterns

Composes well with:

Sources

Skeleton-of-Thought: Prompting LLMs for Efficient Parallel Generation (Ning et al., 2023)

From the Agentic Atlas by Joran Kikke · Edit on GitHub · Suggest a correction