4 patterns
Efficiency & Cost
Spend fewer tokens, dollars and seconds: escalate only when needed, reuse what was already computed, parallelize generation, and batch what can wait.
Patterns in Efficiency & Cost
Model Cascade
Efficiency & Cost · 5 nodes · 2 LLM calls · branch · 2 scenarios
Try the cheapest model first; a scorer checks the answer and escalates to a stronger, pricier model only when confidence is low.
Caching & Reuse
Efficiency & Cost · 5 nodes · 2 LLM calls · parallel · 1 scenario
Avoid recomputing. Reuse the processed prefix of a stable prompt, or return a stored answer when a request matches an earlier one.
Batch & Throttled Fan-out
Efficiency & Cost · 6 nodes · 3 LLM calls · loop · parallel · 1 scenario
Run thousands of independent requests through a rate limiter with retries, or submit them as one provider batch at half price.
Skeleton-of-Thought
Efficiency & Cost · 7 nodes · 4 LLM calls · parallel · 1 scenario
Write a short outline first, then expand every outline point in parallel and join the results, cutting time to a complete answer.