Ambient Agent

Problem

Most agents wait for someone to type. That fits questions, but much useful work does not start with a question: a new email arrives, a ticket is filed, an alert fires, a scheduled report comes due. If a person has to notice each event and open a chat to hand it to an agent, the agent saves little time. Yet letting an agent loose on a live inbox with no way to check with anyone is worse: it will accept the wrong meeting or answer a question only its owner can.

How it works

An ambient agent is woken by events, not messages. Here the email feed delivers every new message to triage: a cheap model call that labels each one ignore, notify or act. Newsletters go straight to the archive, and notices that need no action, such as a shipping update, go straight to the inbox as a notify card. The agent never runs on either, which keeps costs in line with the events that matter.

For events that need work, the email agent runs in the background with its mail and calendar tools. Routine items it finishes on its own: a customer asking where an order is gets a looked-up, sent reply. Nobody is waiting on it, so a run can take its time.

It reaches a person only through the agent inbox, with one of three kinds of card, as described by Harrison Chase:

• Notify: something is important, but no action is needed.
• Question: the agent is stuck on a fact only the person knows, such as which project code goes on an invoice.
• Review: the agent wants to take a sensitive action, such as accepting a meeting, and shows the exact call for approval, editing or rejection.

When it posts a question or review card, the run saves its state and pauses. The person handles cards when they choose, hours later if need be, and their answer resumes the run where it stopped. Which tools need review is set in code, not left to the model.

When to use

• The work arrives as a stream of events: email, tickets, alerts, pull requests, scheduled jobs.
• Most events are routine and only a few need a person’s judgement.
• People want the leverage of an assistant without starting each task themselves.

When to avoid

• Every action is high-stakes, so the person would review every card anyway; a synchronous tool is simpler.
• Events are rare. A button the user presses is cheaper to build and easier to trust.
• You cannot persist runs across long pauses. Without durable state, a question left overnight is lost.

Trade-offs

Ambient agents trade immediacy for leverage. The person does less, but they also see less, so trust depends on a clear activity log and well-chosen review gates: too many cards and the inbox becomes another chore; too few and the agent acts beyond what its owner would allow. Running on every event costs money even when nothing happens, which is why a cheap triage step sits in front. And because runs pause for people, the system needs durable storage and expiry rules for cards nobody answers.

Failure modes

• Card fatigue: the agent asks about everything and the person stops reading. Tune triage and review gates from what people actually approve unchanged.
• Silent overreach: the agent sends something its owner would have stopped. Keep sensitive tools behind review in code and log every autonomous action.
• Event storms: a mail loop or alert flood starts thousands of runs. De-duplicate events and rate-limit per source.
• Stale answers: a card answered days later resumes a run whose world has moved on. Re-check key facts on resume and expire old cards.
• Prompt injection: events come from outsiders, so an email can contain instructions. Treat event content as data and keep risky tools gated.

Implement the Ambient Agent pattern in this codebase, using the LLM client and language we already use.

## Goal

Run an agent in the background on a stream of events, so it handles routine items on its own and reaches a person only through notify, question and review cards in an inbox.

## How it works

1. Subscribe to the event source (a mail webhook, a queue, a schedule) and start one run per event, keyed by the event id so a redelivered event is not handled twice.
2. Call a triage model with the Triage prompt to label each event ignore, notify or act. Archive ignored events; post notify events to the inbox as an information card.
3. For act events, run the agent loop with the Background agent prompt and its tools, which are split into safe tools (draft, search, read calendar) and review-gated tools (send to a new contact, accept a meeting, pay).
4. When the agent calls ask_user, post a question card to the inbox, save the run's state, and stop. When it calls a review-gated tool, post a review card with the exact arguments instead of running it.
5. When the person answers or decides, resume the run from its saved state with the answer, or with the approved, edited or rejected tool call, and continue the loop.
6. Record every action the agent took on its own in an activity log the person can browse and undo.

## Prompts

### Triage prompt

```markdown
You screen incoming email for {{user_name}}. Label the email with exactly one of:

- ignore: newsletters, marketing, automated notices that need nothing
- notify: {{user_name}} should know, but no action is needed
- act: it needs a reply, a scheduling decision or another action

<preferences>{{triage_preferences}}</preferences>
<email>{{email}}</email>

Reply in JSON: {"label": "ignore|notify|act", "reason": "one sentence"}
```

### Background agent prompt

```markdown
You handle email for {{user_name}} in the background. They are not watching,
so act on routine items yourself and use the inbox only when you must.

<email>{{email}}</email>
<preferences>{{preferences}}</preferences>

- Use the tools to draft replies, look up context and check the calendar.
- If you need a fact only {{user_name}} knows, call ask_user with one short question.
- Some tools need {{user_name}}'s approval; call them normally and the system
  will ask. Do not try to work around a rejection.
- When the email is fully handled, reply DONE with one line on what you did.
```

## Stop rule

Each run stops when the agent replies DONE, after a step cap (default 15), or when it is waiting on an inbox card. Cards that go unanswered for a set time (default 3 days) expire and the run is closed with a notify card saying what was left undone.

## Requirements

- Persist pending runs and inbox cards in durable storage, so a question can wait days and survive restarts.
- Make the list of review-gated tools configuration, not prompt text; the code, not the model, decides what needs approval.
- De-duplicate events by id and rate-limit runs per sender, so a mail loop or a flood of alerts cannot start thousands of runs.
- Log every event with its triage label, every tool call, and every card with how long it waited for a person.
- Add a test with a stub LLM where triage labels a meeting invite act and the agent calls accept_invite; assert a review card is created, the calendar tool is not called, and it is called exactly once after the card is approved.

You screen incoming email for {{user_name}}. Label the email with exactly one of:

- ignore: newsletters, marketing, automated notices that need nothing
- notify: {{user_name}} should know, but no action is needed
- act: it needs a reply, a scheduling decision or another action

<preferences>{{triage_preferences}}</preferences>
<email>{{email}}</email>

Reply in JSON: {"label": "ignore|notify|act", "reason": "one sentence"}

You handle email for {{user_name}} in the background. They are not watching,
so act on routine items yourself and use the inbox only when you must.

<email>{{email}}</email>
<preferences>{{preferences}}</preferences>

- Use the tools to draft replies, look up context and check the calendar.
- If you need a fact only {{user_name}} knows, call ask_user with one short question.
- Some tools need {{user_name}}'s approval; call them normally and the system
  will ask. Do not try to work around a rejection.
- When the email is fully handled, reply DONE with one line on what you did.