Human-in-the-Loop

Approval gate

Implement the Human-in-the-Loop pattern in this codebase, using the LLM client and language we already use.

## Goal

Pause an agent before any high-risk tool call so a person can approve, edit or reject it, then resume the run from exactly where it stopped.

## How it works

1. When the agent proposes a tool call, classify it with the Risk classifier prompt: high if it is irreversible, moves money, messages people outside the company or changes production data; otherwise low. Rules may mark some tools as always high.
2. Run low-risk calls at once and return the result to the agent.
3. For a high-risk call, save the run's state to durable storage, then send the reviewer the Approval request message with the tool, the arguments, the rationale and the effect.
4. Wait for a reply of approve, edit (with changed arguments) or reject (with a reason). Nothing runs while the run waits.
5. On approve, run the call as proposed; on edit, run it with the edited arguments; on reject, return the reason to the agent as the tool result so it replans.

## Prompts

### Approval request message

```markdown
**{{agent_name}} wants to take an action that needs your approval.**

Action: `{{tool_name}}`
Arguments: {{arguments}}
Why: {{rationale}}
Effect if approved: {{effect}}
Reversible: {{reversible}}

Reply with one of: approve, edit (give the changed arguments), reject (give a reason).
```

### Risk classifier prompt

```markdown
Classify the risk of this proposed tool call.

<tool>{{tool_name}}</tool>
<arguments>{{arguments}}</arguments>

High risk means it is irreversible, moves money, sends messages to people
outside the company, or changes production data. Otherwise it is low.
Respond in JSON: {"risk": "high" | "low", "reason": "one sentence"}
```

## Stop rule

The run ends when the agent finishes its task. A pending approval expires after 24 hours (configurable); on expiry, treat it as a rejection with the reason 'no reviewer response' and tell the requester.

## Requirements

- Persist the paused state so a restart or a deploy cannot lose a pending approval, and resume each run exactly once.
- If the risk classifier's reply is not valid JSON, retry once, then treat the call as high risk.
- Validate edited arguments against the tool's schema before running them; if they fail, send the edit back to the reviewer.
- Record who approved, edited or rejected each call, when, and the arguments that finally ran, as an audit log.
- Add a test with a stub agent that proposes a $480 refund and a fake reviewer that rejects it; assert the refund tool was never called and the agent's next input contains the rejection reason.

Agent asks

Implement the Human-in-the-Loop pattern (Agent asks variant) in this codebase, using the LLM client and language we already use.

## Goal

Let an agent pause mid-task to ask a person for information it cannot find or safely guess, and resume with the answer as if it were a tool result.

## How it works

1. Give the agent an ask_human tool that takes a question and optional choices, and tell it when to use it with the Agent instructions for asking.
2. When the agent calls ask_human, save the run's state to durable storage and send the question to the person through your usual channel (chat, email or a task inbox).
3. When the person answers, load the saved state and return the answer to the agent as the result of the ask_human call.
4. Let the agent continue its loop, calling its other tools as normal.

## Prompts

### Agent instructions for asking

```markdown
You work for {{team}}. You have tools, including ask_human.

Call ask_human only when you are missing information that you cannot
find with your other tools and that you must not guess, such as a
customer's choice, a price or a date.
Ask one specific question, give the choices if there are any, and say
in one line why you need it. Then wait for the answer.
Never use ask_human to get approval for work you can do yourself.
```

## Stop rule

The run ends when the agent finishes its task. An unanswered question expires after 24 hours (configurable); the run then ends with a status saying it is waiting on a person, and the question stays visible.

## Requirements

- Cap questions per run (default 3). On a 4th ask_human call, send nothing to the person; return a tool error telling the agent to continue on a stated assumption, and to state that assumption in its result.
- Require a single, specific question with the reason it is needed; reject an empty question as a tool error.
- Persist the paused state so a restart cannot lose a pending question, and resume each run exactly once.
- Log every question, its answer, who answered and how long it took.
- Add a test with a stub agent that calls ask_human once and then create_quote, and a fake person who answers 'Team'; assert the run paused once and create_quote received plan Team.

Act, then review

Implement the Human-in-the-Loop pattern (Act, then review variant) in this codebase, using the LLM client and language we already use.

## Goal

Let an agent make reversible changes without waiting, by saving a checkpoint before each change and letting a person review afterwards and roll it back.

## How it works

1. Allow this mode only for tools whose effects can be undone; send irreversible calls through an approval gate instead.
2. Before each change, save a checkpoint: the current values of every record the call will touch, with a checkpoint id.
3. Run the call at once, then put a review item in the reviewer's queue with the checkpoint id and a before-and-after diff.
4. On approve, mark the review done and keep the checkpoint until it expires.
5. On reject, write the saved values back through the same tool, check that each record now matches the checkpoint, and tell the agent and the requester.

## Prompts

### Approval request message

```markdown
**{{agent_name}} wants to take an action that needs your approval.**

Action: `{{tool_name}}`
Arguments: {{arguments}}
Why: {{rationale}}
Effect if approved: {{effect}}
Reversible: {{reversible}}

Reply with one of: approve, edit (give the changed arguments), reject (give a reason).
```

### Risk classifier prompt

```markdown
Classify the risk of this proposed tool call.

<tool>{{tool_name}}</tool>
<arguments>{{arguments}}</arguments>

High risk means it is irreversible, moves money, sends messages to people
outside the company, or changes production data. Otherwise it is low.
Respond in JSON: {"risk": "high" | "low", "reason": "one sentence"}
```

## Stop rule

A change is final when it is approved or its review window ends (default 24 hours), when the checkpoint is discarded. A rejection after that window must be fixed by a new change, not a rollback.

## Requirements

- Make the review window a parameter (default 24 hours).
- Before restoring a record, check it has not changed since the agent's change; if it has, do not overwrite it, and flag it for the reviewer.
- Make the restore idempotent, so retrying it after a failure is safe.
- Log every change, review decision and restore with the checkpoint id.
- Add a test with a stub agent that reassigns 3 contacts and a fake reviewer that rejects the change; assert all 3 contacts are back with their original owner and the restore ran once.