Hierarchical Memory

OS-style paging

Implement the Hierarchical Memory pattern in this codebase, using the LLM client and language we already use.

## Goal

Give a long-running chat agent memory beyond its context window by treating the window like RAM: a small always-visible core, a log of evicted messages and a searchable archive, paged with tool calls.

## How it works

1. Build every prompt from the Memory-aware system prompt: the core memory block, then the running summary, then the recent messages.
2. Give the agent five tools: core_memory_append(text) and core_memory_replace(old, new) for core memory, recall_search(query) over the evicted-message log, and archival_insert(text) and archival_search(query) for the long-term store.
3. Run the agent loop: execute each tool call it makes, append the result, and call it again until it replies without a tool call.
4. If a tool call fails or its arguments do not parse, return the error text as the tool result so the agent can retry or move on.
5. After each turn, count the prompt's tokens. Above the threshold, evict the oldest messages: store them verbatim in recall memory and fold them into the running summary with the Running summary prompt.

## Prompts

### Memory-aware system prompt

```markdown
You are {{agent_name}}, a long-running assistant with limited context.

<core_memory>{{core}}</core_memory>
Core memory is always visible. Keep it short: stable facts about the user and yourself.

Tools:
- core_memory_append(text): save a durable fact to core memory.
- core_memory_replace(old, new): rewrite or shorten a fact in core memory.
- recall_search(query): find exact past messages that were evicted from view.
- archival_insert(text): save a fact or document to the long-term store.
- archival_search(query): look up documents and facts saved to the long-term store.
When core memory is full, move older facts out with archival_insert.
Evicted messages are saved to recall memory automatically, and a running
summary of them stays at the top of your context.

When the user states a lasting preference, save it. When you lack a fact
you should know, search before you say you don't know.
```

### Running summary prompt

```markdown
The context window is full and these messages are being evicted. Update the
running summary so it still covers them.

<previous_summary>{{previous}}</previous_summary>
<evicted_messages>{{evicted}}</evicted_messages>

Keep names, decisions, numbers and open questions. Drop pleasantries.
Write at most {{max_words}} words.
```

## Stop rule

Each turn stops when the agent replies without a tool call, or after 6 tool calls; at the cap, ask for a reply with tools disabled. The memory itself persists across sessions and is never reset.

## Requirements

- Make the eviction threshold (default 70% of the context window), the core memory limit (default 2,000 characters) and the per-turn tool cap (default 6) parameters.
- Reject a core_memory_append that would pass the limit, with an error telling the agent to move older facts out with archival_insert or shorten them with core_memory_replace.
- Store recall and archival memory outside the process, keyed by user, so they survive restarts.
- Log every memory write and every eviction, with token counts before and after.
- Add a test with a stub token counter that crosses the threshold on turn 3; assert the oldest messages are in recall memory verbatim, the summarizer was called once, and those messages are no longer in the prompt.

Rolling summary

Implement the Hierarchical Memory pattern (Rolling summary variant) in this codebase, using the LLM client and language we already use.

## Goal

Give a chat agent long-term memory with no memory tools: keep one running summary per user, rewritten after each session and placed at the top of every later prompt.

## How it works

1. When a session starts, load the user's running summary (empty for a new user) and put it at the top of the system prompt.
2. Run the session as a normal chat. The agent reads the summary but has no memory tools.
3. When the session ends (after 30 minutes idle), send the previous summary and the full session log to the summarizer with the Session summary prompt.
4. Save the returned text as the user's new running summary, keeping the previous version.
5. If the summarizer fails or returns more than the word limit, keep the old summary and fold both session logs in at the end of the next session.

## Prompts

### Session summary prompt

```markdown
Update the running summary of what you know about this user, using
the session that just ended.

<previous_summary>{{previous}}</previous_summary>
<session>{{session_log}}</session>

Keep names, dates, preferences, decisions and open tasks. If the user
corrected an earlier fact, replace it rather than keeping both.
Write at most {{max_words}} words.
```

## Stop rule

One summary update per session. The summary is never reset; it stays under the word limit because the summarizer must drop the least important facts to fit.

## Requirements

- Make the word limit (default 400) and the idle time that ends a session (default 30 minutes) parameters.
- Keep the last 5 summary versions per user, so a bad rewrite can be rolled back.
- Log each update with its word count and the session id it came from.
- Add a test with a stub summarizer that returns 'User is vegetarian.'; assert the first prompt of the next session contains 'User is vegetarian.' above the first user message.

Fact store (Mem0)

Implement the Hierarchical Memory pattern (Fact store (Mem0) variant) in this codebase, using the LLM client and language we already use.

## Goal

Keep a de-duplicated store of facts about each user: extract candidate facts after every exchange, reconcile each with similar stored facts, and retrieve the relevant ones into later prompts.

## How it works

1. After each exchange, send the user message, the reply and a short running summary to the extractor with the Fact extraction prompt, and parse its JSON list of candidate facts.
2. For each candidate, find the 5 most similar stored facts by embedding similarity.
3. Call the model with the Memory update prompt, passing the candidate and those facts, and parse its list of operations: ADD (new text), UPDATE (a fact id and the text with added detail), DELETE (the id of a contradicted fact) or NOOP.
4. Apply the operations in order to the fact store, keeping each fact's id, text, timestamps and source exchange.
5. Before each agent turn, retrieve the 5 stored facts most similar to the new message and put them in the prompt.
6. If the extractor or the reconcile reply does not parse, skip the memory update for that exchange and log it; the chat carries on.

## Prompts

### Fact extraction prompt

```markdown
Extract lasting facts about the user from this exchange.

<summary>{{summary}}</summary>
<exchange>{{user_message}} / {{assistant_reply}}</exchange>

Include preferences, plans, relationships and corrections. Skip small talk.
Respond in JSON: {"facts": ["short standalone fact", "..."]}
```

### Memory update prompt

```markdown
Decide how a new fact changes the user's stored memory.

<new_fact>{{candidate}}</new_fact>
<similar_facts>{{similar}}</similar_facts>

List the operations to apply:
- ADD the new fact if no stored fact covers it.
- UPDATE a stored fact only to add complementary detail to it
  (for example "Lisbon trip in May" becomes "Lisbon trip 12-19 May").
- DELETE every stored fact the new fact contradicts, then ADD the new fact.
- NOOP if the fact is already known.
Respond in JSON:
{"ops": [{"op": "ADD|UPDATE|DELETE|NOOP", "id": "<fact id or null>", "text": "<new text or null>"}]}
```

## Stop rule

Memory updates run once per exchange, after the reply is sent, and never block it. There is no loop to stop; the store persists across sessions.

## Requirements

- Make the similar-fact count (default 5) and the retrieved-fact count (default 5) parameters.
- Run extraction and reconciliation after the reply is sent, so the user never waits for them.
- Keep updated and deleted facts in a history table, so a wrong operation can be reverted.
- Log every operation with its candidate, the matched fact and the result.
- Add a test with the stored fact 'User loves seafood' and a stub that returns DELETE for it plus ADD 'User is vegetarian'; assert the store holds the single fact 'User is vegetarian' and the history holds the deleted text.