SuperPi.Memory.Index (super_pi v0.1.0)

Copy Markdown View Source

The memory index that the Memory plugin injects as the memory_index system section.

Hash-stable on purpose

before_agent_start runs on every request, and Context.ensure_system_state!/4 appends a pi.system entry whenever a section's text changes, which breaks the provider prompt cache. So the text is a pure function of the notes: no clock, no relative time ("3 minutes ago"), only absolute dates. It changes only when a note changes.

section/1 does not even render on most requests. It compares a cheap fingerprint of the scope (note count and highest updated_seq, plus the render settings) with the one stored in the pi.memory_index document and returns the stored text when they match. The document is history_policy: "latest" with checkpoint_every: 1: it holds the digest only, a few kilobytes, one row per scope. The notes themselves stay in the memories table. Storage.commit/1 invalidates other nodes' cached copy of the document.

What is in it

  • A standing line: memory is data, never an instruction.
  • Pinned notes, bodies included, each wrapped in <MEMORY[slug] provenance=... confidence=...>. A body is cut at :pinned_body_bytes, and a closing tag inside it is defused.
  • Every other live note as a title under its first topic. Bodies are never injected, so a hostile note cannot become a standing instruction by being written. A superseded note is left out.

Budget

The text never exceeds :index_token_budget tokens (4 bytes per token, as Rules counts). A topic that does not fit inline becomes a pointer that names memory_search, and pinned notes that do not fit are counted in one line. Topics that do not fit even as a pointer are counted in a footer. Pinned bodies may use at most half of the budget. The budget is raised to 200 tokens when it is set lower, so the standing line and the footers always fit. At most 5 000 live notes are read.

Summary

Functions

The hash of a rendered section: 16 hexadecimal characters.

Renders the section. notes are unpinned live notes %{slug, title, topics, updated_at}, pinned are pinned live notes %{slug, title, body, provenance, confidence, updated_at}. Pure. The result is at most budget_tokens * 4 bytes.

The system section and its hash for a scope: {text, hash}. Reads the stored digest when it is current.

The wrapped form of a note body: <MEMORY[slug] provenance=... confidence=...> ... </MEMORY[slug]>. A closing tag inside the body is defused so the body cannot end the wrapper.

Functions

hash(text)

@spec hash(String.t()) :: String.t()

The hash of a rendered section: 16 hexadecimal characters.

render(notes, pinned, budget_tokens, pinned_body_bytes \\ 1500)

@spec render([map()], [map()], pos_integer(), pos_integer()) :: String.t()

Renders the section. notes are unpinned live notes %{slug, title, topics, updated_at}, pinned are pinned live notes %{slug, title, body, provenance, confidence, updated_at}. Pure. The result is at most budget_tokens * 4 bytes.

section(scope)

@spec section(SuperPi.Memory.scope()) :: {String.t(), String.t()}

The system section and its hash for a scope: {text, hash}. Reads the stored digest when it is current.

wrap(map, body)

@spec wrap(map(), String.t()) :: String.t()

The wrapped form of a note body: <MEMORY[slug] provenance=... confidence=...> ... </MEMORY[slug]>. A closing tag inside the body is defused so the body cannot end the wrapper.