Two memory systems we shipped. Both running.

Chunking strategy — per-file-type-aware

Every file gets a summary chunk at index 0: filename + folder path + head + tail. That summary alone catches a surprising number of queries before any vector math runs — people remember filenames and locations more than they remember content. Paginated documents (PDF, DOCX) get one chunk per page so a hit can resolve to the right page. Long unpaginated text uses a 1500-char sliding window with 200-char overlap. Hard cap: 20 chunks per file. Embed input capped at 1800 chars per chunk. That ceiling matters — without it, a single 400-page transcript would dominate the index and starve everything else of recall budget.

Hybrid fusion math — the load-bearing 5 lines

The fusion block sits in query/retriever.py. Both scores are normalized 0–1 before mixing. We pull 200 candidates (not 20) so the reranker has headroom. Then a parent_id dedupe collapses to one row per file — best chunk wins.

# query/retriever.py — load-bearing 5 lines
_VECTOR_WEIGHT = 0.7
_KEYWORD_WEIGHT = 0.3
combined = _VECTOR_WEIGHT * v + _KEYWORD_WEIGHT * kscore

Why dedupe is load-bearing: Gemini embeddings have a 0.5–0.7 baseline cosine on plain English text. Raw vector scores aren’t the signal — relative ordering after dedupe is. If you don’t collapse parent files, the top-20 ends up being twenty chunks of the same PDF and the actual answer falls off the visible result page.

7 ranking knobs — each justified by ablation

• Synonym expansion — query “lease” also fires “rental agreement”, “tenancy”.
• Extension boost ×2 — if the query mentions “the PDF” and a candidate ends in .pdf, lift it.
• Wrong-extension penalty — the reciprocal. Query says “the spreadsheet” and the candidate is a PDF? Push it down.
• UUID penalty — Telegram caches images as AgADxxx_4321.jpg. These should lose to anything with a human-typed filename. Critical signal: a single regex saves the entire system from a corpus of opaque names.
• Folder-synonym boost — query mentions “taxes”, candidate is in /2024-Returns/. Boost.
• Filename-overlap — rewards exact token matches in the filename above content matches.
• Filename-coverage bonus — rewards a filename that contains MOST of the query tokens, not just one.

Each one is gated by an env var: FILE_FINDER_ABLATE_KNOB=<name>. We can drop a knob and re-run the 50-query benchmark to see exactly how much it earned. No black-box ranking. No “we tuned it and it works”.

Edit-history JSONL sidecar — survives Chroma rebuilds

One JSONL file per indexed file, at {chroma_path}/edits/{file_id}.jsonl. Two event types: upsert and delete. Append-only. This is the pattern most RAG teams design in 18 months after shipping, when product asks for “the lease before the addendum” and the index can’t answer. We designed it in on day one. Because the sidecar is filesystem-resident, not Chroma-resident, it survives a full vector-store rebuild — which we do every time the embedding model changes.

File watcher — debounced, single worker

watchdog Observer with a 3-second debounce. Save a file twice in two seconds: one index pass, not two. A single worker thread drains the queue and runs index_single_file per change. Single-writer keeps Chroma happy and rate-limit math sane.

Free-tier engineering as architecture posture

FILE_FINDER_BATCH_DELAY_S=12.0 by default — that’s the throttle that keeps a free-tier Gemini key under 15 RPM with headroom. Multi-key round-robin in shared/llm_round_robin.py rotates across home-email accounts when one cools. Per-provider cooldown circuit breaker (shared/llm_cooldown.py): Gemini quota = 1 hour cool, Mistral = 60s, stub ProviderUnavailable exception = 24 hours dark. The system never tries a provider it knows is down.

Per-provider Chroma collections

Chroma fixes vector dimensionality at collection creation. So we have files_lmstudio_2560 and files_gemini_3072 as separate collections, never mixed. The indexer refuses to fall back across providers — that would corrupt a dim-locked collection. The live retriever, in contrast, does fall back when LMStudioUnavailable fires — it queries the Gemini collection instead. Asymmetric: writes are strict, reads are forgiving.

The hybrid atomic record (Phase 1A schema)

Every memory is one atomic record with four layered blocks. Skeleton holds the prose summary and a list of verbatim_quotes. Perspective warp holds the per-character distortion: salience, valence, dominant_emotion, intensity, interpretation_bias. Epistemic provenance tracks how the character came to know it: acquisition (witnessed / told / overheard / lied_to), confidence, is_secret, tainted_by_liar. Optional open_loop block flags an active promise, debt, threat, or oath. Optional trauma block holds severity, trigger anchors, resolution state, and intrusion frequency. The atomic record is intentionally fat — that’s what makes downstream stages cheap.

Polyglot stores per save folder

• SQLite (WAL) — the coordinator. Owns the outbox for cross-store writes (the SealTransaction payload is gzipped JSON), every tunable lives in *_tuning tables (no magic numbers in code), plus recall_log, working memory, override_judge_state, scheduler.
• Kuzu — the graph. 14-axis relationship edges with familiarity decay. The epistemic graph (who told whom, who was tainted by a liar). Trauma anchors. Faction membership. Temporal edges with superseded_at semantics so “the way it was before” is queryable.
• LanceDB — per-NPC vector + BM25 hybrid store. episodic_memories, episodic_chunks with Matryoshka 2560/256 embeddings (use the long vector for high-stakes recall, the short one for fan-out), and semantic_memories.belief_vec for “what this character believes about the world” in vector form.

Cross-store contract

SQLite-authoritative with Kuzu reconciliation. SQLite commit is the durability point — if it commits, the memory exists. LanceDB and Kuzu writes happen post-commit. If Kuzu writes fail, we enqueue a kuzu_repair_belief job that replays from the SQLite outbox until the graph catches up. Not ACID, but recoverable. UUIDv7 round-trips across all three stores so a memory ID means the same thing everywhere.

13-stage retrieval pipeline

Multi-query fan-out generates four parallel sub-queries (literal / emotional / relational / HyDE), runs them all, then merges via RRF. Stage 4 applies a stack of biases: live decay, graph reweighting, formative-memory bump, grief bump, rumination signal, graph-walk injection. Stage 5 is avoidance suppression — candidates inside the trauma adjacency cone get pushed down (the character looks away from what hurts). Stage 7 is the lens-prefixed bge-reranker. Stage 10b is the parallel side-channel: group beliefs and cultural facts get returned in a separate field, not injected as candidates — the prompt builder chooses whether to use them.

Live decay function — six terms, multiplied at query time

Recall strength = f(current_tick - last_accessed) × g(access_count) × encode-time emotion_intensity × graph_centrality × pinned flag × trauma current_intrusion. Stored column exists only for offline sort-order. Live recall recomputes from current state because the character’s current emotional condition is itself an input — you can’t cache that.

Involuntary pipeline — the 50ms fast path

Runs per beat. Hard 50ms cap. Zero LLM calls. Three lanes in parallel: state-match (ANN over state_vec), sensory match (FTS over the denormalized sensory_anchors_flat table), and the trauma path (Kuzu HAS_TRAUMA traversal). The trauma path can fire on a small detail — smell, sound, posture — and bypasses avoidance suppression because intrusive memory bypasses avoidance in real psychology too. We modelled the pathology.

Consolidation (Phase 1E)

A job queue with priority bands 90 (highest) down to 15. Decay maintenance runs daily. Stage promotion walks raw → scene_summary → arc_summary → core. Reconsolidation is probabilistic lens replacement: when an existing memory’s pressure term crosses 5, the lens gets re-rolled against current state — the same event remembered differently because the character has changed. Trauma resolution runs as a small state machine. Sleep recombination harvests episodic memories into semantic memories. Then cleanup.

Annotated RetrievalQuery extract

What the orchestrator passes into the pipeline. Every field tells the downstream stages how to behave:

# vaelith/retrieval/types.py — RetrievalQuery (extract)
class RetrievalQuery(BaseModel):
    npc_id: UUID
    speaker_id: UUID
    text: str
    intent_type: Literal["recall", "plan", "reflect", "react"]
    scoring_profile: Literal["lite", "standard", "full"]   # picks 30/200/900ms rail
    vector_resolution: Literal[256, 2560]                       # Matryoshka tier
    lens_prefix: str                                          # compact char-state encoding
    fan_out: list[Literal["literal", "emotional", "relational", "hyde"]]
    trauma_aware: bool = True                              # enables stage-5 cone
    include_side_channel: bool = False                      # 10b group / cultural
    budget_ms: int                                            # hard cap, not advisory

scoring_profile picks the latency rail. vector_resolution picks the Matryoshka tier — short vectors for fan-out, long vectors for the final rerank candidates. lens_prefix is the compact character-state string that prefixes every reranker call. budget_ms is a hard cap — the pipeline drops stages from the tail end if it’s running long. Graceful degradation is built into the contract.