Memory Systems for AI Agents

Executive Summary

1. Current Landscape

The Ten Systems

Key Observations

Convergence on files: Three independent high-profile systems (Claude Code, GBrain, Manus) converged on markdown files as the memory substrate. Harrison Chase (LangChain) explicitly stated: "I very, very strongly believe that if you're building a long-horizon agent, you need to give it access to a file system." This is not a coincidence — files are inspectable, versioned, portable, and work with every tool.

MemGPT's conceptual influence: Even systems that don't use MemGPT's code adopt its mental model — tiered memory with working/short-term/long-term layers and agent-managed promotion between tiers. This is the dominant conceptual framework in the field.

Mem0 is the production benchmark: With 186M API calls/quarter and published benchmarks (26% accuracy improvement over OpenAI, 91% lower p95 latency, 90% token cost savings), Mem0 is the system to beat on quantitative metrics. Its graph memory variant adds entity relationship tracking.

GBrain's specific contribution: The "compiled truth + timeline" page model is genuinely novel. Each entity page has a rewritable top section (current understanding) and an append-only bottom section (evidence trail with dates). This separates the compiled state from the raw evidence — you can always trace why the system believes something.

2. Retrieval Architectures

The Four Approaches

Vector-only (pgvector, Pinecone, Weaviate, Chroma)

• pgvector: 5-8ms HNSW at <5M vectors; pgvectorscale hit 471 QPS at 99% recall on 50M vectors
• Best for semantic similarity ("planned parenthood" → "reproductive rights")
• Worst for exact matches (entity names, error messages, project codes)
• Cost: managed vector DBs run $50-200/GB/month vs $0.02/GB for local disk

Keyword-only (FTS5, Elasticsearch)

• SQLite FTS5: sub-millisecond BM25 search, zero dependencies, single-file deployment
• Best for exact intent — "what you search is what you get"
• A contrarian benchmark showed FTS5 beating Pinecone on 4,300 agent memories: <1ms vs 50-200ms
• Worst for semantic fuzzy matching

Hybrid (RRF + reranking)

• BM25 + dense retrieval fused via Reciprocal Rank Fusion, then cross-encoder reranking
• TREC iKAT 2025: nDCG@10 improved from 0.4218 to 0.4425 with RRF before reranking
• The consensus "best pipeline": BM25 + vector → RRF → cross-encoder reranker → top-5
• GBrain uses this pattern natively with pgvector + Postgres FTS

Graph-based (Neo4j, GraphRAG, Graphiti)

• Microsoft GraphRAG: LLM builds entity knowledge graph, pregenerates community summaries
• Zep/Graphiti: 94.8% on DMR benchmark (vs MemGPT's 93.4%), bi-temporal model
• Best for multi-hop reasoning across entities
• Worst for setup cost and maintenance; Hamel Husain: "Graph databases are unnecessary — simpler solutions like CSV or Postgres typically suffice"

What Luci Uses Today

Luci currently uses grep + glob (keyword search over markdown files). This is equivalent to basic keyword retrieval with no semantic matching and no ranking. It works because:

1. The corpus is small (~200 memory files, ~50 vault files)
2. Entity names are explicit and greppable
3. MEMORY.md index provides a manual routing layer

The gap: as the corpus grows, grep won't surface semantically related memories that don't share exact keywords.

Recommendation for Luci

Skip vector search for now. At Luci's scale (<1,000 memory entries), FTS5 + the existing grep/glob is sufficient. Vector search adds value at 10K+ entries. Revisit if the corpus grows.

Skip graph memory. Entity relationships can be captured in compiled entity pages (markdown links) rather than a graph database. The maintenance overhead of Neo4j/Graphiti is not justified for a single-user agent.

3. Data Model Patterns

The Four Models

Markdown-as-source-of-truth (Luci's current model, Karpathy's LLM Wiki, Claude Code)

• Files are the canonical store. The LLM reads, writes, and maintains them.
• Git provides versioning and audit trail.
• Human-readable and editable — Elmar can inspect and correct any memory.
• Scales poorly beyond ~10K notes without indexing. Temporal queries ("what happened last week?") require additional metadata.

Append-only event logs (MemGPT/Letta)

• Every interaction is an immutable event. The agent pages relevant events into working context.
• Preserves full history — nothing is lost.
• Requires active management (the agent must decide what to promote/archive).
• Can grow unbounded without consolidation.

Compiled entity pages (GBrain, Karpathy Wiki)

• Auto-generated pages for people, projects, and concepts that aggregate all mentions.
• Top section: rewritable current understanding (compiled truth).
• Bottom section: append-only evidence trail with dates and sources.
• The knowledge compounds — each new source enriches existing pages.

Conversation threads (OpenAI Assistants, Memoria)

• Raw conversation history as the memory substrate.
• Simple but expensive (re-processes full thread per query).
• OpenAI's approach scored worst on LOCOMO benchmark (52.9% accuracy).

What Luci Should Adopt

The compiled entity page pattern from GBrain/Karpathy is the single highest-value addition to Luci's memory. Here's why:

1. Luci already has the raw data. vault.db has entities, edges, and file references. email.db has contacts. whatsapp-messages.db has conversation history. The data exists but isn't compiled.
2. Entity pages solve the "who is X?" problem. When Elmar mentions "Stephan" or "Chazelle" or "NMG", Luci should be able to pull up a compiled page: who they are, recent interactions, key facts, contact details, linked projects.
3. The compilation can run as a dream cycle. A nightly script scans vault.db, email.db, and whatsapp-messages.db, extracts entity mentions, and compiles/updates markdown pages in a ~/.claude/memory/entities/ directory.
4. Markdown stays the source of truth. The compiled pages are derived artifacts — regenerable from the raw data. Git tracks changes. Elmar can inspect and correct.

4. Entity Extraction and Enrichment

Current State of the Art

• Few-shot LLM extraction now matches or beats supervised NER models without training data. GPT-4/Claude can extract entities, relationships, and temporal expressions with high accuracy.
• Auto-disambiguation remains hard. "Apple" in tech vs food contexts. Joint NER + entity disambiguation (2025 state of art) helps but isn't perfect.
• Temporal validity is an open problem. MemoryBank uses Ebbinghaus forgetting curves (unaccessed memories decay). A-MEM uses memory evolution (new facts update old memories). No system handles both gracefully.

Who Does It Well

• Mem0: Graph memory variant with entity relationship tracking. 26% accuracy improvement.
• A-MEM: Zettelkasten-inspired linked notes with auto-updating. SOTA on 6 foundation models.
• Zep/Graphiti: Bi-temporal entity tracking (when it happened vs when it was ingested). 94.8% on DMR.

What Luci Should Do

This is Tier 2 work — clear requirements, known patterns. The dream cycle script could be a Python scheduler task that calls Claude via the API for extraction.

5. Dream Cycles / Overnight Consolidation

Who Does It

The Sleep-time Compute Paper (Lin et al., 2025)

This is the strongest academic justification for dream cycles. Key findings:

• ~5x reduction in test-time compute for equivalent accuracy
• Up to 13% accuracy improvement on GSM-Symbolic, 18% on AIME
• 2.5x cost reduction per query through amortization
• Effectiveness correlates with query predictability — consolidation should focus on information the user is likely to ask about again

The insight: decompose the prompt into static context (pre-processable between sessions) and dynamic query (real-time). Sleep-time compute enriches the static context during idle periods.

What Luci Should Build

This is the GBrain dream cycle pattern adapted for Luci's infrastructure — no Postgres, no pgvector, just markdown + SQLite + Claude API.

6. Academic Foundations

The Three Essential Papers

1. Generative Agents (Park et al., Stanford, 2023)

• Introduced the memory stream + reflection + planning architecture
• Reflection was the single biggest quality contributor
• Retrieval scoring: recency x importance x relevance (all three needed)
• Limitation: no forgetting, memory grows unbounded

2. Sleep-time Compute (Lin et al., Letta, 2025)

• Academic foundation for dream cycles
• 5x compute reduction, 13-18% accuracy improvement
• Key insight: amortize reasoning across queries during idle time
• Authors are MemGPT founders — direct lineage

3. Episodic Memory Position Paper (Pink et al., 2025)

• Argues Tulving's episodic/semantic distinction is critical for agents
• Episodic: raw events with timestamps ("Elmar said X on 2026-04-10")
• Semantic: extracted facts ("Elmar prefers direct communication")
• The consolidation pathway (episodic → semantic) is the dream cycle

Supporting Papers

• MemoryBank (Zhong et al., 2023): Ebbinghaus forgetting curves for AI memory. Unaccessed memories decay.
• A-MEM (Xu et al., 2025): Zettelkasten method for agent memory. Memory evolution: new memories update old ones.
• Recursively Summarizing (Wu et al., 2023): Hierarchical compression of conversation history. The algorithm for the dream cycle's "merge" operation.
• Memory Survey (Zhang et al., 2024): 39-page taxonomy. Three dimensions: sources, forms, operations.
• Memory in the Age of AI Agents (Hu et al., 2025): Most comprehensive survey. 47 authors.
• Zep Paper (Rasmussen et al., 2025): Temporal knowledge graphs. Bi-temporal model (event time vs ingestion time).

7. Expert Commentary and Contested Questions

Where Experts Agree

1. Memory is essential for agents — every named expert agrees on this
2. Human-readable, inspectable memory is preferable — Karpathy, Willison, Chase, Anthropic all converge
3. File systems are a legitimate memory substrate — Chase: "give it access to a file system"; three independent projects converged on markdown
4. Consolidation/dream cycles add measurable value — Stanford reflections, Letta sleep-time compute, GBrain dream cycles

Where Experts Disagree

RAG vs Long Context

Vector DBs — Necessary?

Graph Memory — Worth the Overhead?

The Contrarian Case: "Just Use Markdown Files"

Three strong arguments:

1. Cost: $0.02/GB/month local disk vs $50-200/GB managed vector DBs
2. Inspectability: Human can read, edit, correct. No black-box embeddings.
3. Convergence: Claude Code, GBrain, Manus all independently chose markdown. This is a strong signal.

Counter-arguments:

• Markdown doesn't scale past ~10K files without indexing
• No semantic search (grep misses conceptual matches)
• Temporal queries require additional metadata
• Multi-hop reasoning across files requires iterative LLM calls

Production Failure Lessons

• Context rot: Models struggle with information buried in the middle of large contexts — a "dead zone" that expands with larger datasets (Boschi, 2026)
• Manus context budget: ~30% overhead from managing state files
• Retrieval misses are catastrophic for coding agents: Wrong chunks → broken code → cascading failures (MindStudio)
• No implicit memory extraction: Systems capture explicit statements but miss behavioral patterns (swyx)

8. Tradeoffs Matrix for Luci

9. Decision Framework

The Four Options

Option A: Keep current markdown approach (do nothing)

• Pro: Working, simple, aligned with expert consensus
• Con: No entity compilation, no consolidation, no ranked retrieval
• Risk: Memory quality degrades as corpus grows
• Cost: Zero

Option B: Add entity pages + dream cycle + FTS5 (recommended)

• Pro: Highest value per effort, stays on markdown, adds the three missing capabilities
• Con: Requires building the dream cycle script and entity extraction pipeline
• Risk: Low — it's additive, doesn't change existing infra
• Cost: ~1 week of dev time (Tier 2 ticket)
• This is the "compiled people-pages from existing infra" option from the ticket

Option C: Adopt GBrain prototype

• Pro: Battle-tested by Garry Tan, comprehensive feature set, MIT licensed
• Con: Requires Postgres (PGLite or full), opinionated toward Garry's workflow, tightly coupled to OpenClaw ecosystem
• Risk: Medium — introduces Postgres dependency, may not integrate cleanly with Claude Code's native file tools
• Cost: ~2-3 weeks including migration and adaptation

Option D: Adopt a different pattern (Mem0, Letta, etc.)

• Pro: Most mature production systems with benchmarks
• Con: API dependency (Mem0 hosted) or significant self-hosting complexity (Letta), reduced inspectability
• Risk: Medium-high — vendor lock-in or maintenance burden
• Cost: ~2-4 weeks

Recommendation: Option B — Enhanced Markdown

Patterns to Steal Without Adopting the Whole Stack

10. Contested Questions — Where I (Luci) Stand

RAG vs long context? At Luci's scale (<1,000 memory entries), long context wins. The entire memory corpus fits in the 1M token window. But Chip Huyen's Context Expansion Law applies — as Luci grows, we'll need retrieval. FTS5 is the cheapest hedge.

Vector DBs necessary? No, not yet. pgvectorscale is impressive (471 QPS at 99% recall), but FTS5 at <1ms for keyword search covers Luci's needs. Revisit at 10K+ entries.

Graph memory? No. Entity relationships are better captured as markdown links between compiled entity pages. The overhead of Neo4j/Graphiti is not justified for a single-user agent.

GBrain? Respect the ideas, don't adopt the stack. GBrain's best concepts (compiled truth + timeline, dream cycles, hybrid retrieval) can be implemented on Luci's existing markdown + SQLite infrastructure without Postgres.

Is markdown enough? Yes, with enhancements. Karpathy, Willison, Chase, and Anthropic all validate this approach. The failures reported in production (Boschi 2026) are at scales (500+ files, multi-hop cross-file reasoning) that Luci hasn't hit yet. When we do, FTS5 + entity pages + dream cycles provide the safety net.