Vault and Memory Overview

How Luci's knowledge persistence works: the vault architecture, directory structure, database schema, MCP server tools, indexer, and the two-DB split between vault.db and mc.db.

Core Principle

Markdown files are the source of truth. vault.db is a derived query index. mc.db is Luci's operational database.

The vault.db can be rebuilt from files at any time via python index.py --rebuild. Only the activity_log table is authoritative in the DB (append-only, not derived from files).

Two-DB Architecture

Implemented 2026-04-07. Two databases with clear ownership, one MCP server that queries both transparently.

vault.db -- Knowledge Graph (Lucienne owns)

• Location: PKA/Vault/vault.db (tracked in git)
• Written by: Lucienne only (Luci NEVER writes -- excluded via .git/info/exclude)
• Rebuilt by: python index.py --rebuild (file index) + python session_indexer.py (session transcripts)
• Purpose: Query index over markdown files, knowledge graph, full-text search

mc.db -- Mission Control Ops (Luci owns)

• Location: ~/workspace/mission-control/mc.db on Luci's server
• Written by: mc_pickup.py, scheduler.py, heartbeat.py, MC API (app.py), session_indexer.py
• Read by Lucienne: Via MC API at 100.118.207.3:3001 (never direct DB access)
• Purpose: Tickets, task runs, heartbeats, sessions, queued messages

Rules

1. Lucienne NEVER writes to mc.db (access via API only)
2. Luci NEVER writes to vault.db (read-only via git pull)
3. Cross-system queries go through the MC API, not direct DB access
4. vault.db travels via git sync (every 15 min)

Directory Structure

PKA Vault (PKA/Vault/)

Project-specific structured data:

Vault/
  vault.db          # SQLite query index (derived from files)
  vault.db-wal      # WAL journal
  vault.db-shm      # Shared memory
  schema.sql        # DDL for vault.db (v2, 2026-03-30)
  memory/           # Memory files (durable reference knowledge)
    MEMORY.md       # Memory index -- lists all memories
    auto/           # Auto-generated memories from Claude sessions
    *.md            # Individual memories (feedback, project, reference, user)
  tasks/            # Task files with YAML frontmatter
  projects/         # Project tracking files
  notes/            # General notes

Personal Vault (~/.claude/vault/)

Cross-project personal data shared between PKA, CoWork, and other projects:

~/.claude/vault/
  identity/         # KYC profiles (Elmar, Nicolette, children)
  entities/         # Company profiles (ENCFT, Werda, YellowNickel), group financials
  contacts/         # Master contact list (people + companies)
  work/             # FlySafair role/team/tools, glossary
  secrets/          # API keys, POPIA passwords, SSH keys
  preferences/      # About me, communication style
  infrastructure/   # Server profiles, network/SSH config, service inventory
  references/       # Reference documents
  scripts/          # Utility scripts
  session_tracker.py  # Session registration
  sessions.json     # Session log
  next-session.md   # Carry-forward notes

Database Schema (vault.db)

Schema v2 (2026-03-30). Key tables:

Core Tables

• files -- One row per indexed .md file. Columns: path, root (pka or personal), title, type, status, priority, due_date, assigned, project, frontmatter (JSON), content_hash, file_modified
• edges -- Relationships between files (knowledge graph). Columns: source_path, target_path, relation, strength, notes, source (frontmatter or wikilink). As of MC-462: 197 edges (up from 42) after auto-generation from shared projects, entities, and content similarity.
• relation_types -- 6 active relation types after MC-462 pruning (was 18): related_to, subtask_of, belongs_to, depends_on, assigned_to, references. Unused types removed.
• tags / tag_assignments -- Tag taxonomy with hierarchical parent support

Operational Tables

• activity_log -- Append-only, authoritative (NOT derived from files). Columns: date, actor, category, summary, details, memory_file. Categories: task_completed, decision, milestone, error, scheduled_run, etc.

Removed (MC-462, 2026-04-09): vitals_log and task_runs were dropped from vault.db. Both were always empty — all task runs go to mc.db, and vitals were never populated. vault_cleanup.py handles this cleanup.

Search

• search_fts -- FTS5 virtual table over path, title, content_text, tags. Uses porter + unicode61 tokenizer.
• sessions_index / sessions_fts -- Indexed session transcripts (added by session_indexer.py)

Views

• v_upcoming_tasks -- Active tasks sorted by due date and priority
• v_active_projects -- Projects with open/done task counts
• v_team -- Team member roles and specialties
• v_graph -- Edge list with source/target metadata (for visualization)
• v_file_tags -- Files with concatenated tag lists
• v_memory_health -- Memory staleness tracking (checks last_reviewed against stale_after)

Vault MCP Server

Script: vault_mcp.py (runs as stdio MCP server, configured in .mcp.json)

Provides 8 tools for Claude Code sessions to query the knowledge graph:

The three cross-boundary tools (session_search, activity_log, luci_status) call the MC API at http://100.118.207.3:3001 over HTTP with bearer token auth (mc-lucienne-2026).

Circuit breaker (MC-462, 2026-04-09): HTTP fallback calls use a circuit breaker — after 3 consecutive failures, the breaker opens for 5 minutes and returns a clear error instead of hanging. This prevents silent degradation when Luci is down.

Ticket context auto-logging: When memory_recall or file_lookup is called during a worker session (detected via MC_TICKET_ID env var), the consulted vault files are automatically logged to mc.db's ticket_context table via the MC API. This creates a ticket↔vault bridge so tickets know which knowledge was consulted.

Connection: Read-only to vault.db (file:...?mode=ro). Never writes.

Indexer

Script: index.py

Scans markdown files from two roots and populates vault.db:

Scan Roots

1. PKA root (label: pka) -- Team/.md, Vault/memory/.md, Vault/tasks/.md, Vault/projects/.md, Vault/notes/*.md, Lucienne.md, CLAUDE.md, SETUP.md
2. Personal vault (label: personal) -- identity, contacts, entities, preferences, work, infrastructure (excludes secrets/)

What It Does

1. Collects all .md files matching glob patterns
2. Parses YAML frontmatter (title, type, status, priority, due_date, assigned, project, tags)
3. Computes content hash for incremental updates (skips unchanged files)
4. Upserts into files table
5. Creates edges from:
6. Frontmatter related: field (relation: related_to)
7. Typed frontmatter fields: project -> belongs_to, assigned -> assigned_to, depends_on, blocks, parent -> subtask_of
8. [wikilinks](/wiki/wikilinks) in body text (relation: references)
9. Populates FTS5 index (search_fts)
10. Cleans up deleted files from all tables
11. Logs the run to activity_log

Usage

python index.py              # Incremental (skip unchanged files)
python index.py --rebuild    # Full rebuild (preserves activity_log, vitals_log, task_runs)
python index.py --stats      # Show index statistics

Rebuild safety: --rebuild drops and recreates the DB from schema.sql, but first saves and restores activity_log, vitals_log, and task_runs rows.

Memory File Format

Memory files use YAML frontmatter followed by markdown content:

---
name: Memory Title
description: One-line summary
type: project | feedback | reference | user
created: 2026-04-07
last_reviewed: 2026-04-07
status: active
owner: lucienne | luci | both
stale_after: 30d | never
---

## Content

Markdown body with details, decisions, and context.

Key frontmatter fields: - type -- Categorizes the memory (project, feedback, reference, user) - status -- Lifecycle state (active, archived, etc.) - stale_after -- Duration before memory is considered stale (e.g., 30d, never). The v_memory_health view uses this to flag memories needing review. - last_reviewed -- Date of last human/system review - owner -- Who maintains this memory (lucienne, luci, both)

Memory index: Vault/memory/MEMORY.md is the manually curated index that lists all memories grouped by category (User, Project, Reference, Feedback).

Session Transcript Indexing

• session_indexer.py parses JSONL session transcripts from ~/.claude/projects/
• Extracts: user requests, summaries, tools used, key files
• Inserts into sessions_index + sessions_fts (FTS5) tables
• Local: sessions in vault.db; Luci: sessions in mc.db
• Run manually: python session_indexer.py (incremental) or --rebuild

Git Sync

• vault.db is tracked in git (added 2026-04-07)
• Luci's git-sync task (every 15 min) pulls the repo, getting the latest vault.db
• Luci excludes vault.db from commits via .git/info/exclude (read-only consumer)
• mc.db and .bak files are also excluded from git

Related Articles

• 01-architecture/overview -- System architecture
• 02-mission-control/overview -- MC tickets and mc.db
• 05-authentication/overview -- Credential storage
• 07-skills/overview -- Skills that query the vault

Key Takeaways

• Markdown is the source of truth; vault.db is a rebuildable index over those files
• Two-DB split: vault.db (knowledge graph, Lucienne owns) vs mc.db (operations, Luci owns)
• Vault MCP server provides 8 tools; 3 of them cross the DB boundary via MC API
• Indexer scans PKA + personal vault, parses frontmatter, builds edges from wikilinks and typed fields
• Memory files use YAML frontmatter with staleness tracking (stale_after + last_reviewed)
• index.py --rebuild is safe: preserves activity_log
• Session transcripts are separately indexed by session_indexer.py into FTS5 tables
• vault_cleanup.py removes dead tables (vitals_log, task_runs) and can prune orphaned graph nodes
• vault_mcp.py uses a circuit breaker for HTTP fallback calls (3 failures → 5 min cooldown)
• Worker sessions auto-log consulted vault files to mc.db ticket_context table