You're offline — showing cached data

MC-4323

Enable + test hybrid vector recall on Luci's vault.db
2026-06-13 08:54:07 SAST
Home Board MC-4323

Enable + test hybrid vector recall on Luci's vault.db

## Goal Explain, set up, and test **hybrid (keyword+vector) recall** on Luci's own vault.db — the feature shipped to Lucienne (Mac) on 2026-05-27. Luci currently runs keyword-on...
State Done Next Action Closed Owner Luci Runtime Closed Age 16d ago
MC-4323
Ticket is done; runtime is closed. · profile claude_opus_1m_medium · cwd /home/lucienne/workspace/PKA · uptime 16d 12h · last activity 15d 2h ago

Description

MC-4323
## Goal Explain, set up, and test **hybrid (keyword+vector) recall** on Luci's own vault.db — the feature shipped to Lucienne (Mac) on 2026-05-27. Luci currently runs keyword-only (no regression, just no upgrade) because it lacks the embedder deps + an embed run. ## Background (what was built on Mac) vault.db gained a semantic/vector retrieval leg: a `sqlite-vec` `vec0` table + local `nomic-embed-text-v1.5` embeddings (via `fastembed`, ONNX, no API) + Reciprocal Rank Fusion with the existing FTS5 in `_hybrid_search` (vault_mcp.py). Code is already in the PKA repo and synced to Luci via git. - Plan: `docs/plans/2026-05-26-memory-system-v2-hybrid-recall.md` - Wiki: `wiki/memory-system.md` -> "Hybrid vector recall (added 2026-05-27)" - Key files: `scripts/vec_embed.py`, `scripts/embed_memories.py`, `_hybrid_search` in `vault_mcp.py`, `index.py` (_is_archived + CA-4 re-embed-after-rebuild, portable interpreter pick). ## Setup steps (on Luci) 1. Install deps on a Luci python that supports `enable_load_extension`: `pip install sqlite-vec fastembed --break-system-packages` (or in a venv). NOTE: Luci's `/usr/bin/python3` currently lacks sqlite_vec AND may block extension loading — verify and use a python that allows it (build/brew/conda python, not system). 2. Verify the vector leg loads: `python3 -c "import sqlite3,sqlite_vec; c=sqlite3.connect(':memory:'); c.enable_load_extension(True); sqlite_vec.load(c); print(c.execute('select vec_version()').fetchone())"`. 3. Build vectors into Luci's vault.db: `cd ~/PKA && <that-python> scripts/embed_memories.py` (embeds curated Vault/memory/** — ~300 files, a few minutes). Confirm `SELECT count(*) FROM vec_memories` > 0. 4. **BLOCKER to fix first** — Luci's `~/PKA/.mcp.json` vault server `args` point at the Mac path `/Users/elmar/PKA/vault_mcp.py` (does not exist on Luci) and `command` is the Mac brew pin. So Luci's vault MCP server likely does not run today. Fix for Luci: correct the path to `/home/lucienne/PKA/vault_mcp.py` and set `command` to the deps-having python. (This predates the hybrid work — surface to Elmar; decide whether Luci should serve vault MCP at all.) 5. Confirm CA-4: `python index.py --rebuild` should repopulate vec tables on Luci via the portable interpreter pick (index.py commit 256c78a5). If Luci's python lacks the deps it skips cleanly (FTS-only). ## Test / acceptance - Run ~10 realistic recall queries through `memory_search` (see the Mac bake-off in the plan). On Mac, natural queries hit the right memory at #1 ~10/10 vs 4/10 keyword-only. Confirm Luci shows a similar lift. - Confirm graceful FTS-only fallback when deps absent (no crash). - Restart Luci's claude session / MCP to load the new vault_mcp. - Record the before/after result in this ticket. ## Decision flag (assess before fully wiring) Does Luci even NEED hybrid recall? Luci is the ops/PM orchestrator, not doing Lucienne's curated-note lookups. Keyword-only may be perfectly adequate. Worker should judge the value and flag Elmar if low — don't over-build. Scope this cycle was deliberately Mac-only; this ticket is the deferred Luci follow-up. ## Constraints - Per eval-isolation: this touches Luci's vault.db + MCP, NOT MC core (port 3001 / scheduler / mc.db). Keep it to the PKA retrieval layer. - vault.db is per-machine + gitignored — embedding is a local Luci operation, nothing to commit. Expected check-in: 2026-05-29T06:44:03+02:00 — Shepherd released after PKA origin/master now contains the hybrid-recall code from MC-4405; worker should run the Luci-side hybrid recall validation and close or report exact blocker.

Activity

done
Luci is working...
Live
No activity yet
Help