MentisDB Blog

MentisDB 0.10.8.53 — Technical Debt: Sidecar WAL Brick, CLI Thought Types

This is a technical-debt release. It closes the two open bug reports filed against the 0.10.x line — a vector sidecar bug that could make a chain stop opening, and a CLI flag that was documented but not wired — and removes three copies of the thought-type table that had already drifted apart.

1. A chain could brick itself after 32 appends

The sidecar WAL added in 0.10.6 stores one integrity-chained record per append and compacts the JSON snapshot every 32 records. Compaction collapsed snapshot + WAL into a single snapshot carrying the full-corpus digest, but it never told the live in-memory sidecar. The next append then chained its WAL record to the pre-compaction digest.

Nothing failed until the process restarted. On the next open, replay saw a snapshot digest that did not match the first WAL record:

InvalidData("vector sidecar WAL digest chain mismatch")

Because the chain could not be opened, the daemon rejected every write and most reads on it, persistently, and the dashboard/REST rebuild button was unreachable (it needs an open chain). The only recovery was deleting *.json.wal by hand. That is a derived artifact taking down the canonical log, which is exactly what the design says must never happen.

Three fixes, one contract:

• Compaction rebases the in-memory WAL chain head onto the persisted snapshot digest.
• The managed append path rebases the live cached sidecar when it schedules a compaction (the last point that still holds &mut self, so the persist-wins read-lock design is untouched).
• A full snapshot write clears any sibling .wal before the atomic rename, so a crash can only leave a stale — rebuildable — snapshot without a WAL, never a newer snapshot paired with an older WAL.

An unreadable sidecar now degrades to a rebuild on open, matching the three other sidecar load sites and the documented contract. Regression tests cover append-after-compact replay, stale-WAL clearing, reopening across two compaction boundaries, and a foreign-generation WAL.

2. mentisdb add was missing thought_type

Both the help text and the struct doc promised --type <type> Thought type (default: fact-learned), but the CLI only sent thought_type when --type was passed. A plain mentisdb add "text" therefore failed with HTTP 422 missing field thought_type, and a typo like --type note produced an opaque server 400.

add now always sends the field, defaulting to fact-learned, and validates an explicit --type locally:

$ mentisdb add "bad" --type note
add failed: Unknown ThoughtType 'note'. Valid types: PreferenceUpdate,
UserTrait, RelationshipUpdate, Finding, Insight, FactLearned, PatternDetected,
Hypothesis, Mistake, Correction, LessonLearned, AssumptionInvalidated,
Constraint, Plan, Subgoal, Decision, StrategyShift, Wonder, Question, Idea,
Experiment, ActionTaken, TaskComplete, Checkpoint, StateSnapshot, Handoff,
Summary, Surprise, Reframe, Goal, LLMExtracted

The outgoing payloads are now built by pure build_add_body / build_ranked_search_body helpers with regression tests, so the CLI and the REST schema cannot drift again silently.

3. One canonical thought-type parser

server.rs, llm.rs, and lib.rs each had their own copy of the ThoughtType string table, and they had drifted: the server rejected Goal and LLMExtracted even though both are valid variants. There is now a single FromStr/Display/as_str/ALL implementation with a typed error that lists the valid options, and all three call sites delegate to it.

Upgrade

cargo install mentisdb --locked --force

Then restart the daemon. Full LoCoMo and LongMemEval benchmarks were not re-run: this release changes no retrieval scoring.

Issues closed: #20 (sidecar WAL digest mismatch) and #17 (CLI missing thought_type).