Continuous learning: encoding model that improves from operational experience

Vision

The encoding model should improve from operational experience. Every encoding it produces, every recall that succeeds or fails, every feedback signal — all of this is training data that currently gets discarded. The model that encodes memory #10,000 should be measurably better than the one that encoded memory #1.

Full design document: docs/DESIGN_continuous_learning.md

Architecture

Four tiers that build on each other:

Tier 1 — Experience Collection (always running, zero cost)

Verification gate scores every encoding (EPR, FR, TE) at write time
Feedback from recall sessions links back to the encodings
Experience buffer accumulates gold + bad encoding pairs

Tier 2 — Curriculum Generation (nightly, during dreaming)

Bad encodings get re-encoded by Gemini to produce corrected versions
Creates (input, bad_local, good_gemini) tuples — DPO training data
Hard example mining focuses on the model's weakest areas

Tier 3 — Model Update (triggered when ≥50 new training pairs)

Short spoke fine-tuning (SFT, later DPO) on new experience
30% replay from base dataset prevents catastrophic forgetting
Quality gate: must pass EXP-25 probes before deployment
Atomic spoke deployment with rollback

Tier 4 — Self-Assessment (metacognition-driven)

Rolling quality window detects encoding drift
Domain shift detection flags new patterns
Autonomous training trigger when quality degrades

What Makes This Special

The feedback signal is downstream and delayed — an encoding is "good" not because it looks right at write time, but because it leads to useful recalls days or weeks later. The model learns what makes memories findable and useful, not just structurally correct.

Key Dependencies

Verification gate (designed in EXP-29, not yet implemented)
EXP-25 probe evaluation framework (exists)
Spoke training pipeline (exists)
Dreaming agent as training orchestrator (exists, needs extension)
Gemini API for corrective re-encoding (available)

Implementation Roadmap

Phase	Work	Estimate
1. Experience Collection	DB migration, verification gate, experience buffer	2-3 days
2. Recall-Encoding Linkage	recall_feedback table, feedback processing	1-2 days
3. Curriculum Generation	Dreaming phase extension, Gemini re-encoding	2-3 days
4. Automated Training	Spoke training integration, quality gate, deployment	3-5 days
5. Metacognition Integration	Quality tracking, domain shift, training triggers	2-3 days
6. DPO (stretch)	Preference optimization from corrective pairs	2-3 days

Total: ~2-3 weeks

Success Criteria

1 month: ≥500 experience buffer entries, ≥2 training cycles, EPR +5pp
3 months: ≥10 training cycles, handles new domains without quality drop
6 months: model adapted to user's patterns, exceeds Gemini on user's actual inputs

Design Review Progress — 2026-04-10

Approach Selected: Phased Automation (Approach C)

Three independently valuable, independently reviewable phases. Each ships as its own PR.

Key Design Decisions

Model-agnostic architecture — The continuous learning pipeline works regardless of which base model wins EXP-29 (Qwen 3.5 2B, 4B, Gemma E2B, or Bespoke-pruned model from EXP-28). Verification gate is pure string matching, experience buffer stores quality metrics about encodings regardless of producer, training trigger is a config value pointing at a checkpoint.
Hybrid orchestration (Option C) — Go metacognition agent decides whether to train. If yes, daemon writes a training request file and exits gracefully. A systemd path unit watches for the file, runs the Python training script, validates quality gate, restarts daemon with new or old spokes. Daemon never restarts itself. Respects Unix philosophy, gives systemd proper supervision, keeps PyTorch in Python where it belongs.
Classification metrics: EPR + recall_score + TED — Fabrication Rate (FR) is monitoring-only, NOT used for gold/needs_improvement classification. EXP-25 showed 25.8% FR on perfect outputs due to legitimate semantic expansion ("WAL mode on." → "database"). FR has >25% false positive rate on correct outputs. Classification thresholds:
- Gold: EPR > 0.9, TED = false, AND (recall_score > 0.8 OR no recalls yet + no flags)
- Needs improvement: EPR < 0.7 OR TED = true OR recall_score < 0.3 (with recall_count >= 3)
- Ambiguous: Everything else
- The recall_count >= 3 guard prevents a single irrelevant rating from condemning an encoding (design doc risk mitigation for feedback noise).
Phase 1 signals: write-time verification + recall linkage — Gemini comparative baseline deferred to Phase B (curriculum generation). Write-time verification (EPR, TED, MIG) gives immediate signal. Recall feedback starts accumulating from day one; takes weeks to become dense but is the real downstream quality signal. Observe before intervening.

Phase A: Runtime Verification & Experience Collection (~1 week)

Pure observation. No model changes, no training, no Gemini calls.

A.1 Runtime Verification Gate

New file: internal/agent/encoding/verification.go

Pipeline insertion point: After compressAndExtractConcepts() returns (agent.go ~line 1046), before embedding generation:

compressAndExtractConcepts() → verifyFaithfulness() → generateEmbedding() → persist()

Checks computed (Go port of eval_faithfulness.py patterns):

Check	Method	Cost
EPR (Entity Preservation Rate)	Regex extraction of proper nouns, numbers, paths, versions from raw input. Fraction found in compression's content-bearing fields.	~0.1ms
TED (Template Echo Detection)	Check output against known instruction phrases from encoding prompt. Boolean.	~0.05ms
MIG (Minimal Input Guard)	If raw < 50 chars and output content > 300 chars → flagged as padded.	~0.01ms
FR (Fabrication Rate)	Extract entities from output, fraction not in input. Monitoring only.	~0.1ms

Behavior: Soft mode only (no rejections). Log warning, store metrics, persist regardless. Reduce salience by 0.1 if TED=true. No hard mode until we have baseline data to calibrate thresholds — observe, then intervene.

A.2 Schema Migration

-- Verification metrics on memories table
ALTER TABLE memories ADD COLUMN encoding_epr REAL DEFAULT NULL;
ALTER TABLE memories ADD COLUMN encoding_fr REAL DEFAULT NULL;
ALTER TABLE memories ADD COLUMN encoding_flags TEXT DEFAULT NULL;

-- Recall-encoding linkage
CREATE TABLE recall_feedback (
    id TEXT PRIMARY KEY,
    query TEXT NOT NULL,
    memory_id TEXT NOT NULL,
    feedback TEXT NOT NULL,  -- helpful, partial, irrelevant
    recall_session_id TEXT,
    created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
    FOREIGN KEY (memory_id) REFERENCES memories(id)
);
CREATE INDEX idx_recall_feedback_memory ON recall_feedback(memory_id);

-- Experience buffer
CREATE TABLE experience_buffer (
    id TEXT PRIMARY KEY,
    raw_id TEXT NOT NULL,
    memory_id TEXT NOT NULL,
    encoding_epr REAL,
    encoding_fr REAL,
    encoding_flags TEXT,
    recall_score REAL,
    recall_count INTEGER DEFAULT 0,
    category TEXT DEFAULT 'ambiguous',
    used_in_training INTEGER DEFAULT 0,
    created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
    updated_at DATETIME DEFAULT CURRENT_TIMESTAMP,
    FOREIGN KEY (memory_id) REFERENCES memories(id)
);
CREATE INDEX idx_experience_buffer_category ON experience_buffer(category);

A.3 Experience Buffer Population

On every encoding: Insert experience_buffer row with EPR, FR, flags. Initial category = ambiguous.
On every recall feedback: Insert recall_feedback row. Update experience_buffer.recall_score as running weighted average (helpful=1.0, partial=0.5, irrelevant=0.0). Increment recall_count.
Reclassification: Runs during dreaming cycle. Applies gold/needs_improvement/ambiguous thresholds. The recall_count >= 3 guard prevents noisy single-rating condemnations.

A.4 Metacognition: Quality Drift Detection

New observation type: encoding_quality_drift. Extends auditMemoryQuality():

Rolling averages over last 100 encodings: mean EPR, TED rate, flag distribution
Compare against baseline (first 100 encodings after Phase A ships)
Emit trend: improving / stable / degrading
If degrading for 2+ consecutive audit cycles → flag for attention (triggers training in Phase C)

A.5 Store Interface Additions

WriteVerificationResult(ctx, memoryID, epr, fr, flags) error
WriteExperienceEntry(ctx, entry) error
UpdateExperienceRecallScore(ctx, memoryID, feedback) error
ReclassifyExperienceBuffer(ctx) (int, error)
ListExperienceByCategory(ctx, category, limit) ([]ExperienceEntry, error)
WriteRecallFeedback(ctx, rf) error
GetRecallHistory(ctx, memoryID) ([]RecallFeedback, error)

Phases B and C design in progress — will be added as subsequent comments.

Phase B: Curriculum Generation (~1 week)

Runs during dreaming hours. Builds the training data that Phase C consumes. Still no model changes.

B.1 Dreaming Agent Extension

New phase: Phase 4.5 in internal/agent/dreaming/agent.go (after noise prune, before cross-project linking).

Trigger conditions:

Experience buffer has >= 10 entries categorized as needs_improvement
At least 24 hours since last curriculum generation run
Gemini API is configured and reachable (graceful skip if not)

Algorithm per entry (up to 50/run):

Load original raw input from raw_memories
Send to Gemini API with build_production_prompt() — identical prompt the local model saw
Run verification (EPR, FR, TED) on Gemini's output
If Gemini passes → store as corrected_output on experience_buffer entry (training pair created)
If Gemini also fails → reclassify entry as "ambiguous", log for manual review

B.2 Schema Addition

ALTER TABLE experience_buffer ADD COLUMN corrected_output TEXT DEFAULT NULL;
ALTER TABLE experience_buffer ADD COLUMN corrected_epr REAL DEFAULT NULL;
ALTER TABLE experience_buffer ADD COLUMN corrected_fr REAL DEFAULT NULL;
ALTER TABLE experience_buffer ADD COLUMN correction_source TEXT DEFAULT NULL;  -- 'gemini', 'manual'
ALTER TABLE experience_buffer ADD COLUMN corrected_at DATETIME DEFAULT NULL;

B.3 Hard Example Mining

After curriculum generation, analyze the experience buffer for systematic failure patterns:

Detection heuristics:

Dense numbers: raw input contains >= 5 distinct numeric values AND EPR < 0.8
Short input: raw input < 30 chars AND any flag set
Template echo: TED = true (direct detection)
Domain gap: cluster needs_improvement by concept overlap — if 5+ share concepts not in training data, flag as new domain

Output: curriculum_report stored as metacognition observation. Tells Phase C what kind of training data to prioritize — targeted improvement, not random sampling.

B.4 Gemini API Integration

Uses existing llm.Provider interface — daemon already has Gemini configured. Same ChatCompletion() method, production encoding prompt, temperature 0.3.

Rate limiting: Max 50 corrections per dreaming cycle. ~$0.01/night at Gemini 2.5 Flash pricing.

continuous_learning:
  curriculum:
    enabled: false              # opt-in
    max_corrections_per_cycle: 50
    min_needs_improvement: 10
    cooldown_hours: 24

B.5 Training Data Assembly

Assembled data written to training/data/continuous_learning/ with timestamped filenames. This is the Go→Python handoff point.

SFT pairs (gold examples):

{"input": "<raw input>", "output": "<local encoding rated gold>"}

Corrective SFT pairs (Gemini re-encodings):

{"input": "<raw input>", "output": "<gemini corrected encoding>"}

Future DPO triples (stretch goal):

{"input": "<raw input>", "chosen": "<gemini encoding>", "rejected": "<local bad encoding>"}

Phase C: Automated Training & Deployment (~1-2 weeks)

Closes the loop. Metacognition decides when to train, systemd orchestrates it, Python runs PyTorch, quality gate validates.

C.1 Training Request Protocol

When metacognition detects degrading quality trend (or manual MCP trigger), the daemon:

Assembles training manifest (JSON):
- Request ID, timestamp, trigger reason
- Experience buffer stats (gold/needs_improvement/corrected counts)
- Current model info (base, spokes checkpoint, GGUF path, baseline metrics)
- Training data path
Writes to ~/.local/share/mnemonic/training_requests/pending.json
Exits gracefully (clean shutdown, releases VRAM)

C.2 Systemd Integration

Three new systemd units:

mnemonic-train.path — watches for training requests:

[Path]
PathExists=%h/.local/share/mnemonic/training_requests/pending.json
Unit=mnemonic-train.service

mnemonic-train.service — runs training pipeline:

[Service]
Type=oneshot
ExecStart=%h/Projects/mem/training/scripts/continuous_train.sh
TimeoutStartSec=1800    # 30 min max

continuous_train.sh — orchestrator:

#!/bin/bash
set -euo pipefail
# 1. Stop daemon (free VRAM)
systemctl --user stop mnemonic
# 2. Run training
python3 training/scripts/continuous_train.py --request "$REQUEST" --result "$RESULT"
# 3. Archive request
mv "$REQUEST" "${REQUEST%.json}.$(date +%Y%m%d%H%M%S).completed.json"
# 4. Restart daemon (new spokes if passed, old if not)
systemctl --user start mnemonic

C.3 Training Script (`continuous_train.py`)

Training data mix per batch:

30% replay from v7 base dataset (prevents catastrophic forgetting)
50% gold examples from experience buffer
20% corrective pairs from Gemini re-encodings
Fixed seed for reproducibility

Key parameters:

LR: 1e-4 (conservative for incremental updates)
max_steps: 500 (2-8 epochs on 50-200 examples)
batch_size: 1, grad_accum: 8
gradient_checkpointing: true
Reuses train_qwen_spokes.py infrastructure (proven in EXP-18/20/25)

C.4 Quality Gate

Reuses existing eval infrastructure. Runs after training completes:

Export temporary spokes to GGUF via export_qwen35_spokes.py
Start llama-server with temporary GGUF
Run eval_faithfulness.py on 25 gold-standard probes
Run stress_test_hallucination.py (7 tests)
Compare against baseline stored in training request

Pass criteria:

Metric	Threshold	Rationale
EPR	>= 90%	EXP-25 target
SC	>= 95%	Allow 1/25 edge case
Stress test	>= 6/7	Current production baseline
No metric regression	<= 5pp	Monotonic improvement enforced

FR is NOT a gate criterion — monitoring only, consistent with classification decision.

On pass: Export final GGUF, quantize to RQ4, update config.yaml, daemon restarts with new spokes.
On fail: Discard, log failure, daemon restarts with old spokes unchanged.

C.5 Spoke Deployment

Version-suffixed GGUFs: qwen35-2b-spokes-cl001-f16.gguf
Quantized via rotorq_quantize_gguf.py (BetaQ RQ4)
Config.yaml updated to point to new GGUF
Previous 3 versions retained for rollback
Daemon loads new spokes on restart automatically

C.6 Manual Training Trigger

New MCP tool: trigger_training(reason: string) -> {request_id, status}

Allows manual trigger without waiting for quality drift. Same flow as automated trigger. Useful during development and pipeline testing.

C.7 Full Config Block

continuous_learning:
  enabled: false                    # opt-in, off by default
  training:
    min_new_examples: 50
    max_examples_per_run: 200
    replay_ratio: 0.30
    lr: 1.0e-4
    max_steps: 500
    seed: 42
    quality_gate:
      min_epr: 0.90
      max_fr: 0.10
      min_sc: 0.95
      min_stress: 6
      max_regression_pp: 5
    rollback_versions: 3
  curriculum:
    enabled: false
    max_corrections_per_cycle: 50
    min_needs_improvement: 10
    cooldown_hours: 24
  trigger:
    auto: false
    manual: true
    training_window: "02:00-06:00"

Summary: What Each Phase Delivers

Phase	Ships	Value
A	Verification gate, experience buffer, recall linkage, drift detection	Encoding quality observability. Every encoding scored. Baseline established.
B	Curriculum generation, hard example mining, training data assembly	Training data curated automatically. Failure patterns identified.
C	Automated training, quality gate, spoke deployment, rollback	Self-improving model. Closed loop from experience → training → deployment.

Each phase is independently valuable and ships as its own PR.

Implementation Spec Complete

Spec committed: docs/SPEC_continuous_learning.md at 7df03b3 on feat/exp29-candidate-eval

Key additions beyond the original design doc:

Detailed Go types (VerificationResult, ExperienceEntry, ExperienceStats, RecallFeedback, EncodingQualityWindow)
Complete store interface additions with method signatures
Entity extraction regex patterns (ported from eval_faithfulness.py)
Template echo phrase list
Systemd unit files with trap-based failure recovery (daemon always restarts)
Training script pseudostructure with hyperparameter justification table
Testing strategy: unit tests, parity test (Go vs Python), integration tests
Full config block with defaults and comments
Risk/mitigation matrix with phase assignments
Dependency table with current status

Next: Writing detailed implementation plan (step-by-step, file-by-file).

Phase A Implementation Plan Complete

Plan committed: docs/PLAN_continuous_learning_phase_a.md at 5aa702d

12 Tasks, TDD throughout:

Task	What	Files
1	Schema migration (007)	`migrations/007_continuous_learning.sql`, `schema.go`
2	Store interface — ContinuousLearningStore	`store.go` (types + interface)
3	SQLite implementation	`sqlite/continuous_learning.go`
4	Verification gate (EPR, TED, FR, MIG)	`encoding/verification.go`
5	Verification tests	`encoding/verification_test.go`
6	Wire into encoding pipeline	`encoding/agent.go`
7	Hook feedback into experience buffer	`mcp/server.go`
8	Dreaming reclassification phase	`dreaming/agent.go`
9	Metacognition quality drift	`metacognition/agent.go`
10	Configuration	`config/config.go`, `config.yaml`
11	Build, test, deploy to daemon	Integration validation
12	Go/Python parity test	Verify EPR matches eval_faithfulness.py

Each task has exact file paths, complete code blocks, test commands, and commit messages. Designed for subagent execution.

Phase B and C plans will be written after Phase A ships and we have baseline quality data.

docs: implementation spec for continuous learning pipeline (#391)

docs: Phase A implementation plan for continuous learning (#391)

Phase A Progress — Tasks 1-10 Complete

Runtime verification and experience collection infrastructure is implemented and passing all tests.

What shipped:

Schema migration 007: encoding_epr/encoding_fr/encoding_flags columns on memories, plus recall_feedback and experience_buffer tables
Store interface: ContinuousLearningStore with 9 methods (verification results, experience CRUD, recall linkage, quality window)
SQLite implementation: Full implementation with reclassification logic
Verification gate: Go port of eval_faithfulness.py entity extraction (numbers, paths, versions, proper nouns, CamelCase, @mentions) + EPR/FR/TED/MIG checks. Wired into encoding pipeline — every encoding now runs through verifyFaithfulness() after compression
MCP feedback hookup: feedback tool now propagates to experience buffer (running weighted average) and records recall-encoding linkage
Dreaming reclassification: Phase 4.5 reclassifies experience buffer entries (gold/needs_improvement/ambiguous) during dreaming cycles
Metacognition drift detection: New encoding_quality_drift observation type — warnings at EPR < 0.85 or TED > 5%, critical at EPR < 0.70 or TED > 15%
Config: ContinuousLearningConfig with training/trigger settings, disabled by default

Test status:

11 new unit tests (entity extraction, EPR, TED, MIG)
19/20 test packages pass (1 pre-existing failure unrelated to this work)
golangci-lint run: 0 issues

Remaining:

Task 12: Go/Python parity test on EXP-25 gold probes (optional validation)
Phases B and C (Gemini re-encoding, automated training) are future work

feat: continuous learning Phase A — runtime verification & experience…

test: Go/Python verification parity test (#391)

Phase A Complete — All 12 Tasks Done

Update to previous comment: Tasks 11-12 are now also complete.

Task 11 (build/test/lint): 20/20 test packages pass, 0 golangci-lint issues
Task 12 (Go/Python parity test): 25/25 gold probes within tolerance. Go entity extraction is slightly more conservative on dense-number inputs (24/25 within 15%, 25/25 within 20%)
Migration fix: Inline migration in InitSchema (migration SQL files are reference only). Tables verified live — first encoding scored EPR 1.0
Gemma chat template: Added chat_template config field to embedded provider ("chatml" for Qwen, "gemma" for Gemma 4 E2B). Full inference chain ready for Gemma spoke deployment

Verification gate is live in production — every encoding through the daemon now gets EPR/FR/TED/MIG scoring with results persisted to the experience buffer.

Phase A observation period begins now. Phases B and C depend on data collected during this period.

EXP-30 Inference Pipeline — Status Update

Training: Confirmed working. Python HF generate() with EXP-30 spokes produces valid faithful JSON (100% in testing) on both NF4 and bf16 base. Best eval loss 1.2002 (PPL 3.3).

llama.cpp deployment: Failed. Two issues found:

GGUF export bug (fixed): export_gemma4_spokes.py converted gemma4.attention.sliding_window_pattern from arr[BOOL] to arr[INT32], corrupting SWA/global attention layer assignments. Fix committed on feat/gemma-e2b-spokes.
Gemma 4 generation via llama.cpp (unresolved): Even with fixed GGUF, the base Gemma 4 E2B model doesn't generate correctly through llama.cpp's /completion endpoint with Gemma token IDs — prompt echoing, not generation. The /v1/chat/completions endpoint with ChatML fallback produces partial JSON but with hallucinations and wrong template format. GBNF grammar enforces structure but can't fix degenerate content. This is a Gemma 4 + llama.cpp compatibility issue, not spoke-specific.

Decision: Stop fighting llama.cpp for Gemma 4 spoke inference. Explore bespoke inference path — either a minimal Python serve script (proven working) or a custom Go/C engine using ggml primitives directly. The spokes are good; the inference runtime is the bottleneck.

Next: Test Python serve as option 1, then evaluate building a bespoke inference engine for production deployment with hot-swappable spokes.

EXP-30 Systematic Characterization — Verdict Corrected to PARTIAL

Previous assessment was overstated

Session 4 concluded spokes "work" based on a few manual tests with Python HF generate() producing parseable JSON. Session 5 ran a systematic characterization (10/25 gold probes via serve_gemma_spokes.py, plus manual diagnostic probes) and the picture is different.

Characterization results (session 5)

Schema compliance: 0% of probes pass all checks. Every response has structural issues:

Failure mode	Frequency	Description
`concepts` wrong type	Every response	Generated as `dict` (object with sub-keys) instead of `list[str]`
`summary` missing	Most responses	Skipped in field order, swallowed into wrong nesting
`structured_concepts` mixed types	Every response	Arrays mix objects with bare strings
JSON truncated	~20%	Unclosed brackets on longer outputs

Content faithfulness: Confirmed. Entity preservation, file paths verbatim, correct factual content. The model knows what to say but can't maintain how to say it.

Root cause analysis

Training data is clean — audited all 5,880 targets: 100% correct field order, types, structure. Zero inconsistencies.
System prompt mismatch ruled out — A/B tested training format (no system) vs daemon format (with system). Both produce the same violations.
PPL 3.3 is too high — ~3.3 choices of uncertainty per token. For deterministic structural tokens ("summary":, [), this uncertainty allows the base model's JSON priors to override spoke training.
Grammar enforcement fails — outlines 1.2.12 Pydantic schema enforcement produces degenerate output (model's token distribution fights the grammar).

Corrected verdict

PARTIAL — content faithfulness CONFIRMED, structural compliance REFUTED. EXP-30's useful training only occurred in phase 2 (steps 1800-4800, LR ~3e-5). The peak LR phase was counterproductive.

gemma.cpp: not viable

Investigated as alternative inference engine. Three hard blockers: no Gemma 4 support (ISWA/GDN/PLE unimplemented), no GPU (CPU-only), custom weight format (no GGUF).

Next: EXP-31 registered

Constant LR 3e-5 (no cosine, no warmup) with extended training. Karpathy overfit test first (10 examples, 500 steps) to verify spoke capacity before full run. Evaluation via characterize_serve_output.py on all 25 gold probes.

Root Cause Found: All Gemma 4 Spoke Training Was Broken

Spent 2 days chasing Gemma 4 spoke training failures (0/10 schema compliance, PPL stuck at 3.3). Found the root cause: HF's gradient_checkpointing_enable() forces use_cache=False, which breaks Gemma 4's ISWA attention.

The Bug

Gemma 4 has KV sharing layers that reuse key/value states from earlier layers via past_key_values.shared_layers. When past_key_values=None (forced by use_cache=False), these layers fall back to value_states = key_states, producing garbage attention output (2% accuracy, PPL 2.7M vs 68.6% accuracy with cache present).

Every training run since gradient checkpointing was "fixed" was training on corrupted forward pass output. The LR analysis, cosine regression, and schema compliance failures in EXP-30 were all artifacts of training on garbage.

The Fix

TrainingCache — wraps DynamicCache with idempotent update() so checkpoint recomputation doesn't double KV entries
SpokeWrappedLayer owns gradient checkpointing — bypasses HF's gradient_checkpointing_enable() entirely, preserves use_cache=True
bf16 training (not NF4) per earlier decision

Validation

Overfit test (10 examples): loss 1.86 → 0.0096 (PPL 1.0) in 1000 steps
Inference produces valid JSON with all 10 schema fields
Base model already gets 68.6% accuracy — spokes only need to fix the remaining ~31%

What This Means for #391

The spoke training pipeline works. EXP-31 (full training run on 5,238 examples) is ready to go. Once we have a working Gemma spoke model, the continuous learning pipeline from this issue can build on top of it.

Branch: feat/gemma-e2b-spokes

EXP-31 Complete: 25/25 Gold Probes Pass

Full training run finished. Results:

Eval loss: 0.5217 (PPL 1.7), down from 1.82 baseline
Training: 11,400 steps (early stopped), 17.1 hours on RX 7800 XT
Schema compliance: 25/25 gold probes pass (100% JSON valid, 100% field presence, 100% type correctness, 100% enum compliance)
48 consecutive new bests during training, zero regressions

This matches Qwen 3.5 2B spoke quality (100% schema, 7/7 stress test). The only gap is inference speed — 17 tok/s via HF generate vs Qwen's 95 tok/s via llama.cpp. Inference engine work is the next blocker for Gemma production deployment.

Checkpoint: checkpoints/exp31_gemma4_fixed/best_spokes.pt
Branch: feat/gemma-e2b-spokes
WandB: exp31_gemma4_fixed

EXP-31 Fully Validated

All tests complete:

Schema compliance: 25/25 gold probes (100%)
Stress test: 4/7 pass, 0 hallucination failures (3 truncations from token limit on complex inputs)
Daemon integration: Spoke routing works end-to-end — encoding agent routes to Gemma spoke server, confirmed by failure when server stopped
Dep update: transformers 5.5.0→5.5.3 includes the upstream fix for the KV sharing bug ([gemma4] Dissociate kv states sharing from the Cache huggingface/transformers#45312)

PR: #400

Next blocker for this issue: inference speed. Currently 17 tok/s via HF generate (bf16, ~11GB VRAM). Need ~100 tok/s at <3GB VRAM for production deployment. Exploring quantization + optimized inference paths.

Inference Speed Benchmarks (2026-04-13)

Exported EXP-31 spokes to GGUF and tested through llama.cpp on RX 7800 XT:

Quantization	VRAM	tok/s	Status
HF generate (bf16)	~11GB	17.0	baseline
Q4_K_M (llama.cpp)	3.5GB	79.5	production ready
BetaQ RQ4 (f16 PLE)	8.3GB	98.6	too much VRAM
BetaQ RQ4 (all quant)	—	—	crash (CalebisGross/betaq#2)

Q4_K_M at 79.5 tok/s / 3.5GB VRAM is ready to deploy as the embedded provider. BetaQ hybrid quantization (RQ4 weights + Q8_0 embeddings) would hit 100+ tok/s at ~3GB but needs a fix for the get_rows kernel.

Model Prerequisite Satisfied (2026-04-13)

The continuous learning pipeline's key dependency — a production-quality local encoding model — is now in place:

Model: Gemma 4 E2B spokes (EXP-31), BetaQ all-RQ4 quantization
Performance: 101 tok/s at 3.0 GB VRAM on RX 7800 XT
Quality: 25/25 gold probe schema compliance (100%)
Deployment: Running embedded in the mnemonic daemon via in-process llama.cpp with ROCm GPU acceleration. Zero cloud dependency.
PR: docs: add llama.cpp inference benchmarks to EXP-31 #402 (merged)

Phase A (experience collection) is already collecting data against this model. The verification gate, experience buffer, and recall-feedback linkage are all operational.

Next: Phase B (curriculum generation) can proceed — the local model produces encodings that the verification gate scores, and Gemini API provides corrective re-encodings for the DPO training pairs. Phase C (automated training) can use the existing spoke training pipeline (training/scripts/train_spokes.py --model-type gemma) with the accumulated experience data.

The continuous_learning.enabled: false config flag gates activation. All infrastructure is in place to flip it on once Phase B is implemented.

feat: continuous learning Phase B — curriculum generation (#391)

feat: training data assembly for continuous learning (#391)

feat: continuous learning Phase C — training trigger & orchestration (#…

refactor: adapt training data format for Gemma prep pipeline (#391)

docs: systemd training orchestration design (#391)

refactor: systemd-orchestrated training to prevent GPU crashes (#391)

feat: enable continuous learning by default (#391)

Merge pull request #406 from AppSprout-dev/feat/continuous-learning-p…

fix: training crash loop safety + episode synthesis quality (#391)

Continuous learning: encoding model that improves from operational experience #391

Description

Vision

Architecture

What Makes This Special

Key Dependencies

Implementation Roadmap

Success Criteria

Activity

CalebisGross commented on Apr 10, 2026

Design Review Progress — 2026-04-10

Approach Selected: Phased Automation (Approach C)

Key Design Decisions

Phase A: Runtime Verification & Experience Collection (~1 week)

A.1 Runtime Verification Gate

A.2 Schema Migration

A.3 Experience Buffer Population

A.4 Metacognition: Quality Drift Detection

A.5 Store Interface Additions

CalebisGross commented on Apr 10, 2026

Phase B: Curriculum Generation (~1 week)

B.1 Dreaming Agent Extension

B.2 Schema Addition

B.3 Hard Example Mining

B.4 Gemini API Integration

B.5 Training Data Assembly

Phase C: Automated Training & Deployment (~1-2 weeks)

C.1 Training Request Protocol

C.2 Systemd Integration

C.3 Training Script (continuous_train.py)

C.4 Quality Gate

C.5 Spoke Deployment

C.6 Manual Training Trigger

C.7 Full Config Block

Summary: What Each Phase Delivers

CalebisGross commented on Apr 10, 2026

Implementation Spec Complete

CalebisGross commented on Apr 10, 2026

Phase A Implementation Plan Complete

12 Tasks, TDD throughout:

CalebisGross commented on Apr 10, 2026

Phase A Progress — Tasks 1-10 Complete

What shipped:

Test status:

Remaining:

CalebisGross commented on Apr 10, 2026

Phase A Complete — All 12 Tasks Done

CalebisGross commented on Apr 11, 2026

EXP-30 Inference Pipeline — Status Update

CalebisGross commented on Apr 11, 2026

EXP-30 Systematic Characterization — Verdict Corrected to PARTIAL

Previous assessment was overstated

Characterization results (session 5)

Root cause analysis

Corrected verdict

gemma.cpp: not viable

Next: EXP-31 registered

CalebisGross commented on Apr 12, 2026

Root Cause Found: All Gemma 4 Spoke Training Was Broken

The Bug

The Fix

Validation

What This Means for #391

CalebisGross commented on Apr 13, 2026

EXP-31 Complete: 25/25 Gold Probes Pass

CalebisGross commented on Apr 13, 2026

EXP-31 Fully Validated

CalebisGross commented on Apr 13, 2026

Inference Speed Benchmarks (2026-04-13)

CalebisGross commented on Apr 13, 2026

Model Prerequisite Satisfied (2026-04-13)

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

C.3 Training Script (`continuous_train.py`)