Repository navigation
Description
Delivery child of #193. Runtime stage A implemented and measured; PR #210 awaits independent audit and merge. The persistence foundation from #196/#201 is merged in PR #206, merge commit c6bd882f550f80c83a87f82eb4b0fedd81eb493c. Start from current origin/main containing that commit. This issue update performs no model acquisition or runtime deployment.
Current delivery status — 2026-09-27
- Persistence foundation: Curation persistence slice 1: default configuration, manual compositions and publication authority (#196, #201) #206 is merged. This supplies configuration/profile identities and manual composition/publication records, not executed personalities.
- Runtime stage A (roadmap item 2 below): implementation and bounded live proof are complete in open PR #210, head
cb2e236917c67286dc13c67c5ab2258bb87d9feb. All 32 planned calls are retained in the committed results artifact. All nine CodeRabbit threads are resolved, and CodeRabbit/GitGuardian checks pass. The implementer reportsmix precommit: 19 doctests, 2,335 tests, zero failures. Independent audit and merge are outstanding; this is not final acceptance or production readiness. - Model outcome: 9B is a conditional development baseline; 4B abstained on all tested calls under the current instruction. Fix and version the duplicate-selection/satire instructions, address fixture confounds and re-benchmark before Persona panel pilot: five versioned editors, fixed quorum and ten-page evaluation #197 depends on them. Repeated fixed-seed outputs are timing samples, not independent quality judgments.
- Operational checkpoint: model weights and runtime files are external; the model endpoint is stopped, benchmark code/results are committed and two isolated evidence databases remain for audit. Do not discard those databases or perform a new heavyweight run without verified capacity. Internal free space was approximately 4.6 GiB at the status check.
- Storage: Move dictionary storage, PostgreSQL and agent workspaces to the external 2 TB drive — execution in #211 #209 owns the full dictionary relocation. Its latest owner schedule is after separate routing Stage 2 acceptance. This curation status update does not override that schedule. Routing Stage 2 under Routing before launch: On overviews, standards-informed namespaces, and stable subject URLs #194 is not the same as curation roadmap item 2 and is not complete. Safe read-only review and migration inventory may proceed; insufficient capacity still blocks heavy runs.
- Still ahead: sourced personality dossiers and genuine panel decisions/publication (Persona panel pilot: five versioned editors, fixed quorum and ten-page evaluation #197), visit-driven refresh (Visit-driven curation refresh: coalesced jobs, fenced leases and atomic publication #198), page bindings (Routing before launch: On overviews, standards-informed namespaces, and stable subject URLs #194), later Mini measurement (Local curation portability: measured Mac Studio to Mac Mini handoff #199), and post-MVP public dossiers/history (Curation transparency: equal votes, reproducible selection, profiles and version history #203/Editorial method: declared perspectives, checkable facts and accountable curation #204). No running panel or finished public curation is claimed.
Runtime stage A scope and acceptance contract
Deliver one reviewable PR providing a private local inference adapter, explicit model provenance, safe admission/accounting and a repeatable hardware benchmark. Success means an operator can submit one bounded, frozen evidence packet and receive a validated result or a precise refusal, with durable attempt/accounting evidence. The UI and shared database are not prerequisites for exercising this contract.
This slice does not implement personality dossiers, panel ballots/decisions, automatic publication, page bindings, refresh-on-visit scheduling, model training or fine-tuning. Those remain #197, #194 and #198. Five personalities will share one loaded generative model; they are versioned editorial inputs, not five model installations. An unresolved benchmark is reported as unresolved, never converted into a claim that the panel works.
Product contract inherited from the merged foundation
Routine composition publication will use a genuine finalized panel decision, configured consensus and evidence/policy checks, with no per-selection human sign-off. The optional operator path handles manual compositions, overrides/corrections and withdrawal. Semantic illustrates claims retain their separate review policy under #190; consensus never accepts a claim. Configuration/profile administration is separate from routine publication. Stage A creates neither publication authority nor pretend panel records, and does not activate the manual-only default as panel-ready.
Concrete deliverables and integration boundaries
- Read the merged implementation first. Inspect
docs/curation/persistence-slice-1.md, curation schemas/services and existing discovery/Oban/Req test patterns. Reuse existing ownership and eligibility rules. Add only the model configuration and runtime attempt/admission records actually needed here; generate additive migrations throughmix ecto.gen.migration. Never rewrite applied migrations or add fake run/participant IDs to satisfy future relationships. Operational endpoint configuration stays separate from immutable model/template/policy provenance. - Preflight and explicit setup. Inventory actual chip, unified memory, OS, free disk, current Ollama/version/models and competing workloads. Verify
/Volumes/LLM Modelsis a mounted external volume before creating anything below it. Reuse existing compatible artifacts; any setup/download must be an explicit operator action, never readiness, request handling or retry behavior. Record resolved artifact digests and license. For the recorded proof, code/PostgreSQL remained internal. The owner’s later Move dictionary storage, PostgreSQL and agent workspaces to the external 2 TB drive — execution in #211 #209 policy moves dictionary source/databases and bulk artifacts externally at its coordinated checkpoint; do not relocate active resources under this runtime task or perpetuate the old preference as the future default. Document reproducible private service start/stop/restart and restore any temporary benchmark settings. - Narrow Req adapter. Accept immutable packet/model configuration identity and a bounded output contract. Use nonstreaming structured JSON; validate input size, output size/shape, known candidate/meaning/revision IDs, exact quotations and current evidence/rights eligibility on the server. Treat excerpt instructions as untrusted data. Disable automatic HTTP generation retries; only the confirmed pre-dispatch retry cases below are eligible. Do not invent source evidence, invoke model-proposed tools, or keep raw reasoning/prompt dumps in durable logs.
- Global admission and durable attempts. All callers sharing the same physical service use one authoritative inference slot, including different worker nodes and test configurations. Persist idempotency/request identity, owner/fence, lifecycle transitions and budget reservation before dispatch. Distinguish pre-dispatch refusal from an uncertain dispatch and a completed result. Client timeout, caller death, expired lease or midnight cannot alone prove generation stopped or free the slot. If the pinned runtime cannot positively reconcile an attempt, quarantine the service until controlled stop/restart establishes that old generation ended. Reject late or duplicate results by attempt/fence and settle accounting once. This is runtime infrastructure; full panel run/ballot persistence remains Persona panel pilot: five versioned editors, fixed quorum and ten-page evaluation #197/Curation persistence: versioned panels, ballots and reviewed page compositions #196.
- Runnable harness and bounded benchmark. Provide a documented command that accepts a frozen permitted packet, checks readiness and records a validated result without changing public compositions or claims. Compare pinned 4B/9B candidates on identical text packets with a generic editor instruction. Include a useful ordinary case, an ambiguous-meaning case, a sparse/abstain case and a fictional adverse/injection case. Explicitly separate this adapter smoke evaluation from Persona panel pilot: five versioned editors, fixed quorum and ten-page evaluation #197's sealed ten-page panel experiment. State repetition counts before measurement, obey the daily budget, and report unfinished measurements honestly.
- Review evidence. Commit a concise report naming real hardware/artifacts/settings, raw permitted timing/counter summaries, cold/warm sample counts and quantiles, memory/swap measurements or explicit unavailability, validation failures and the go/no-go decision. Model size/download size is not a memory benchmark. Recommend the smallest candidate meeting the recorded criteria, or explain why neither did. Include deterministic tests, migration rehearsal where applicable,
mix precommitresults and exact final commit in the PR handoff. Leave it for CodeRabbit and independent audit; do not merge/deploy as part of this implementation slice.
Failure-focused acceptance for stage A
| Scenario | Required observable result |
|---|---|
| Service, external mount or artifact missing; served digest changes | Readiness refuses with a stable reason; no download, public write or blocked page read |
| Oversized packet/result, malformed JSON, unknown IDs, wrong meaning/revision, fabricated quote, restricted evidence | Explicit validated refusal; no accepted result or retry loop |
| Two independent callers/nodes target the physical runtime | At most one physical generation; durable reservation enforces this outside a single BEAM process |
| Timeout after dispatch, caller crash, lease expiry or uncertain completion | Slot remains unavailable until reconciliation; no duplicate generation or budget refund on guesswork |
| Duplicate request, stale fence or late response | Same logical receipt or conflict; no second accepted write/accounting settlement |
| UTC budget rollover or concurrent reservations | One global occupancy slot, bounded daily reservations, no budget reset loophole |
| Unsupported thinking controls or missing usage metrics | Detect/document capabilities; enforce the bounded supported behavior; missing counters remain unknown |
| Hardware/model unavailable for live benchmark | Deterministic implementation continues; report actual blocker and incomplete measurements, without claiming runtime/panel readiness |
Use Req.Test/controlled local fakes for deterministic tests, plus committed-transaction or independent-process checks for coordination and accounting races. A single-process mutex test is insufficient evidence for the shared slot. Never call external inference from the ordinary test suite. Run all database work against an explicitly isolated test partition; shared development/production migrations and source-data mutations are outside this implementation task.
Full runtime contract (stage A and subsequent panel integration)
Runtime contract
Use an existing pretrained quantized generative model served persistently through Ollama on the available Mac Studio. Confirm chip, macOS, unified memory (previously reported 64 GB), free disk and competing processes before the spike. Phoenix/Oban/PostgreSQL own work and history; Req calls the private local HTTP API. Persona configuration and model endpoint configuration are separate. No Python subprocess per generation, training job, or permanently running process per persona.
Start by comparing pinned Qwen3.5 4B and 9B artifacts on the same text evidence packets; the catalog lists 3.4 GB and 6.6 GB downloads as checked 2026-09-26. These are not resident-memory or quality measurements. Larger models are optional follow-up experiments after a measured failure of the small baseline. Do not assume sentiment classifiers can generate curatorial judgments.
The Ollama structured-output API accepts a JSON schema in /api/chat's format. Use nonstreaming responses for these short structured jobs. Validate schema, allowed IDs, exact source/revision references, quote fidelity and rights again in Elixir; syntactically valid JSON is not verified evidence. No requirement for Bumblebee/Nx to make an HTTP call. Native model dependencies belong to separately benchmarked supported assessor/embedding work in #101; Bumblebee support is architecture-specific.
Acquisition, storage and startup
- Implementation records repository/tag, resolved weight manifest/blob digests, quantization, license, runtime version, chat template, context/output settings and sampling parameters. Immutable model-config versions; a mutable tag cannot silently change an existing run. Verify the served digest before dispatch.
- Follow Move dictionary storage, PostgreSQL and agent workspaces to the external 2 TB drive — execution in #211 #209 for the coordinated external relocation of dictionary source/PostgreSQL; preserve current active paths until that cutover. Recommended explicit model-cache root:
/Volumes/LLM Models/dictionary/ollama, configured withOLLAMA_MODELS. Preflight the mounted volume before creating directories or starting acquisition; absent drive means unavailable, not a fallback download onto the internal mount-point directory. An intentional internal model-cache location can be configured later, never silently selected. - Manage one persistent local service with a documented start/stop/restart procedure. Bind loopback when colocated; never expose unauthenticated inference. Explicit readiness checks verify service version, installed model digest and a bounded structured-output smoke call before admitting work.
- Warm-load once at the beginning of the operating window, with a bounded
keep_alive; distinguish cold load from warm inference timings. An external-drive disconnect or missing artifact makes readiness fail. Do not download implicitly during page requests or worker retries. - The Ollama FAQ documents model storage, warm loading and parallelism controls. Set one loaded model and one parallel request as defense in depth, and enforce a single global inference slot in the application/service gateway, shared by every persona and worker/node. Per-node Oban concurrency alone is insufficient. A timed-out HTTP caller must not free the slot until generation cancellation/completion is confirmed; fence stale owners and quarantine an uncertain service rather than overlap work.
Initial operating limits (tunable, not benchmarks)
One text-only packet contains at most 12 candidates and selected sense revisions. Start at 8K total context (test 4K if adequate), reserve output room, trim by deterministic candidate/excerpt rules and reject oversized packets rather than silently omit the target meaning. Cap generated output including thinking at 1,024 tokens per call; explicitly disable thinking where the pinned model/runtime supports it, otherwise count/cap it and verify behavior in the spike. Per-call deadline 120 seconds, per-panel wall deadline 20 minutes.
The five-person panel has ten logical generation calls maximum (two rounds each). At most one schema-repair generation per whole panel is allowed: up to 11 physical generation requests, all charged to the ledger. Transport retries: at most two per logical request only for confirmed pre-dispatch connection failures; uncertain dispatch/completion is reconciled via the gateway attempt ID before another generation. No unlimited model retries.
Begin with a 30-minute global inference occupancy budget per day (UTC accounting), a 30-unit pending-work cap, and feature-flagged operator-initiated runs. One unfinished/held composition and each separate pending claim count as work units; this is an admission limit, not a requirement for human review of routine compositions. During panel integration reserve budget and up to four units (one composition plus at most three linked claims) transactionally before admitting a packet; release unused/completed composition reservations and retain pending claim reservations until resolved. Stage A reserves its bounded runtime work without fabricating compositions or claims. Occupancy, failed/uncertain attempts, readiness smoke calls and repair calls all consume the same global budget; record cold-load overhead separately and never exclude it from occupied time. Pause on memory pressure/swap growth, repeated failures or exhausted limits; provide explicit operator resume. Provider quotas remain separate and continue through discovery.
Record request/attempt ID, run/participant/round, model digest, input hash, cold/warm latency, prompt/output/thinking tokens where available, load time, peak memory, swap delta, timeout/schema/fact-validation failures and budget consumed. Never treat unreported thinking tokens as zero. Failure/offline status leaves jobs queued/deferred and existing eligible pages readable; no silent hosted fallback.
Reuse lessons, not the wrong model stack
The predecessor model registry, Elixir client, and Python classifier use configurable DistilBERT/Twitter-RoBERTa/FinBERT sentiment models and NER, return per-model JSON, and calculate consensus in Elixir. Reuse configuration and per-result observability ideas, not shell invocation or classifiers as generative curators.
Also inspected locally in Argus at db4cbe006c823a69096ca82fa24084db531ad063: /Users/holden/Code/projects-2026/argus/lib/argus/research/artifact_store.ex (immutable content-addressed artifacts and verified reads), registry.ex (versioned scorer allowlist), runner.ex (input/model lineage). These local files were verified; this commit was not available from the GitHub API, so no unverified GitHub links are asserted. Copying code is not required.
Acceptance
- Publish repeatable cold/warm measurements for 4B and 9B on the Studio; record actual hardware, versions, quality failures, p50/p95 latency, memory/swap and validated useful selections per machine-hour and separately measured evaluation effort.
- Missing drive/model/service fails readiness without downloading or blocking the reader; restarting resumes bounded work.
- Two worker nodes cannot overlap physical inference; timed-out and late attempts cannot double-write results or exceed reservations.
- Malformed JSON, fabricated IDs/quotes, prompt injection in source excerpts, unavailable model and uncertain transport completion are covered by deterministic adapter tests.
- Document runtime setup and protocol for Local curation portability: measured Mac Studio to Mac Mini handoff #199; run
mix precommiton implementation changes.
Official runtime references (rechecked 2026-09-27): chat API, structured output, Qwen3.5 catalog. Verify the installed runtime's capabilities rather than assuming its version matches current online documentation. In particular, discover supported thinking controls and test their actual effect before relying on output limits.
Activity
Address review of PR #210: recover, unloads, packets, pids, checks (#195
Address CodeRabbit's second pass on PR #210 (#195)
holden commentedon Sep 27, 2026
Stage A (PR #210) at the consolidation checkpoint: preserved, not accepted
PR #210 is at
cb2e236, with CodeRabbit's nine threads resolved. It is integrated intocodex/consolidation-2026-09-27, per #193, and is not merged.Evidence status. The session that wrote #210 also cut the checkpoint, so what follows is the author's re-verification, not independent acceptance. At the integrated head it ran:
Re-read against the audit list:
pg_control_system()is PUBLIC-executable, so a non-superuser role keeps working after the move.No new defect was found. An independent audit is still required.
Retained:
0b7253a7…for the results);devils_dictionary_runtime_bench(35 attempts, 105 ledger entries, fence 35, epoch 1) and_bench_b;Limitations that stand:
Prerequisite before any #197 panel work: a corrected instruction, and fixtures that don't label themselves (real permitted evidence), re-pinned as a new configuration and re-benchmarked.
Incident. While fixing review findings on #210, I migrated the shared
devils_dictionary_v2by mistake. #210's two runtime migrations were among the five applied, the hardening one in an early draft. No data changed. The rollback awaits the owner: details on #194.After the database moves, the runtime's authority marker still names the old cluster, so readiness will refuse until an operator runs
mix dd.runtime service start --rebindagainst the relocated database.Merge pull request #210 from razrfly/codex/runtime-195-stage-a
Merge pull request #220 from razrfly/claude/210-shape-repair