protocol: Hermes-managed evolutionary training provider for z0 autoresearch

Goal

Integrate SouthpawIN/evolutionary-training as the first external training provider for Evolution Lab's autoresearch loop, using the existing Hermes GYM-TRAINER.md profile and training/evolution scripts instead of reimplementing them.

User fork: https://github.com/kvnloo/evolutionary-training
Pinned upstream/fork baseline inspected: 68ecf2057586a232bea45ad0fafcdfdfae56089f

Credit: the Gym Trainer profile, Darwin/CMA-ES evolution loop, SFT/GRPO training loop, data ingestion, staged lineage, and benchmark orchestration are SouthpawIN's work. This integration should preserve that provenance.

Architectural boundary

Hermes profile(s)
    │ propose / operate / recover
    ▼
Evolution Lab
    │ owns experiment protocol, budgets, candidate lineage,
    │ grouping, selection, promotion gates, receipts
    ▼
Training provider: kvnloo/evolutionary-training
    │ existing data / train / evolve / benchmark machinery
    ▼
candidate artifacts + raw measurements
    │
    ├──> Evolution Lab selection / Pareto archive
    │
    └──> z0evals freeze/export after revisions are pinned
                 │
                 ▼
          z0intelligence shadow/canary promotion

Do not vendor or rewrite Southpaw's training scripts into Evolution Lab. The fork remains upstream-syncable and implements a provider contract.

Do not turn z0evals into the online trainer or mutable judge. Its current contract says it is the frozen evidence/publication layer. Evolution Lab may consume pinned eval manifests/suites and export frozen results to z0evals, but live proposal/training/search stays here.

Why this fits the current lab

Evolution Lab already owns:

experiment genomes;
locked/grouped splits;
autoresearch propose → bench → select;
promotion gates;
receipts and lineage;
external/fail-closed backends.

Southpaw's repo already owns:

a Hermes gym-trainer profile;
continuous Darwin/CMA-ES evolution;
SFT/GRPO/QLoRA training;
data ingestion/preparation;
benchmarking;
staged model lineage;
reporting/recovery loops.

The integration should therefore be a protocol adapter, not a new trainer.

Protocol: `z0.training.autoresearch.v1`

A provider manifest should declare capabilities and commands, not embed implementation logic.

Suggested phases:

preflight
- resolve provider commit, environment, hardware, model/data prerequisites
- fail closed if protected eval data or secrets would enter training
propose
- Hermes proposes one bounded experiment from the allowed search surface
- proposal records hypothesis, parent candidate, changed knobs, budget
prepare_data
- provider prepares training/validation data
- work-item/project lineage preserved
- protected confirm/OOD material excluded
train
- invoke existing Southpaw training path (SFT/GRPO/QLoRA/etc.)
- no duplicate Evolution Lab implementation
evolve
- optional Darwin/CMA-ES/model-merging arm
- treated as a methodology arm, not implicitly superior
evaluate
- run pinned z0eval-compatible suite/contract
- self-generated reward is never ground truth
- verifier/judge revision must be outside evolvable surface
select
- Evolution Lab applies hard gates + Pareto / promotion rules
- candidate cannot edit its judge, gates, or protected splits
promote
- only to a reversible z0intelligence shadow/canary target
- runtime security/capability policy remains deterministic
freeze
- pin provider/model/data/eval commits
- export allowlisted evidence into z0evals
- failed/negative hypotheses are retained

Minimal provider manifest

{
  "schema": "z0.training.autoresearch.v1",
  "provider": {
    "repo": "kvnloo/evolutionary-training",
    "upstream": "SouthpawIN/evolutionary-training",
    "revision": "<sha>"
  },
  "hermes_profile": "GYM-TRAINER.md",
  "capabilities": [
    "data_prepare",
    "sft",
    "grpo",
    "darwin_merge",
    "cma_es",
    "benchmark"
  ],
  "commands": {
    "preflight": "...",
    "prepare_data": "...",
    "train": "...",
    "evolve": "...",
    "evaluate": "..."
  },
  "artifacts": {
    "candidate_manifest": "...",
    "training_receipt": "...",
    "eval_result": "...",
    "lineage": "..."
  }
}

The actual command map should adapt the existing scripts rather than rename/recreate them.

Candidate / receipt contract

Every candidate should minimally record:

candidate ID + parent IDs;
Evolution Lab experiment/genome ID;
provider repo + exact commit;
Hermes profile revision;
training method + exact config;
dataset manifests + grouped lineage IDs;
model/base checkpoint digest;
random seeds;
wall time, GPU-ms/VRAM peak where measurable, tokens/cost;
raw train/validation metrics;
eval suite ID + exact z0eval revision;
independent verifier/result;
hard-policy violations;
promotion status;
immutable artifact digests.

Hermes role

Start by adapting Southpaw's existing GYM-TRAINER.md, not replacing it.

The z0-specific profile should:

read the Evolution Lab experiment contract;
operate only within the allowed candidate/search surface;
call the provider's existing scripts;
inspect failures and retry/recover within budget;
never modify protected evals/gates;
emit structured receipts after each phase;
stop/escalate on judge drift, data leakage, secret exposure, repeated OOM, or budget exhaustion.

Longer term, the single Gym Trainer profile can delegate bounded subroles (data, trainer, evaluator, replicator) while Evolution Lab remains the authority for selection.

Relationship to existing issues

eval: tool-calling SLM tournament and Agent0-style self-evolution track #23: this becomes a reusable implementation substrate for the Agent0/self-evolution training arm.
experiment: learn state selection, retrieval routing, and observe-vs-act from verified trajectories #24: learned state/retrieval policies can use the same provider protocol without giving the trainer authority over runtime policy.
experiment: factorized z0 stack — SoL-Pi × RLM × memory × SLM × Bend #27: training/evolution is another factor; singleton evidence must exist before composing it with memory/RLM/SLM/Bend mechanisms.
z0evals remains the protected/frozen evidence layer.
z0intelligence remains runtime promotion/shadow/canary owner.

First slice

No model training required yet.

Define/validate z0.training.autoresearch.v1 provider manifest.
Add a read-only adapter that discovers kvnloo/evolutionary-training and pins its revision.
Adapt GYM-TRAINER.md into a z0 Hermes profile that only runs preflight + dry-run command resolution.
Produce a synthetic candidate receipt.
Verify that the adapter refuses:
- missing provider revision;
- protected eval data in training inputs;
- provider command outside declared capabilities;
- candidate attempts to mutate judge/gates;
- unpinned z0eval suite.
Only then wire one real low-cost training arm.

Acceptance

claiming for autodevelop
claimant: kvnloo
base: 0212fee
expires: 2026-10-06T17:09:00Z
scope: Add the dry-run z0.training.autoresearch.v1 provider adapter, Southpaw Gym Trainer profile adaptation, manifest, receipts, and fail-closed tests.

CLAIM — Hermes research lane, backlog mode; EL-31 external-provider qualification only.

Task: EL-31-PREFLIGHT-RESEARCH-20261010. Existing autodevelop lease expired 2026-10-06; current open PR file inventory has no overlap. This is a bounded research/test receipt pass, not a claim that the full provider adapter is implemented.
Branch: factory/hermes/31-provider-preflight-research-20261010; Evolution Lab base 0212fee31a78cdba506f4d2684813962bfe0efb0.
Exclusive paths: docs/experiments/training-provider-20261010/ only. Separate isolated checkout; private evidence key oss-factory/20261010-hermes-research-el.

Two cheapest source-backed questions, using the original external libraries unchanged:

Does the existing extractor's output directly reach the trainer formatter without silent sample loss? Positive native ShareGPT control and explicit synthetic tool/provenance negatives; no raw history export, conversion engine or training.
Is the provider's advertised dry-run actually read-only, or can failed health readiness trigger a completions request? Exercise the real provider main against an owned loopback HTTP fixture, not production servers; no inference or model download.

External source pins: ai-data-extraction 9176c2dd81c8a101b52debd4cec88573284e2b30; evolutionary-training 68ecf2057586a232bea45ad0fafcdfdfae56089f. Source credit remains with sybil-solutions and SouthpawIN. No external library edits/vendoring; no Claude extractor files, Codex judges/runtime files, candidate skill installation, upstream post, PR auto-merge or production changes. Failed hypotheses remain failures; fixture observations do not establish model quality or savings.

READY-EVIDENCE / RELEASED — Hermes EL-31 research backlog slice

Published exact fork ref factory/hermes/31-provider-preflight-research-20261010 at 29cf074c0071663b9d54961d7df5b3a8149333c9 (remote readback verified), base 0212fee31a78cdba506f4d2684813962bfe0efb0. Only docs/experiments/training-provider-20261010/ changed. Full report/commands: https://github.com/kvnloo/evolution-lab/blob/29cf074c0071663b9d54961d7df5b3a8149333c9/docs/experiments/training-provider-20261010/README.md

Two bounded cycles used the original external libraries unchanged:

Data interoperability, 8cb2d35e21df74ed6549d5ad721b234ffb3cd43a: actual extractor CLI emitted one explicit synthetic conversation with two messages/two tool events; original skill sampler accepted 1/1, but original trainer formatter produced zero conversations. Its native ShareGPT user/assistant/tool control produced one. Direct interchangeability is falsified; no replacement converter or trainer was built.
Dry-run no-inference contract, f2eb54b385fc435ca8971e5798977068bd1701fd: original training_loop.main --dry-run attempted one completions POST after health GET 503; owned fixture refused POST with 405 before inference. Healthy control made GET only. Wrong real provider revision rejected with exit 2/no receipt. No production endpoint was contacted.

Independent read-only reproduction of f2eb54b: exit 1, four tests, two expected hypothesis failures/two positive controls, zero errors/skips; source hashes/observations exactly matched. Private reproduction digest 8964310f289929ed7df9aea2b9eab2399b3a1920a6ef8f0b784605193887c033; sanitized receipt is in the published directory. Final-head reproduction again had the same four tests/two RED hypotheses. This is not a sealed z0evals #92/#93 verdict or proof of model-quality improvement.

Exact-head owning fork CI passed: https://github.com/kvnloo/evolution-lab/actions/runs/38027267255 at 29cf074c0071663b9d54961d7df5b3a8149333c9, 271 discovered unit tests (269 passed, 2 skipped) plus existing toy recovery gym. This live result supersedes the frozen pre-push CI=NOT_RUN snapshot in gates.json. Local reference and candidate each had eight identical missing-JAX errors/two skips; no local install or GREEN claim hid them. The research hypotheses remain RED despite the owning suite's CI success.

Cost coverage: first two local probe windows measured 1.044646 s wall / 0.040969 s process+child CPU; independent reproduction 1.040590 s / 0.032837 s. Probe inference/training calls zero, but coordinator and reviewer model usage is additional: preserved partial coordinator counters and seven reviewer API calls. Earlier discovery, current delivery/resume, startup/import/tool-server cost, full task totals, invoice cost, GPU-ms and joules remain incomplete/unknown. No measured savings or compute-efficiency improvement claimed.

External pins: ai-data-extraction 9176c2dd81c8a101b52debd4cec88573284e2b30; evolutionary-training 68ecf2057586a232bea45ad0fafcdfdfae56089f. sybil-solutions/SouthpawIN credit preserved; no library vendoring/edits. No upstream action, PR/default merge, production change, model download or skill installation. Claude extractor and Codex evaluator/runtime files untouched.

This research path lease is RELEASED, not the whole issue closed. Remaining EL #31 gates: thin versioned provider manifest, explicit representation/lineage boundary, protected split/judge-mutation negatives, zero-inference-safe preflight selection, then separately authorized and independently judged low-cost training. No model/capability promotion is eligible from these fixture observations.

Additional original-tool reuse control (coordinator-executed, not included in the independent review above): the existing scripts/mega_training_data.py::format_to_sharegpt at provider 68ecf2057586a232bea45ad0fafcdfdfae56089f already maps the same synthetic extractor record into the trainer's representation. Running that original function, then the original SFT formatter, produced one two-message user/assistant conversation; direct handoff still produced zero. Reuse this function rather than building a replacement converter. The two separately stored tool events were not embedded, and grouped source/protected-data eligibility remains unqualified. No training/inference/network or source edit. Private control receipt SHA-256 31e62cf4b52fe6b6af04c5efe6690c7d11859d4f694242e6c7ddf27eb38e90f6; original preparer digest a6be6b819919cd656f1cc6aa8ec89bc69d8c3480e76cef4aaf076ba8c59876e3. This narrows the remaining representation work; it is not an additional model-quality result or a new independently reviewed implementation.

CLAIM — Hermes z0 learning integration, P0 preparation-only

Task: EL31-Z0-LEARNING-INTEGRATION-20261010. Explicit owner assignment; prior research lease was released. Branch factory/hermes/31-learning-integration-20261010, policy worker-base nightly 0212fee31a78cdba506f4d2684813962bfe0efb0.

Exclusive Evolution Lab write set: evolution_lab/training_provider.py, tests/test_training_provider.py, docs/experiments/training-provider-integration-20261010/**. Separate isolated worktree; one implementation writer and separate exact-head verifier. No capacity_queue, conversation skill producer, extractor repair, runtime receipt aggregation or protected evaluator edits.

Reuse original external preparation functions unchanged (SouthpawIN provider 68ecf2057586a232bea45ad0fafcdfdfae56089f; sybil-solutions extractor 9176c2dd81c8a101b52debd4cec88573284e2b30). Real extractor → existing preparer/formatter → explicitly untrained candidate/lineage. Preserve origin and grouped train/validation boundaries; reject unsupported tool semantics rather than silently losing them. Dry-run must never call the unsafe provider health/training/evaluation path. Network/inference/training and model installation disabled.

Prior negative research 29cf074c0071663b9d54961d7df5b3a8149333c9 and completed eight-cycle secondary report d48d3fa64162f0e07200026c9c71e2f1f53fb23a remain historical and unchanged. New RED/GREEN qualification concerns the adapter, not another repetition of those investigations. Independent z0evals verification will use an unchanged pinned verifier and owned UNIT fixtures, never protected live state; no preparation success represented as model efficacy or savings.

Codex #150/#95 and Claude #132/#134/#20/#141/extractor ownership excluded. No paid fallback, provider trial activation, SDK, new scheduler/store, live routing, auto-training/promotion, upstream publication or default merge. Canonical result will record exact baseline/candidate, evidence, cost coverage and next eligible z0 task. Evidence key: oss-factory/20261010-hermes-z0-learning-integration.

HOLD — Hermes EL31 independent-review triage and bounded repairs

Baseline 04974fa72f87a95f9f78df045edeff2985044620; prior candidate 522b7beace91a678b69515a494d783514f03c4a6; current local candidate 9c50af79c792f02578649d1a13238b886b116a81. These integration commits are local, not published refs or accepted PRs. Existing exclusive claim/write set remains unchanged.

Reproduced all nine retained failed assertions against baseline and prior candidate, in exact Git snapshots with network denied. Baseline: nine failures, no errors/skips. Prior: eight assertion failures. Classification:

Six source/message/context cases now refuse with exit 2 and no candidate. Their remaining failures demand the uncontracted literal word source, whereas the actual reason is record differs from pinned native extraction. The underlying baseline projection defects were real; the prior-head remaining wording assertion is an incorrect fixture expectation. The inherited-parent-alias case also changes extractor project context; this is not evidence that every possible path alias is forbidden.
Two cross-partition cases remained genuine defects: identical conversation content under changed timestamps/identifiers, and reused native branch-event UUIDs under distinct work labels. Small repair 5d28a1340ebcc401b374fa70388d4f22913adae9 excludes these before candidate writes. Both original independent assertions pass unchanged on that exact head.
The output-directory substitution was a genuine baseline defect. Its apparent prior-head pass is invalid: the fixture injects at a hardcoded source line that moved. A new real-filesystem timing fixture reproduced the defect RED on 5d28a13. Repair 9c50af7 binds writes to an owned directory descriptor and refuses changed output identity; no writes reach the substituted target. The independently authored timing fixture still needs exact-revision rebinding and verification; no false pass is accepted.

Current exact-head local gate: 18 focused tests passed, no skips, including actual extractor/preparer/dry-run CLI and output-identity controls. Additional 19 adjacent tests and Ruff passed before the final commit on equivalent source. Only evolution_lab/training_provider.py and its owned tests changed in the two new repairs. Original sybil-solutions extractor and SouthpawIN preparer remain unchanged. No raw personal history, protected-evaluator edit, training, model download, live provider inference, promotion, or default merge.

Independent reviewer status remains content_policy_blocked; it was not automatically retried or routed around. Coordinator execution of retained independent tests is not a completed reviewer verdict. P0 stays HELD pending corrected deterministic-fixture qualification and the unchanged z0evals independent return leg. Producer-asserted work/protected mappings remain unqualified historical lineage; no learned capability or savings claim.

Private durable evidence key: oss-factory/20261010-hermes-z0-learning-integration; retained matrix, nine-row classifications, RED/GREEN receipts and mission checkpoint. Three nine-assertion executions measured approximately 1.277s, 1.275s and 1.263s wall; final focused suite 3.637s. These are bounded test measurements, not end-to-end economics. Blocked reviewer used 14 recorded API calls; total parent/worker/verifier tokens, invoice cost and energy remain unknown.

Next: validate exact-head fixture timing without weakening material assertions, qualify the source-pinned z0evals return leg, then select a receipt-backed recurring capability against deterministic/Jev controls. Separately, existing host-supervisor qualification is read-only and in progress; no persistent continuation service enabled. Its old broad-backlog prompt conflicts with the new z0-only mission, and unattended spending limits are not configured. Groq/Cerebras remain UNKNOWN/NOT_ADMITTED.

HOLD — exact-head corrected deterministic review

Candidate 9c50af79c792f02578649d1a13238b886b116a81 now passes 13 retained independently-authored deterministic fixtures: all nine reviewed cases, actual extractor/preparer positive flow, both completion-marker I/O controls, and concurrent-output ownership. Exit 0; zero failures/errors/skips; measured wall 2.388874s. No production source changed in this qualification slice.

Fixture corrections are explicit and preserved separately from originals: six incidental diagnostic-word expectations now match the actual native-extraction refusal; exit-2 and no-candidate assertions are unchanged. Fault timing is bound to artifact-open/write coordinates at this exact SHA, and the directory-replacement test additionally requires its fault to fire exactly once. Target-empty and nonzero-result assertions remain intact. The apparent old-coordinate pass is not counted. Harness SHA-256 0c69aaaee235bb79c8d9bf21c3ec320bf4d4c1b6d3edd80029021ff52225d218; adapter source SHA-256 b5df0b74df272cb14e2b3629d46bf9da99e289eb8d5e5401ea066d542ff871ca.

This is coordinator-adapted/executed deterministic qualification, not model-reviewer acceptance. The blocked reviewer remains blocked and was not retried. P0 remains HELD pending the unchanged pinned z0evals return leg and full acceptance reconciliation. No canonical sealed evaluation, training, inference, learned gain, or savings claimed. Total model expense remains unknown. Existing provider/extractor ownership and original algorithms unchanged.

Private evidence key oss-factory/20261010-hermes-z0-learning-integration/corrected-deterministic-review-9c50af7; durable review status updated. Next: independently owned synthetic return-leg validation against the unchanged verifier, not live protected truth/state. Host-continuation qualification remains a separate read-only task; no service enabled.

HOLD — existing host is not qualified for persistent z0-only continuation

Read-only qualification of existing supervisor a0b1ffc21e3cd9e31318df21d671a71f1b747439: 29 deterministic host tests passed in 46.929s under denied-network isolation. Coordinator rechecked exact HEAD and clean source. These legacy tests do not qualify the standing mission: build_prompt permits unrelated tracked projects and a legacy test requires that behavior; narrower focus does not remove it.

Missing gates include persisted terminal refusal handling, shared writer ownership, cross-invocation aggregate budget/checkpoint admission, resource admission, backoff and useful-progress idle detection. Account-bound unattended spending authorization remains UNKNOWN. Existing lane locks and finite shifts do not establish indefinite safe execution. No service/timer installed or enabled and no replacement P0 writer launched. The proposed binding reuses the existing single-lane supervisor and systemd user manager; it is not another scheduler. No automatic policy-refusal retry is authorized.

Private evidence key oss-factory/20261010-hermes-z0-learning-integration/host-qualification; qualification/report/verification retained. Host review used 21 recorded model API calls; complete tokens/invoice totals unknown. Host fixture workers made no inference/training calls. P0 preparation candidate remains HELD at local 9c50af79c792f02578649d1a13238b886b116a81; its corrected 13-fixture deterministic gate is not a sealed or learned-capability result. Next eligible work remains the unchanged z0evals return leg; host activation additionally requires repaired/requalified gates and explicit aggregate spending limits.

HOLD — unchanged z0evals return-leg UNIT execution completed

Candidate 9c50af79c792f02578649d1a13238b886b116a81 now has an executed bounded data path: actual original extractor/preparer candidate (2 train/1 validation/1 protected) → artifact/row/eligibility checks → unchanged z0evals 588c21aeedf8f896fb83988c18351e7a4e100b9f. Verifier bytes matched Git before import; checkout remained clean. Owned public-synthetic truth/state were outside all contestant repositories, with network denied. No canonical protected state/truth was accessed.

Three engineering UNIT checks passed for original bytes. An owned altered assistant body failed digest and train-projection checks. Unchanged verifier returned positive NOT_COMPARABLE and altered-body DISCARD, with sealed_credit=false for both. These synthetic boolean checks are not task gold, learned quality, or a savings pilot. Measured return-leg wall 0.125807s. Initial isolated interpreter lacked PyYAML; retry used an existing isolated Hermes venv with required dependencies. Nothing installed and no fake evaluator substituted.

Expanded retained corrected suite: 31 executed, 30 passed, one failed, no errors/skips, 4.944s. Remaining failure copied a native file to another project but kept stale extracted project metadata while expecting acceptance. A separate counter-control used actual output from the unchanged extractor on that single owned relocated file; adapter accepted it with independently_qualified=false and all efficacy/promotion claims false. Original failing suite is retained and is not relabeled 31-pass. No production changes in this slice.

Private evidence key oss-factory/20261010-hermes-z0-learning-integration; return-leg receipt, full31 result, relocated-source classification and durable review state updated. Blocked model reviewer remains blocked; no retry. P0 remains HELD for independent acceptance reconciliation and absent real-history lineage/gold qualification. No training/inference/provider call, promotion, evaluator edit, upstream action or default merge. Full model expense unknown. Next eligible prerequisite: independent scoped acceptance/source qualification; separately the existing host remains unqualified and unenabled.

Claim lease expired (expires_at 1791306540 <= now 1792188286). Returned to claimable. Workers do not merge. A new claimant may take this issue.

Navigation Menu

Repository navigation

protocol: Hermes-managed evolutionary training provider for z0 autoresearch #31

Goal

Architectural boundary

Why this fits the current lab

Protocol: `z0.training.autoresearch.v1`

Minimal provider manifest

Candidate / receipt contract

Hermes role

Relationship to existing issues

First slice

Acceptance

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

protocol: Hermes-managed evolutionary training provider for z0 autoresearch #31

Description

Goal

Architectural boundary

Why this fits the current lab

Protocol: z0.training.autoresearch.v1

Minimal provider manifest

Candidate / receipt contract

Hermes role

Relationship to existing issues

First slice

Acceptance

Activity

kvnloo commented on Oct 5, 2026

kvnloo commented on Oct 10, 2026

kvnloo commented on Oct 10, 2026

kvnloo commented on Oct 10, 2026

kvnloo commented on Oct 10, 2026

kvnloo commented on Oct 10, 2026

kvnloo commented on Oct 10, 2026

kvnloo commented on Oct 10, 2026

github-actions commented on Oct 16, 2026

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions

Protocol: `z0.training.autoresearch.v1`