Repository navigation
Armonia: one shared coordination service, people as principals, distributed execution sites #1872
Description
Important
Problem — Amicode is single-tenant. The hub already runs one shared session store with sticky-routed engines, and /global/event already streams every session to any authenticated client, but nothing knows who is calling: authentication is a shared password, an open service behind a tunnel, or a per-machine peer token. Sessions and prompts carry no author, visibility is all-or-nothing, Research Environments are configuration rather than access boundaries, budgets live in prompts, and compute and lab hardware are reached through per-machine paths with no durable job identity. Ten Harmoniqs researchers cannot share one engine — see each other's agents, delegate between them, and run unattended campaigns across HPC and lab machines — without trusting every caller with everything.
Approach — One team deployment: a single Amicode coordination service that people connect to, which owns identity, environment membership, sessions, campaign authorization, delegation records, and the activity view; work executes at execution sites (agent workers, compute nodes, batch clusters, lab computers) that dial out to it and lease durable jobs. Research Environments become the membership and knowledge boundary (projects may restrict further, never broaden). Every session has an accountable human owner; observation follows membership, control is by owner invitation. Campaign budgets and tool authority are enforced by the server, not the model; delegation creates durable work requests that inherit, never exceed, the originator's budget and permissions. Hardware authority stays lab-side: Strumento decides whether an experiment executes. The first deployment is a private AWS host with Bedrock inference; the same contracts must later run against a self-hosted AMD inference pool, a Slurm cluster, and a university lab without topology-specific code.
Approaches considered — (1) One large shared server running every workload — simplest, but experiments and failures disrupt everyone and isolation is weak. (2) Federated personal engines (extend Fleet Studio's independent-peer model across people) — preserves local machines but makes shared state, access enforcement, and collaboration depend on every peer being reachable. (3) Central coordination with isolated, outbound-connected execution sites — chosen: one coherent view of work and authority, independent scheduling of agents, inference, science, and hardware, and a framework a lab can deploy with its own cluster and workstation.
Scope — in: person identity and attribution; environment membership and per-session context resolution; visibility-filtered lists and streams; invite-only control; per-agent isolated working copies; server-enforced campaign authorization; durable delegation requests; execution-site registry and outbound worker protocol; Strumento lab-site adapter (simulator first); batch-scheduler site adapter; artifact references; training-eligibility opt-in; Bedrock instance-role credentials; a parameterized private EC2 deployment; a two-person end-to-end acceptance test. out: clustered multi-host execution of the coordination service itself; cross-team federation; self-hosted model serving and post-training runs; GPU ports of the scientific stack; enabling unattended real-hardware execution before Strumento's lab-side gate is enforced; public internet exposure.
Assumptions — One coordination authority per team deployment; the shared session store stays single-writer for the first deployment. The first team is internal Harmoniqs (~10 simultaneous researchers), so a deployment-local people registry is acceptable before SSO. Execution sites can make outbound authenticated connections but may not accept inbound ones. Strumento's lab-side authority and durable task records are the hardware contract.
Acceptance Criteria
- A request without a valid person or site credential is rejected at every entry point; every accepted request carries a resolved principal.
- Every session and every admitted prompt records the authenticated person who created or sent it, and that attribution cannot be set or changed through client-supplied metadata.
- A person sees sessions, events, work requests, and artifacts only for environments they belong to, minus any projects that restrict them.
- Only a session's owner or an invited collaborator can prompt, abort, or answer permission requests in it; everyone else with visibility can observe.
- A campaign stops issuing provider turns, tool calls, and job submissions when its budget or stopping condition is reached, regardless of what the model requests.
- A delegated work request draws from its originator's budget, carries no broader permissions than the originator, and survives a coordinator restart.
- An execution site that disconnects and reconnects mid-job yields exactly one authoritative result for that job.
- A lab-site job is executed only if Strumento accepts it; a Strumento rejection or approval requirement is reported as such, never retried around.
- Two people on separate machines complete the end-to-end acceptance scenario against a private EC2 deployment using Bedrock with an instance role.
Testing Decisions
Each slice extends the suite that owns its seam (engine server/session/permission tests, the service runner and frontdoor tests, amico-run executor parity fixtures, the Strumento record-bridge fixtures). The parent closes on one new two-person end-to-end test that runs against a real deployment, including injected disconnects and restarts.
Key Decisions
- Principal, not machine. Peer and site tokens remain for machines, but every user-facing action resolves to a person. Machine credentials never stand in for a person.
- The frontdoor authenticates; the engine verifies. The edge resolves the person and forwards a signed principal; engines reject any principal they cannot verify, so bypassing the edge grants nothing.
- Environment is the membership boundary. Projects can only narrow access. Membership is a server-side record, not editor workspace state or environment variables.
- Owner + collaborators. Observation follows membership; control requires owner or invitation; every instruction and approval is attributed.
- Authority outside the model. Campaign budgets, tool allowlists, and stopping conditions are checked by the server before each provider turn, tool call, and submission.
- Delegation is a record. A work request names deliverable, acceptance checks, budget slice, permission set, and assignee; it is leased, fenced, and visible.
- Sites dial out. Sites register with their own credential and lease jobs by durable identity; reconnect reconciles rather than resubmits.
- Hardware authority is lab-side. Coordinator approval is necessary but never sufficient for hardware execution.
- Training is opt-in per environment. Collaboration access never implies eligibility for post-training datasets.
Data Contracts
- Principal — person id, display name, deployment, credential kind (person | site | machine); immutable per credential.
- Membership — environment id × person id × role; project restriction lists narrow it.
- Session attribution — owner, created-by, collaborators; per prompt: author, admitted-at.
- Campaign authorization — owner, environment/project scope, budget (cost, provider turns, solves, wall time), tool allowlist, device access level, stopping conditions, consumed totals.
- Work request — id, originating campaign and session, deliverable, acceptance checks, budget slice, permission set, assignee, lease (holder, generation, expiry), state, result reference.
- Execution site — id, kind (agent worker | compute | batch | lab), capabilities, credential, health; Job — durable id, site, lease generation, state, result reference.
- Artifact reference — content hash, site of custody, provenance (job, session, campaign), access policy, verification status.
Constraints & Invariants
- Single writer for the session store in this deployment; no second coordinator against a copied store.
- No new routing authority beyond what The connective tissue: one authority for location + liveness — the five-router architecture is one bug with many costumes #1760 converges on — identity rides the existing edge and engine path.
- Every user-visible number and result keeps its provenance (who, which campaign, which site).
- An expired lease holder can never publish an authoritative result.
- Real-hardware execution stays disabled until Strumento's lab-side gate covers every hardware entry point.
- Nothing in the deployment is exposed to the public internet; access is private network plus person credentials.
Prior Art
- The hub frontdoor, sticky shard routing, and merged session listing; the gateway-secret middleware.
- Fleet Studio's grant model (observe, control, lifecycle-admin), the control request→approve flow, the generation fence, and peer tokens (ADR 0031–0034).
- The engine's Location-scoped permission service, subagent permission derivation, session lineage edges, and worktree support.
- amico-run's executor contract, capability warrants, and the coordination ledger's claim/heartbeat/release.
- The Strumento task-record bridge and the QICK job-server abstraction.
- Research Environment manifests and resolver; ADR 0012 and ADR 0021.
Source
- Design discussion 2026-10-25 (team engine brainstorm); companion external proposal "Autoresearch for Quantum Hardware".
- Related: The connective tissue: one authority for location + liveness — the five-router architecture is one bug with many costumes #1760 (one authority for location + liveness), Add durable Work Column record and lease journal #979 (durable record + lease journal), Fleet Studio: in-window, multi-machine sessions & workspaces #1436 / Fleet Studio B2b — remote peer-studio control + live streaming on the observation path #1540 (Fleet Studio), Amicode for the outside-lab researcher — the journey, the campaign repertoire, and the seven seams (codesign PRD) #679 (outside-lab researcher codesign PRD), vault writer: attributed team writes, schema check before write, quarantine on failure #1863 (attributed team writes), feat(protocol): autonomous R&D system v2 — one source of truth, protected self-improvement, verified compaction #1820 (autonomous R&D system v2).
Notes
- AMD/Lux, self-hosted inference, and post-training are downstream consumers of these contracts, tracked separately.
Activity
aarontrowbridge commentedon Oct 26, 2026
Reassigned to @aarontrowbridge: the amicode R&D system is taking this work on directly. @jack-champagne, thanks; your aws-infra #322 lease/reservation design is the prior art for #1880/#1882 (see #1873). The deployment target is being re-evaluated: Modal as the hub with Bedrock for inference, in place of #1887's EC2. Details to follow here.
[-]Team engine: one shared coordination service, people as principals, distributed execution sites[/-][+]Armonia: one shared coordination service, people as principals, distributed execution sites[/+]