Repository navigation
docs(metric-benchmark): design registered-agent Helix execution - #185
Merged
mstaats-nvidia merged 24 commits intoNov 13, 2026
Merged
Conversation
Signed-off-by: mstaats <mstaats@nvidia.com>
This was referenced Nov 8, 2026
Signed-off-by: mstaats <mstaats@nvidia.com>
Signed-off-by: mstaats <mstaats@nvidia.com>
Signed-off-by: mstaats <mstaats@nvidia.com>
Signed-off-by: mstaats <mstaats@nvidia.com>
Signed-off-by: mstaats <mstaats@nvidia.com>
Signed-off-by: mstaats <mstaats@nvidia.com>
Signed-off-by: mstaats <mstaats@nvidia.com>
Signed-off-by: mstaats <mstaats@nvidia.com>
Signed-off-by: mstaats <mstaats@nvidia.com>
Signed-off-by: mstaats <mstaats@nvidia.com>
Signed-off-by: mstaats <mstaats@nvidia.com>
Signed-off-by: mstaats <mstaats@nvidia.com>
Signed-off-by: mstaats <mstaats@nvidia.com>
Signed-off-by: mstaats <mstaats@nvidia.com>
Signed-off-by: mstaats <mstaats@nvidia.com>
Signed-off-by: mstaats <mstaats@nvidia.com>
Signed-off-by: mstaats <mstaats@nvidia.com>
Signed-off-by: mstaats <mstaats@nvidia.com>
Signed-off-by: mstaats <mstaats@nvidia.com>
Signed-off-by: mstaats <mstaats@nvidia.com>
Signed-off-by: mstaats <mstaats@nvidia.com>
…sion Signed-off-by: mstaats <mstaats@nvidia.com>
Signed-off-by: mstaats <mstaats@nvidia.com>
mstaats-nvidia
marked this pull request as ready for review
November 10, 2026 13:03
aleckhoury
approved these changes
Nov 12, 2026
mstaats-nvidia
deleted the
explore/metric-benchmark-helix-architecture
branch
November 13, 2026 10:52
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Define a reversible Helix benchmark workflow: publish agents and unchanged tasks separately, execute one registered agent against a pinned taskset, and download standard results and exact sources.
The architecture specifies four proposed tools: publication, execution, collection with verification/import, and source download. It includes an SVG resource flow, configuration filenames and ownership, and command examples. Tasks use pinned revisions; agent registrations are mutable, so the design requires content-derived agent names, content checks and a separately retained remote source copy. That retention mechanism remains a downstream requirement to verify. Execution waits for image builds before submitting and saves a retry key before the submission request.
The example directory contains only illustrative YAML configurations and a README. Experiment, case, agent/task manifest, analysis, publication, execution and dummy client files show how the configuration fits together. Referenced agent bundles, tasks and analysis data are deliberately absent. The README explains how case/agent/task names map to registered Helix IDs through references.json, how execution and downloads use that mapping, and what changes from local experiments. No real secrets or generated run evidence are included.
This PR contains architecture documentation and YAML examples only. Proposed commands and registered-agent/signing support belong downstream. The example is intentionally unsigned and not runnable. No merge or downstream restacking is included.
Validation: YAML syntax, YAML-only example inventory, local documentation links, copyright headers, whitespace and DCO hook passed. For the latest documentation revision, copyright headers, whitespace, SVG XML parsing and DCO checks passed; the SVG layout is unchanged from its previous visual inspection. No live Helix, model or container execution. Full tests and broad runtime/dependency checks were not run under the requested focused-check scope; dependencies are unchanged.