Skip to content

Gate generated admission outputs on admission opt-in - #71784

Merged
pelikhan merged 4 commits into
mainfrom
pelikhan-fictional-goggles
Oct 25, 2026
Merged

pelikhan merged 4 commits into
mainfrom
pelikhan-fictional-goggles

Conversation

@pelikhan

Copy link
Copy Markdown
Collaborator

Summary

Split from #71711 (source head 1476c568e699ea08bd9299592fab04e6a4609bfa). Addresses only the admission-output gating part of #71710; the issue remains partially addressed.

  • Reference activation admission outputs only when admission is enabled, excluding dry-run engine execution.
  • Propagate admission enablement into evals and threat-detection execution data.
  • Gate conclusion usage/session admission metadata and omit empty step environment blocks.
  • Preserve the source's admission tests and add an admission-only compilation regression with lint enabled for both opt-in states.

The six assigned files preserve the source PR's net changes against its merge base, with only the narrow additional regression. No checkout lint exceptions, repository-memory changes, runtime authentication changes, or workflow source/lock changes are included. The original PR and other split PRs are untouched.

Validation

Passed:

  • make build
  • make fmt (after restoring missing pinned dependencies with npm ci --prefix actions/setup/js --no-audit --no-fund)
  • go test ./pkg/workflow -run 'TestAdmission|TestUsageActivityGeneration' -count=1
  • go vet ./pkg/workflow
  • golangci-lint run --allow-parallel-runners ./pkg/workflow using the installed GOPATH binary: zero issues
  • make check-workflow-drift
  • git diff --check

make agent-report-progress was run once with tests. Build, scoped custom linters, schema freshness, and sampled impacted Go tests passed, but the aggregate command exited nonzero because another golangci-lint process held the shared runner lock. The standard package lint and workflow drift stages were subsequently run independently and passed. Custom-linter diagnostics were limited to unchanged files and skipped by the change-scoped gate. No workflow runs were dispatched.

Environment limitations

The requested branch rename was attempted three times but blocked by an existing shared packed-refs.lock. No shared lock was removed; the changes were committed and pushed from this session's original dedicated branch instead.

Split the admission-output gating concern from #71711, including an admission-only compile lint regression.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@pelikhan
pelikhan marked this pull request as ready for review October 25, 2026 01:44
Copilot AI max review requested due to automatic review settings October 25, 2026 01:44
@github-actions

github-actions Bot commented Oct 25, 2026 •

Copy link
Copy Markdown
Contributor

✅ Design Decision Gate 🏗️ completed the design decision gate check. See the comment below for the result and any generated ADR draft.

Design Decision Gate 🏗️

Outcome: reported_no_action

The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible.

Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.

Generated from Design Decision Gate 🏗️ · opus55 · ~12 AIC · ⌖ ~37 AIC · ◇ ~3 AIC · ⊞ 1.9K · ◷

🏗️ ADR gate enforced by Design Decision Gate 🏗️

@github-actions

github-actions Bot commented Oct 25, 2026 •

Copy link
Copy Markdown
Contributor

✅ PR Code Quality Reviewer completed the code quality review.

🔎 Code quality review by PR Code Quality Reviewer

@github-actions

github-actions Bot commented Oct 25, 2026 •

Copy link
Copy Markdown
Contributor

🧠 Matt Pocock Skills Reviewer has completed the skills-based review. ✅

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer

@github-actions

github-actions Bot commented Oct 25, 2026 •

Copy link
Copy Markdown
Contributor

✅ Ponytail Reviewer completed successfully!

Ponytail Reviewer

Outcome: reported_no_action

The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible.

Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.

Generated from Ponytail Reviewer · gpt56 · ~7 AIC · ⌖ ~6 AIC · ⊞ 14.8K · ◷

Generated by Ponytail Reviewer for #71784

@github-actions

github-actions Bot commented Oct 25, 2026 •

Copy link
Copy Markdown
Contributor

✅ Test Quality Sentinel completed test quality analysis.

Test Quality Sentinel

Outcome: reported_no_action

The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible.

Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.

Generated from Test Quality Sentinel · sonnet50 · ~14 AIC · ⌖ ~16 AIC · ◇ ~7 AIC · ⊞ 12.7K · ◷

🧪 Test quality analysis by Test Quality Sentinel

Document effective admission opt-in, component budget bindings, conclusion metadata, and focused regression evidence without adding a new threat identifier.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@github-actions

Copy link
Copy Markdown
Contributor

Comment Memory

Peek at saved memory (pr-code-quality-reviewer)
reviewed_at: 2026-10-25T01:46:54Z
review_event: COMMENT
top_themes:
  - dry-run admission output gating consistency
files_reviewed:
  - pkg/workflow/awf_env.go
  - pkg/workflow/compiler_admission_test.go
  - pkg/workflow/evals_steps.go
  - pkg/workflow/notify_comment.go
  - pkg/workflow/notify_comment_test.go
  - pkg/workflow/threat_detection_external.go
comment_count: 1

Note

This comment is managed by comment memory.

Expand the saved memory block to view or edit the persistent context for this thread.
Edit only the text inside the backtick fences; workflow metadata and the footer are regenerated automatically.

Learn more about comment memory

🔎 Code quality review by PR Code Quality Reviewer · copilot · gpt50 · ~5 AIC · ⌖ ~1 AIC · ⊞ 1.6K · ◷
Comment /review to run again

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review summary

One medium correctness issue found: conclusion usage generation does not apply the new dry-run admission-output gate, so dry-run workflows can reference admission outputs that are not produced. Inline comment posted.

Details
  • pkg/workflow/notify_comment.go:127: use the same AdmissionEnabled && !DryRun condition applied to evals, threat detection, and AWF environment wiring, with a regression test for the dry-run conclusion path.
  • No other actionable blocking issues were identified in the changed lines.
  • Static review only; repository tests were not run.

🔎 Code quality review by PR Code Quality Reviewer · copilot · gpt50 · ~5 AIC · ⌖ ~1 AIC · ⊞ 1.6K
Comment /review to run again

Comment thread pkg/workflow/notify_comment.go Resolved

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Skills-Based Review 🧠

Applied /diagnosing-bugs and /tdd (per pr-triage: bug_fix). The fix is correct and well-tested; approving with two minor non-blocking suggestions.

📋 Key Themes & Highlights

Key Themes

  • Root cause addressed, not just symptom: the bug was that admission output expressions (needs.activation.outputs.admission-*) were emitted unconditionally in the conclusion usage step, awf_env.go credits helper, evals job, and threat-detection job — even when admission was never opted into (or during dry-run, when the admission job is stripped). This PR gates every one of those emission sites on AdmissionEnabled (and !DryRun where the consumer is a child job), closing the whole class of bug rather than patching one call site.
  • Regression test quality: TestAdmissionOutputGatingCompilesWithLint is a strong addition — it does a full CompileWorkflow + lint pass for both admission: true/false and asserts on the compiled lock file content, which is exactly the kind of black-box regression test that would have caught this bug originally.
  • Minor duplication risk: the AdmissionEnabled && !DryRun condition is now repeated in three files; a tiny shared helper would make the invariant easier to maintain (see inline comment on awf_env.go).
  • Coverage gap: the new compile-level regression test checks the admission on/off axis but not the dry-run axis directly; the dry-run half of the gate is only unit-tested via applyDefaultMaxAICreditsEnvToMap (see inline comment on compiler_admission_test.go).

Positive Highlights

  • ✅ Verified via trace: data.DryRun is set to true before job-building in prepareDryRunWorkflowData/dryRunConclusionData, and the admission job itself is still built in dry-run (so its outputs exist) — only the consumers (evals, threat-detection, main-job credits env) are correctly gated off referencing them. This matches the design intent described in the PR body.
  • ✅ buildUsageActivityGenerationStep now omits the entire env: block when no env vars apply, avoiding a dangling empty YAML key — a nice defensive touch beyond the minimum fix.
  • ✅ Clean, scoped test updates (compiler_admission_test.go, notify_comment_test.go) that assert both the positive and negative cases (assert.Contains / assert.NotContains).

No blocking issues found. The two inline comments are polish suggestions (reduce duplication, add one more test case) and do not need to hold up this PR.

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer · copilot · sonnet50 · ~169 AIC · ⌖ ~16 AIC · ⊞ 10.2K
Comment /matt to run again

Comment thread pkg/workflow/awf_env.go Resolved
Comment thread pkg/workflow/compiler_admission_test.go Resolved

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Four fixture goldens still expect the removed admission bindings and must be regenerated to match the new compiler output.

1 open issue
What changed in this PR

Updates gh-aw’s workflow compiler so generated admission-output references follow admission opt-in, addressing the admission-related portion of #71710.

Changes:

  • Gate agent, evals, and threat-detection admission bindings, excluding dry-run execution.
  • Conditionally emit conclusion admission metadata while preserving independent environment bindings.
  • Extend regression coverage and document the contract in specification version 1.0.70.
File Description
specs/​compiler-threat-detection-spec.md Defines admission-output gating requirements and evidence.
specs/​compiler-threat-detection-changelog.md Records version 1.0.70 and its mapping audit.
pkg/​workflow/​threat_detection_external.go Propagates effective admission enablement into detection execution.
pkg/​workflow/​notify_comment.go Gates conclusion admission metadata and omits empty environment blocks.
pkg/​workflow/​notify_comment_test.go Verifies disabled admission and independent metadata bindings.
pkg/​workflow/​evals_steps.go Propagates effective admission enablement into evals execution.
pkg/​workflow/​compiler_admission_test.go Extends admission wiring and lint-enabled compilation regressions.
pkg/​workflow/​awf_env.go Requires enabled admission and non-dry-run execution for admission bindings.

Review effort: Max

Comment thread pkg/workflow/awf_env.go Resolved
pelikhan and others added 2 commits October 24, 2026 19:01
Remove admission activation output references from fixtures without admission, matching the compiler's conditional emission.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Cover admission opt-in and dry-run compilation with detection and evals, retaining conclusion metadata from the existing admission jobs without applying admission budgets during dry-run.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@pelikhan

Copy link
Copy Markdown
Collaborator Author

Regarding #71784 (comment): No code change is requested by this status notice, and reported_no_action is not proof of completion or publication. The review batch was independently validated and pushed as 00d341a; no ADR change was needed for this admission-gating clarification.

@pelikhan

Copy link
Copy Markdown
Collaborator Author

Regarding #71784 (comment): The completion notice itself requires no code change. The associated dry-run finding was assessed separately and answered inline, with behavior clarified and regression coverage pushed in 00d341a.

@pelikhan

Copy link
Copy Markdown
Collaborator Author

Regarding #71784 (comment): The completion notice itself requires no code change. Both actionable skills-review suggestions were accepted in 00d341a: centralized execution-budget gating and expanded dry-run compilation coverage.

@pelikhan

Copy link
Copy Markdown
Collaborator Author

Regarding #71784 (comment): This status notice contains no actionable change request, and reported_no_action alone is not evidence of completion or publication. The combined review changes were independently validated and pushed as 00d341a.

@pelikhan

Copy link
Copy Markdown
Collaborator Author

Regarding #71784 (comment): The status notice itself requires no code change and is not being treated as independent test evidence. Focused admission, usage, formal-contract, and golden-fixture tests passed for the batch pushed as 00d341a, along with the no-test pre-PR gate.

@pelikhan

Copy link
Copy Markdown
Collaborator Author

Regarding #71784 (comment): The managed memory summary is informational, so its contents were left unchanged. The dry-run theme was addressed in the associated inline thread with an explanation of retained admission metadata and regression coverage in 00d341a.

@pelikhan
pelikhan merged commit aa89ad1 into main Oct 25, 2026
71 checks passed
@pelikhan
pelikhan deleted the pelikhan-fictional-goggles branch October 25, 2026 02:19
@github-actions

Copy link
Copy Markdown
Contributor

🎉 This pull request is included in a new release.

Release: v0.93.4

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants