Skip to content

ci(policy): run complete policy validation in parallel - #5069

Merged
devantler merged 6 commits into
mainfrom
codex/ci-parallel-policy-5068
Oct 18, 2026
Merged

devantler merged 6 commits into
mainfrom
codex/ci-parallel-policy-5068

Conversation

@devantler

@devantler devantler commented Oct 18, 2026 •

Copy link
Copy Markdown
Contributor

🤖 Generated by the Agentic Engineer

Why

Platform feedback takes a median 56m49s in three successful baseline runs. Nearly fifteen minutes of independent policy checks run serially inside the longest validation job, delaying every later check.

What

Run that complete policy family alongside the remaining validation, preserving every command and safety outcome. Selected policy checks must finish successfully before merge.

The first complete successful run finished in 40m30s, 16m19s / 28.7% shorter than the baseline median. Manifest validation occupied 23m09s while the complete policy job ran alongside it. This is one observation, not a repeated post-change median. Occupied job time increased from baseline median 72m04s to 73m05s; no cost saving is claimed. Repeated outcomes remain on #4569.

Fixes #5068
Part of #4569

The unchanged schema/authorization prerequisite from #5063 is already delivered on main through #5048. Recovery acceptance and the protected production queue remain separate gates; that prerequisite is not this PR's delivery scope.

devantler and others added 3 commits October 18, 2026 18:20
The CRDs-catalog ResourceGraphDefinition schema for KRO 0.9.2 now
requires spec.schema.scope, which failed offline validation on main.
Declare the Namespaced scope both graphs already defaulted to and
re-pin their RGD-CONTENT baseline hashes.

Fixes #5062

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The tenant ResourceGraphDefinition is part of the pinned EKS CI
authorization surface. Approve its new rendered digest; the only change
is the explicit Namespaced schema scope, which matches KRO's default.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@devantler

devantler commented Oct 18, 2026 •

Copy link
Copy Markdown
Contributor Author

🤖 Generated by the Agentic Engineer

Implementation, validation and review receipt

Exact head: 81fb21e4d0253491d1d57da5b87d04963266fb4a; base 7120bf164af2eaf2c8734ef17b1f1e3be514e733; tree bf82816f988319ecd9abf9cad80721258a1c74f3. Local signature verification is good and GitHub independently reports the commit verified. The three changed files are the workflow, its result-boundary regression and the current CI documentation.

The complete policy step's 71 command lines are byte-identical to the original. It runs once in the independent PR-only job under the same existing k8s output. All other policy/admission steps remain in their original jobs. Checkout, Go, YAML-parser and Kyverno pins are unchanged; credentials are not persisted, the new job has only contents:read, and no secret or cluster surface was added. The required aggregate now needs and consumes the job result, while an additional guard rejects selected skips and unknown PR selection. Merge-group deployment and path decisions are unchanged.

TDD: before the workflow change, the new regression failed because the required aggregate did not depend on the policy job. A second RED control showed that an empty selection was incorrectly accepted; the guard now refuses that unknown. All 12 event/selection/result controls pass on both the system Bash 3.2 and the current Bash. They cover success, failure, cancellation, selected skip, pending, unknown/missing result, false selection, unknown/missing selection and intentional merge-group non-selection.

Actionlint, ShellCheck, whitespace checks and the required-workflow source-consumer test pass. The existing deployment/consumer-discovery suite passes 31/31 unchanged; its first run exposed an index-based test assumption when the new guard preceded the established aggregator, so the guard follows that aggregator instead. Existing failure handling remains first, and the new guard catches its intentionally accepted selected-skip case. No existing assertion was weakened.

Self-review and simplification pass: checked the job dependencies, both event/path states, least privilege, complete command parity, tool setup and selected-result refusal. No unresolved local finding remains. This receipt is not readiness: hosted CI 39148281755 must exercise the complete independent job at this exact head, all required/quality analyses must clear, and a substantive current-head independent review is still required. No whole-workflow performance improvement or runner-time saving is claimed yet. The three-run baseline is recorded on #4569; its remaining families and post-change cohort stay open.

Prerequisite integration, 2026-10-18

Current signed head: d41016914d556fd70712a4ab64372531e92365b4, tree 65f1d2add37c8cde2cbed0c27c13eebbce57da1c (GitHub verified). It merges the trusted schema prerequisite befe5d860a9d8486e44cc7309817322608bf2b8d from #5063, so the full hosted validation need not wait to start until that PR merges. Its owner's source branch is unchanged. The 12 result-boundary controls, existing 31-case deployment/consumer contract suite and Actionlint pass again after integration. The policy command block is unchanged by this merge.

This does not clear readiness: #5063 must merge first, the new exact head must finish its full hosted execution and quality analyses, and an independent current-head review remains required. No elapsed-time improvement from the new independent job is claimed before those hosted observations.

Hosted contract failure and correction, 2026-10-18

Current signed head: acf88e6c7a44cb1d76e41eab57204305ababa256, tree 1e898fb29acf07eb505beb64210d968ba4ff40dd (GitHub verified). Run 39149194832 failed before manifest selection because the existing production-authorization workflow test assumed that the required-check job had exactly one step. The new result guard intentionally adds a second step. The same failure was reproduced locally before the correction.

The contract now identifies exactly one genuine aggregate action rather than assuming a step count or index. It still requires authorization as a deployment and aggregate dependency, and still requires the aggregate to consume the authorization result. Missing or duplicate aggregate actions fail. No workflow command or authorization rule changed in this correction.

All four affected Go packages pass on the repository's pinned Go 1.26.9. The 12 policy-result controls, 31 unchanged deployment/consumer admission controls, required-workflow source consumer, Actionlint, ShellCheck and whitespace checks also pass. A broader run on host Go 1.27.2 found an unrelated existing internal/strictjson/TestValidate/truncated_document diagnostic mismatch; that test passes unchanged on Go 1.26.9. The broader run is not reported as green.

New exact-head CI 39150213577 is running. All readiness, independent-review, prerequisite-merge and production-queue gates remain pending.

Completed suite readback

The new head's complete hosted repository suite succeeded in 7m22s, and its separate native retirement suite succeeded in 2m07s. The full workflow is still running.

The broader local Go 1.26.9 run finished with exit 1: every other package passed, but the unchanged GHCR-auth fixture package failed after 592.439s on late fixture-command deadlines. This is reported on existing #5015 as a local observation, not as proof of its historical root cause. The host Go 1.27.2 full run also hit its explicit 600s process bound and is not green. No timeout, test or gate was relaxed to hide either result.

Executable-entry correction

Current signed, GitHub-verified head: 9c1bdc7c7c44caeef288c12f1bf9440482462ae4, tree 1af5d577060379cde1973f00c2dadf748b9c8155. The previous run ultimately failed the existing test-entry execute-bit guard: the new regression was tracked as 100644 instead of the required 100755. Reproduced that exact refusal locally before correcting the mode.

This correction changes only the executable metadata; the script blob is byte-identical. The guard now passes across all 233 test entries, its own regression passes, the direct-invocation guard passes across 51 scripts, and all 12 policy-result controls, Actionlint, ShellCheck and whitespace checks pass again. Two own generated mutation scripts from the bounded local run were preserved outside the checkout; the source worktree is clean.

Fresh exact-head CI 39151727403 succeeded. The final native jobs response contains 19/19 terminal jobs, each successful or intentionally skipped. Created at 17:54:36 UTC, the run's required aggregate completed at 18:35:06: 40m30s feedback. Against the three comparable successful baseline runs on #4569 (median 56m49s), this is one observation 16m19s / 28.7% shorter, not a repeated post-change median.

Manifest validation occupied 23m09s (18:11:51–18:35:00), while the independent complete policy job occupied 15m44s (18:11:51–18:27:35). Both started together after detection, directly observing the intended overlap. The policy block still contains all 71 original command lines. Summed occupied job time was 73m05s, versus baseline median 72m04s: there is no runner-time or billed-cost saving claim.

Earlier component results remain observations of their earlier heads. The narrow source prerequisite has landed through #5048: fresh native main and origin/main both resolve to 5e2221baa85a526d212700f6dbf829bcc91a7701, and all five #5063 source files are byte-identical there. Its owner retains #5062/#5063 closure and active recovery acceptance.

Full current-head Codex review requested in comment 6253842625 completed clean (receipt 6253879978). CodeRabbit's native included-quota refusal on #5052 at 18:46 UTC named a 32-minute retry interval, establishing the ordered fallback without paid billing. Fresh native readback found all 37 visible checks successful or intentionally skipped, CLEAN/MERGEABLE, an empty complete thread census and valid signatures on all six commits. The readiness/user-evaluation receipt records the full proof. The validated draft is being promoted; production enqueueing remains coordinated with the active recovery owner. Final gate readback, protected deployment and merged-source verification remain mandatory. No merge is claimed.

@devantler

Copy link
Copy Markdown
Contributor Author

🤖 Generated by the Agentic Engineer

@codex review for Check byte-preserved policy coverage, both event/path states, selected-skip and unknown-result refusal, unchanged production admission, executable entry point and least privilege.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 18, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-18T19:02:30.551199Z 9c1bdc7 Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@devantler devantler removed the blocked label Oct 18, 2026
@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Keep it up!

Reviewed commit: 9c1bdc7c7c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@devantler

Copy link
Copy Markdown
Contributor Author

🤖 Generated by the Agentic Engineer

Readiness at exact head 9c1bdc7c7c44caeef288c12f1bf9440482462ae4:

  • Full exact-head CI 39151727403 succeeded, with all 19/19 native workflow jobs terminal. All 37 visible checks are successful or intentionally skipped. Native merge state is CLEAN/MERGEABLE and the complete thread census is empty. All six commits have valid GitHub-verified signatures from the registered identity.
  • Codex's completed review names this exact head and reports no findings, after authenticated request 6253842625. The preceding review had completed before this request. CodeRabbit's current included-quota refusal established the ordered fallback; no paid provider or gate exemption was used.
  • User evaluation observes the actual hosted policy job and manifest validation starting together at 18:11:51 UTC and both succeeding. The independent policy job took 15m44s, manifest validation 23m09s, and the aggregate completed at 40m30s. All 71 original command lines are retained. The actual aggregate guard also passed all 12 local event/selection/result controls on current Bash and system Bash 3.2; all 31 existing deployment/consumer controls are unchanged and pass. The executable-entry correction passed the native test-entry guard.

This is one successful elapsed-time observation versus baseline median 56m49s, not a repeated post-change median or a runner-cost saving. Parent #4569 remains open for the comparable post-change cohort and broader remaining families.

The source prerequisite is delivered via #5048: all five #5063 files are byte-identical on current main 5e2221baa85a526d212700f6dbf829bcc91a7701. Its stale source blocker/dependency has been removed with the resolution retained in this receipt and the implementation comment.

Promoting this validated draft is not a merge receipt. Production enqueueing remains coordinated with the active recovery owner to avoid delaying that work. Protected queue/deployment checks, a fresh final gate readback and merged-source verification remain mandatory.

@devantler
devantler marked this pull request as ready for review October 18, 2026 19:04
@devantler devantler removed the blocked label Oct 18, 2026
@devantler
devantler added this pull request to the merge queue Oct 18, 2026
Merged via the queue into main with commit 119968c Oct 18, 2026
37 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: ✅ Done

Development

Successfully merging this pull request may close these issues.

Run complete cluster-policy validation independently

1 participant