Skip to content

feat(rulesets): require the shared .NET tests on dotnet-template - #507

Merged
devantler merged 1 commit into
mainfrom
claude/dotnet-template-tests-ruleset-506
Oct 5, 2026
Merged

devantler merged 1 commit into
mainfrom
claude/dotnet-template-tests-ruleset-506

Conversation

@devantler

Copy link
Copy Markdown
Contributor

🤖 Generated by the Agentic Engineer

Why

The .NET template's pull requests run no build or test check on any platform, although its own instructions say one exists. The request to add it by hand (devantler-tech/dotnet-template issue 424) has waited three weeks, and organization rules can now be declared here instead, as was done for the Go template.

What

Adds a rule that applies only to the .NET template and requires the shared .NET tests on Ubuntu, Windows and macOS, taken from this repository's reviewed main branch. The existing hand-managed rule is left untouched, and nothing changes for any other repository.

Fixes #506

👉 After merge/promotion: the rule takes effect with the next release of this repository; the next .NET template pull request then shows the test checks. Merging this is the decision to add the gate — close it instead to keep the template on its lighter checks.

Adds a managed organization rule that targets only dotnet-template and
requires run-dotnet-tests.yaml from this repository's reviewed main, with
no bypass. A render test pins the target, the workflow and the absence of
a bypass; the managed inventory grows from five to six.

Fixes #506

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@devantler

Copy link
Copy Markdown
Contributor Author

🤖 Generated by the Agentic Engineer

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Oct 5, 2026 •

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@github-code-quality

github-code-quality Bot commented Oct 5, 2026 •

Copy link
Copy Markdown

Code Coverage Overview

Languages: Go, C#

Go / code-coverage/test

The overall line coverage in commit bb4cd58 in the claude/dotnet-templa... branch remains at 50%, unchanged from commit 2f56183 in the main branch.

C# / code-coverage/dotnet

The overall line coverage in commit bb4cd58 in the claude/dotnet-templa... branch remains at 100%, unchanged from commit 2f56183 in the main branch.


Updated October 05, 2026 22:38 UTC

@devantler

Copy link
Copy Markdown
Contributor Author

🤖 Generated by the Agentic Engineer

Evaluation at bb4cd5887c78 (local, macOS, /bin/bash 3.2).

What I ran Result
tests/dotnet-template-tests-ruleset.sh on the rendered deploy/ tree passes
Same test with the target repository id changed fails
Same test with the workflow path changed fails
Same test with enforcement set to evaluate fails
Same test with a bypass actor added fails
tests/world-at-ruin-regression-ruleset.sh (managed inventory, now six) passes
go-template-validation, ci-aggregate, deploy-guards, platform-merge-queue ruleset tests, workflow-execution-inventory, deploy-deletions all pass

What the rule will run. The shared .NET tests were exercised on the template itself in devantler-tech/dotnet-template#477: run 37363379803 passed on Ubuntu, Windows and macOS. The target id 825979604 was read from the repository API, and the workflow declares the pull_request and merge_group triggers a required workflow needs.

What this could not exercise. The rule has no effect until this repository's next release applies it, so there is nothing to run before merge. The trial called the workflow from a caller; the rule starts it directly, with no inputs. The first .NET template pull request after the release is the first observation of that shape, and devantler-tech/dotnet-template#424 stays open until it is seen.

@coderabbitai

coderabbitai Bot commented Oct 5, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Repository YAML (base), Organization UI (inherited)
  • Review profile: ASSERTIVE
  • Plan: Advanced
  • Run ID: 18a70a7c-7209-4ed7-bc4a-0fbeaaf8ff0d
📥 Commits

Reviewing files that changed from the base of the PR and between 2f56183 and bb4cd58.

📒 Files selected for processing (7)
  • .github/workflows/ci.yaml
  • deploy/README.md
  • deploy/organization-rulesets/README.md
  • deploy/organization-rulesets/kustomization.yaml
  • deploy/organization-rulesets/require-dotnet-template-tests.yaml
  • tests/dotnet-template-tests-ruleset.sh
  • tests/world-at-ruin-regression-ruleset.sh

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.

📜 Recent review details
⏰ Context from checks skipped due to timeout. (74)
  • GitHub Check: zizmor
  • GitHub Check: zizmor
  • GitHub Check: [Test] Lint - Passes On Clean Fixture / 🧹 Lint - mega-linter
  • GitHub Check: [Test] Lint - Manual Workflow Fixes Opted In / 🧹 Lint - mega-linter
  • GitHub Check: [Test] Enable Auto-Merge - Actor Trust Enabled / dry-run
  • GitHub Check: [Test] Validate Go Project - Default-Branch Test Opted Out / 🔍 Detect Changes
  • GitHub Check: [Test] Enable Auto-Merge - Actor Trust Enabled / eligibility
  • GitHub Check: [Test] Enable Auto-Merge / dry-run
  • GitHub Check: [Test] Validate Go Project - Manual Workflow Fixes Opted In / 🔍 Detect Changes
  • GitHub Check: [Test] Enable Auto-Merge / eligibility
  • GitHub Check: [Test] Validate Go Project - Default-Branch Maintenance Opted In / 🔍 Detect Changes
  • GitHub Check: [Test] Enable Auto-Merge - Pending Queue Enabled / dry-run
  • GitHub Check: [Test] Enable Auto-Merge - Pending Queue Enabled / eligibility
  • GitHub Check: [Test] Validate Go Project - Signed Fixes Suppressed / 🔍 Detect Changes
  • GitHub Check: [Test] Apply Signed Fixes - Verifies The Tip With No Patch / 📤 Apply signed fixes
  • GitHub Check: [Test] Validate Go Project / 🔍 Detect Changes
  • GitHub Check: [Test] Validate Go Project - Lint Gate Blocks
  • GitHub Check: [Test] Lint - Gate Pins Self-Tested Image
  • GitHub Check: [Test] Validate Go Project - Gate Lockstep
  • GitHub Check: 🧹 Validate manifests
  • GitHub Check: [Test] Scan for TODO Comments - Dry Run (vendor exclusion) / dry-run
  • GitHub Check: [Test] Create Release - No Issue Side Effects / Offline release decisions
  • GitHub Check: [Test] Scan for TODO Comments - Reject Missing Project Token / todos
  • GitHub Check: [Test] Scan for TODO Comments - Dry Run (ignore configured) / dry-run
  • GitHub Check: [Test] Run .NET Tests - Authenticated Packages / test (ubuntu-latest)
  • GitHub Check: [Test] Run .NET Tests / test (ubuntu-latest)
  • GitHub Check: [Test] Scan for TODO Comments - Production Steps Without a Project / todos
  • GitHub Check: [Test] Delete Workflow Runs - Minimal Parameters / Delete workflow runs
  • GitHub Check: [Test] Delete Workflow Runs - All Workflows / Delete workflow runs
  • GitHub Check: [Test] Create Release - Dry Run / Offline release decisions
  • GitHub Check: [Test] Scan for TODO Comments - Production Compatibility Defaults / todos
  • GitHub Check: [Test] Sync Cluster Policies - Allowlist Miss Fails Closed
  • GitHub Check: [Test] Dependency Review - Default Settings / dependency-review
  • GitHub Check: [Test] Go Suite - Default-Branch Trigger Coverage
  • GitHub Check: [Test] Delete Workflow Runs - Specific Pattern / Delete workflow runs
  • GitHub Check: [Test] Govulncheck Trigger Guard - Blocks On Bad Input
  • GitHub Check: [Test] Go Filter Nested Modules
  • GitHub Check: [Test] Govulncheck Allowlist - Gate Uses Self-Tested Implementation
  • GitHub Check: [Test] Govulncheck - Default-Branch & Allowlist Trigger Coverage
  • GitHub Check: [Test] Go Suite Trigger Guard - Blocks On Bad Input
  • GitHub Check: [Test] Govulncheck - Timeout & Memory Safety Controls
  • GitHub Check: [Test] Release PR Lifecycle Guard
  • GitHub Check: [Test] Enable Auto-Merge - Pentad Gate
  • GitHub Check: [Test] Go Filter Nested Modules - Blocks On Bad Input
  • GitHub Check: [Test] Enable Auto-Merge - Author Gate
  • GitHub Check: [Test] Create Release - Least Privilege
  • GitHub Check: [Test] Approve PR - Credential-Free CI
  • GitHub Check: test-enable-auto-merge-on-pr
  • GitHub Check: test-ensure-gh-skill-script
  • GitHub Check: [Test] Dependency Review - Summary Comment (comment-updated)
  • GitHub Check: test-update-agent-skills-mark-internal (ubuntu-latest)
  • GitHub Check: [Test] Dependency Review - Summary Comment (comment-rejected)
  • GitHub Check: test-aggregate-job-checks-unknown
  • GitHub Check: [Test] Dependency Review - Summary Comment (compare-forbidden)
  • GitHub Check: [Test] Dependency Review - Summary Comment (on-failure-vulnerable)
  • GitHub Check: [Test] Dependency Review - Summary Comment (comment-created)
  • GitHub Check: [Test] Dependency Review - Summary Comment (comment-never)
  • GitHub Check: [Test] Dependency Review - Summary Comment (on-failure-clean)
  • GitHub Check: lint-readme-parity
  • GitHub Check: [Lint] Shell Pipeline Assertions
  • GitHub Check: [Test] TODO Scanner - Isolated Image
  • GitHub Check: [Test] Run .NET Tests - Microsoft.Testing.Platform (ubuntu-latest)
  • GitHub Check: [Test] Create Issues From TODOs - Offline Wrapper
  • GitHub Check: catalogue-scope
  • GitHub Check: [Test] Dependency Review - Summary Comment (on-failure-warned)
  • GitHub Check: [Test] Dependency Review - Summary Comment (comments-unreadable)
  • GitHub Check: [Test] Upsert Issue - Offline Behavior
  • GitHub Check: test-setup-ksail-cli (ubuntu-latest)
  • GitHub Check: lint-ci-coverage-parity
  • GitHub Check: [Test] Enable Auto-Merge on PR - Credential-Free CI
  • GitHub Check: eligibility
  • GitHub Check: Analyze (go)
  • GitHub Check: Analyze (actions)
  • GitHub Check: Analyze (go)
🧰 Additional context used
📚 Code guidelines (1)
AGENTS.md — auto-discovered
📓 Path-based instructions (2)
Source excerpt: Every `.github/tests/test-*.sh` is a test entrypoint and must have an explicit invocation in a `ci.yaml` step's `run:` block.

📄 CodeRabbit inference engine (AGENTS.md)

Files:

  • .github/workflows/ci.yaml
Source excerpt: **Manage GitHub declaratively — never imperatively.**

📄 CodeRabbit inference engine (AGENTS.md)

Files:

  • deploy/README.md
  • deploy/organization-rulesets/kustomization.yaml
  • deploy/organization-rulesets/README.md
  • deploy/organization-rulesets/require-dotnet-template-tests.yaml
🔇 Additional comments (7)
deploy/organization-rulesets/require-dotnet-template-tests.yaml (1)

19-33: LGTM!

deploy/organization-rulesets/kustomization.yaml (1)

3-3: LGTM!

Also applies to: 27-27

deploy/organization-rulesets/README.md (1)

47-47: LGTM!

deploy/README.md (1)

44-44: LGTM!

Also applies to: 46-46

tests/dotnet-template-tests-ruleset.sh (1)

1-24: LGTM!

tests/world-at-ruin-regression-ruleset.sh (1)

66-67: LGTM!

Also applies to: 69-70

.github/workflows/ci.yaml (1)

5901-5903: LGTM!


📝 Walkthrough

Walkthrough

The change adds an active organization ruleset for the .NET template’s default branch. The ruleset requires the shared .NET tests workflow from the reviewed main branch. Kustomize and deployment documentation now include the ruleset. A new test checks its rendered configuration, and the manifest validation job runs that test. The existing ruleset regression test now expects six managed rulesets.

Priority: ➖ Normal

Severity of issue fixed: Medium

Merge Risk: ⚪ Minimal · up to bb4cd

No actionable issue is identified before merge. Confirm the effective rule after the publishing release as planned.

Security Architecture Review

Security architecture risk: 🟡 Moderate · up to bb4cd

The new gate references a different repository ID from the repository containing the reviewed workflow. This leaves the intended test enforcement and its security controls unconfirmed. Exposure is limited to the .NET template’s default branch, but reverting the deployment will not automatically remove an already-created gate.

Retained concerns

  • Medium · security · observed: The new required-workflow gate binds to repository 948529001, not the reviewed devantler-tech/.github repository identified as 933213756. Its validation asserts the same discrepant ID. Consequently, the reviewed workflow’s triggers and security controls do not establish the behavior of the workflow actually selected by this gate; rollout could fail to establish the intended control or enforce a different implementation, depending on the external repository’s contents.
Security review details

Security Blast Radius

  • observed — The new policy’s declared enforcement scope is one repository’s default branch. Its rendered-validation scripts run in an existing contents-read CI job and do not introduce a live policy-write operation. Actual exposure through the selected workflow source remains unverified.

Security Findings and Attack Paths

  • observed — The source-identity mismatch is an introduced enforcement-contract concern. The available evidence does not establish who controls repository 948529001, whether its named workflow exists, or whether an attacker can modify it; an exploitable attack path is not verified.

Trust Boundaries and Controls

  • inferred — The intended boundary separates candidate code from the workflow definition that judges it by selecting a separate repository’s main branch. Its effectiveness depends on selecting the correct trusted owner; a path and branch name alone do not establish that ownership.

Hardening Proposals

  • proposed — Before promotion, verify the selected workflow repository’s identity and trusted ownership independently of the literal manifest assertion. Preserve the created ruleset’s numeric external identity and define a declarative correction or disabled-state rollback that retains ownership rather than relying on manifest deletion.
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 2 functions across 2 files. (5 skipped: 5 … Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed [ #506 ] The managed rule targets repository 825979604, requires .github/workflows/run-dotnet-tests.yaml from repository 948529001 at refs/heads/main, and declares no Delete management policy. The…
Out of Scope Changes check ✅ Passed The rule manifest, Kustomize inventory, documentation counts, CI wiring, render test, and managed-ruleset count update all support the [#506] rule or its validation. No unrelated change is evident.
Title check ✅ Passed The title clearly identifies the main change: requiring the shared .NET tests for dotnet-template.
Description check ✅ Passed The description explains why the .NET test gate is needed and what repositories and workflows it affects.
Full details: Docstring Coverage

Explanation

Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 2 functions across 2 files. (5 skipped: 5 unsupported.)

  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@devantler
devantler marked this pull request as ready for review October 5, 2026 22:47
@devantler
devantler merged commit a87e338 into main Oct 5, 2026
271 checks passed
@devantler
devantler deleted the claude/dotnet-template-tests-ruleset-506 branch October 5, 2026 22:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Require the shared .NET tests on the .NET template with a managed rule

1 participant