Skip to content

Add per-checkout data-only acknowledgement for ref warnings - #76806

Merged
pelikhan merged 7 commits into
mainfrom
copilot/checkout-warning-tooling-data-repositories
Nov 11, 2026
Merged

pelikhan merged 7 commits into
mainfrom
copilot/checkout-warning-tooling-data-repositories

Conversation

Copilot AI commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

The checkout-ref heuristic warns whenever host steps follow a dynamic checkout, even when trusted tooling consumes that checkout only as data.

  • Scoped acknowledgement: Add literal boolean data-only, defaulting to false, to suppress only the marked checkout’s ref warning. Every merged request for the same target must opt in.
  • Unchanged execution: The option does not change checkout behavior, credentials, permissions, or other diagnostics. Documentation clarifies that it acknowledges intent—not enforced isolation.
checkout:
  - path: tooling
    ref: ${{ github.event.repository.default_branch }}
  - repository: example-org/content
    ref: ${{ needs.prepare.outputs.branch }}
    path: target
    data-only: true
steps:
  - working-directory: tooling
    run: ./scripts/validate-content ../target

Host tooling must not execute scripts, hooks, plugins, or configuration from the data checkout.

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix checkout warning for tooling and data repositories Add per-checkout data-only acknowledgement for ref warnings Nov 11, 2026
Copilot AI requested a review from pelikhan November 11, 2026 08:55
@pelikhan
pelikhan marked this pull request as ready for review November 11, 2026 09:07
Copilot AI dynamic review requested due to automatic review settings November 11, 2026 09:07
@github-actions

This comment has been minimized.

@github-actions

github-actions Bot commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

✅ Test Quality Sentinel completed test quality analysis.

Test Quality Sentinel

Outcome: reported_no_action

The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible.

Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.

Generated from Test Quality Sentinel · routed: haiku55 xhigh · ~1 AIC · ⌖ ~2 AIC · ◇ ~1 AIC · ⊞ 7.2K · ◷

🧪 Test quality analysis by Test Quality Sentinel

@github-actions

github-actions Bot commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

✅ Ponytail Reviewer completed successfully!

Ponytail Reviewer

Outcome: reported_no_action

The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible.

Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.

Generated from Ponytail Reviewer · routed: gpt61 medium · ~4 AIC · ⌖ ~1 AIC · ◇ ~1 AIC · ⊞ 181 · ◷

Generated by Ponytail Reviewer for #76806

@github-actions

github-actions Bot commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

✅ PR Code Quality Reviewer completed the code quality review.

PR Code Quality Reviewer

Outcome: reported_action

The agent completed and reported one or more actions. The no-op output is a completion signal, not a claim that no action was taken.

Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.

Generated from PR Code Quality Reviewer · gpt56 · ~3 AIC · ⌖ ~1 AIC · ◇ ~1 AIC · ⊞ 1.6K · ◷

🔎 Code quality review by PR Code Quality Reviewer

@github-actions

github-actions Bot commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

🧠 Matt Pocock Skills Reviewer has completed the skills-based review. ✅

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer

@github-actions

github-actions Bot commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

✅ PR Specification Reviewer completed successfully!

Generated by PR Specification Reviewer for #76806

@github-actions

Copy link
Copy Markdown
Contributor

🏗️ Design Decision Gate — ADR Required

This PR makes significant changes to core business logic (160 added lines under pkg/, above the 100-line threshold) and does not link an Architecture Decision Record (ADR).

📄 Draft ADR committed: docs/adr/76806-data-only-checkout-acknowledgement.md. Review and complete it before merging.

🔒 This PR cannot merge until an ADR is linked in the PR body.

📋 What to do next
  1. Review the draft ADR committed to this PR branch. It was generated from the PR diff.

  2. Complete the sections. Add context the gate could not infer, confirm the rationale, and list the alternatives that were actually considered.

  3. Commit the finalized ADR to docs/adr/ on this branch.

  4. Reference the ADR in the PR body, for example:

    ADR: ADR-76806: Data-Only Checkout Acknowledgement for Checkout-Ref Warnings (link unavailable: invalid HTTPS URL)

Once an ADR is linked in the PR body, this gate re-runs and verifies that the implementation matches the decision.

🔎 Evidence used

References: PR #76806 · workflow run 42992498557

🏗️ ADR gate enforced by Design Decision Gate 🏗️ · pi · routed: haiku55 xhigh · ~3 AIC · ⌖ ~2 AIC · ⊞ 8.5K · ◷
Comment /review to run again

@github-actions

Copy link
Copy Markdown
Contributor

Comment Memory

Peek at saved memory (pr-code-quality-reviewer)
reviewed_at: 2026-11-11T09:11:39Z
review_event: REQUEST_CHANGES
top_themes:
  - data-only bypasses the checkout warning for the default workspace checkout
  - warning text weakens explicit guidance against host checkout of untrusted refs
files_reviewed:
  - pkg/workflow/checkout_ref_warning.go
  - pkg/workflow/checkout_manager.go
  - pkg/workflow/checkout_config_parser.go
  - pkg/workflow/checkout_ref_warning_test.go
  - pkg/parser/schemas/main_workflow_schema.json
  - docs/src/content/docs/reference/checkout.md
comment_count: 2

Note

This comment is managed by comment memory.

Expand the saved memory block to view or edit the persistent context for this thread.
Edit only the text inside the backtick fences; workflow metadata and the footer are regenerated automatically.

Learn more about comment memory

🔎 Code quality review by PR Code Quality Reviewer · copilot · gpt50 · ~3 AIC · ⌖ ~1 AIC · ⊞ 1.6K · ◷
Comment /review to run again

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔎 Code quality review by PR Code Quality Reviewer · copilot · gpt50 · ~3 AIC · ⌖ ~1 AIC · ⊞ 1.6K
Comment /review to run again

Comment thread pkg/workflow/checkout_ref_warning.go Outdated
"checkout step %q uses checkout.ref %q with an untrusted or unparseable expression, and host setup steps run after it. "+
"Imported or workflow-owned steps and pre-agent-steps execute outside the sandbox and may run checked-out code. "+
"Keep the agent workspace on a trusted ref and inspect fetched untrusted refs only as data; do not check them out or run their scripts on the host. "+
"Keep executable host tooling on a trusted ref and inspect untrusted refs only as data; do not run their scripts on the host. "+

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The revised warning removes the explicit guidance not to check untrusted refs out on the host. The new wording only warns against running scripts, but checkout itself can expose repo-controlled configuration, hooks, plugins, or tooling to host setup. Please retain the original "do not check them out or run their scripts on the host" guidance for the normal warning, then add the data-only acknowledgement as a separate, explicitly self-attested exception.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Restored the explicit host-workspace checkout warning and kept data-only as a separate, self-attested exception. Commit: 4a12457.

Comment thread pkg/workflow/checkout_ref_warning.go Outdated
continue
}
if !checkoutRefMayBeUntrusted(entry.key.repository, entry.ref) {
if entry.dataOnly || !checkoutRefMayBeUntrusted(entry.key.repository, entry.ref) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

data-only currently suppresses the warning for the default workspace checkout as well as separate data checkouts. A workflow can therefore set checkout: {ref: ${{ needs.plan.outputs.branch }}, data-only: true} while imported or pre-agent host steps execute in that same workspace, bypassing the guard this warning provides and contradicting the documented instruction not to switch the host workspace to an untrusted ref. Please reject or ignore data-only for the workspace-root/default checkout, and add a regression test that still expects the warning there.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Restricted warning suppression to separate non-root checkout paths; default and other workspace-root checkouts still warn. Added regression coverage. Commit: 4a12457.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Found two blocking correctness/documentation issues: the acknowledgement bypasses the warning for the default workspace checkout, and the revised warning weakens the explicit host-checkout safety guidance. The specialist pass completed; findings were independently adjudicated. Static review only; tests were not run per review instructions.

🔎 Code quality review by PR Code Quality Reviewer · copilot · gpt50 · ~3 AIC · ⌖ ~1 AIC · ⊞ 1.6K
Comment /review to run again

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed PR head fcc347a against Checkout Behavior Specification v1.3.4. The intentional data-only behavior is aligned with implementation and tests. Synchronized docs/src/content/docs/specs/checkout-behavior-specification.md to v1.4.0 (2026-11-11): added literal boolean/default semantics, warning-only scope, conservative merge rule, T-CHK-022, checklist coverage, and changelog entry. Specification commit publication is queued via safe output; no blocking findings.

Generated by PR Specification Reviewer for #76806 · copilot · gpt50 · ~5 AIC · ⌖ ~2 AIC · ⊞ 1.6K
Comment /review-specs to run again

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Skills-Based Review 🧠

Applied /grill-with-docs (wording and vocabulary) and /codebase-design (generated docs and schema consistency). Requesting changes on two items. The core logic looks correct.

📋 Key Themes & Highlights

Key Themes

  • Dropped safety guidance: the warning no longer says not to check out untrusted refs on the host, which is the main hazard the warning exists to flag.
  • Generated docs out of sync: frontmatter-full.md is generated from the schema by scripts/generate-schema-docs.js and has no data-only entry.
  • Default checkout suppression: data-only on the default workspace checkout silences the warning even when imported host steps run in that workspace. This matches the stated author-acknowledgement model, but it is worth confirming that is the intent.

Positive Highlights

  • ✅ Merge semantics are correct: dataOnly is ANDed across merged entries, so any request without the flag keeps the warning.
  • ✅ The option is literal-boolean only, and the parser test covers strings, expressions, numbers, and null.
  • ✅ The DataOnly field is excluded from JSON, which is safe because CheckoutConfig is never marshalled. Imported checkouts go through ParseCheckoutConfigs, so the flag survives import.
  • ✅ Targeted tests pass in pkg/workflow (CheckoutRefWithSteps|CheckoutDataOnly) and pkg/parser (CheckoutDataOnly).

Notes

  • The pr-triage step was not run separately. Change type and skill selection were made directly from the diff.

@copilot please address the review comments above.

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer · copilot · routed: haiku55 xhigh · ~5 AIC · ⌖ ~2 AIC · ⊞ 6.6K
Comment /matt to run again

Comment thread pkg/workflow/checkout_ref_warning.go Outdated
"checkout step %q uses checkout.ref %q with an untrusted or unparseable expression, and host setup steps run after it. "+
"Imported or workflow-owned steps and pre-agent-steps execute outside the sandbox and may run checked-out code. "+
"Keep the agent workspace on a trusted ref and inspect fetched untrusted refs only as data; do not check them out or run their scripts on the host. "+
"Keep executable host tooling on a trusted ref and inspect untrusted refs only as data; do not run their scripts on the host. "+

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[/grill-with-docs] The rewritten warning drops the instruction not to check out untrusted refs on the host. The old text said "do not check them out or run their scripts on the host", and that is the core risk: a manual checkout can run Git hooks, which the reference docs already call out. The new text also replaces "agent workspace" with "executable host tooling", but the docs still say agent workspace, so the vocabulary now drifts.

💡 Suggested wording

Restore the removed clause and keep the original terms, for example: "Keep the agent workspace and executable host tooling on a trusted ref, inspect untrusted refs only as data, and do not check them out or run their scripts on the host."

@copilot please address this.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Restored the agent-workspace and no-checkout/no-script guidance, with separate self-attested data-only wording. Commit: 4a12457.

"description": "Configuration for a single actions/checkout step",
"additionalProperties": false,
"properties": {
"data-only": {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[/codebase-design] This schema change is not reflected in the generated reference docs. docs/src/content/docs/reference/frontmatter-full.md is generated from this schema by scripts/generate-schema-docs.js, and it currently has no data-only entry. Regenerate it so the published reference matches the schema.

💡 Command
make generate-schema-docs

@copilot please address this.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated the schema description and regenerated frontmatter-full.md from the schema. Commit: 4a12457.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Ref-warning checks run before path derivation, allowing an untrusted-ref warning to be suppressed even when merged checkout requests lack unanimous data-only opt-in.

2 open issues
What changed in this PR

Adds a per-checkout data-only acknowledgement to gh-aw’s ref-warning heuristic, supporting workflows where trusted host tooling reads dynamic checkouts only as data.

Changes:

  • Accepts literal booleans and requires every merged checkout request to opt in.
  • Suppresses the marked checkout’s ref warning without changing checkout execution.
  • Documents the acknowledgement’s limits and adds validation, merge, and warning tests.
File Description
pkg/​workflow/​checkout_ref_warning.go Applies scoped suppression and updates warning guidance.
pkg/​workflow/​checkout_ref_warning_test.go Tests suppression, merging, parsing, and unchanged generated steps.
pkg/​workflow/​checkout_manager.go Stores the acknowledgement and combines merged requests conservatively.
pkg/​workflow/​checkout_config_parser.go Parses data-only and rejects non-boolean values.
pkg/​parser/​schemas/​main_workflow_schema.json Defines the option’s type, default, and semantics.
pkg/​parser/​schema_checkout_data_only_test.go Tests schema validation for object and list forms.
docs/​src/​content/​docs/​reference/​checkout.md Documents tooling/data separation and safety limitations.

Review effort: Max (auto)

Comment thread pkg/workflow/checkout_ref_warning.go Outdated
continue
}
if !checkoutRefMayBeUntrusted(entry.key.repository, entry.ref) {
if entry.dataOnly || !checkoutRefMayBeUntrusted(entry.key.repository, entry.ref) {

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved the ref-warning check after derived checkout path processing and added coverage for an implicit path merging with an explicit path in 4a12457.

| `max-repos` | integer | Maximum unique repositories per `multi: true` checkout entry. Requires `allowed-repos` in multi mode; defaults to `10`. Must be a positive literal integer. Ignored in single-repository mode. |
| `allow-private-repos` | boolean | With `multi: true`, allow private/internal targets in a runtime list hosted by a private/internal repository. Requires `allowed-repos` in multi mode; defaults to `false` and must be a literal boolean. Public hosts always reject non-public multi targets, even when `true`. Ignored in single-repository mode. |
| `ref` | string | Branch, tag, or SHA to checkout. Defaults to the triggering ref. |
| `data-only` | boolean | Acknowledge data-only consumption by host steps and suppress only this checkout's checkout-ref warning. Defaults to `false`; literal boolean only. Does not enforce isolation or change execution. All merged requests for the target must opt in. See [Separate Tooling and Data Checkouts](#separate-tooling-and-data-checkouts). |

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added data-only to the checkout syntax reference with its literal-boolean, default, merge, suppression, and acknowledgement semantics in 4a12457.

@gh-aw-bot

Copy link
Copy Markdown
Collaborator

@copilot

🍳 A little prep to help this PR along! Copilot assignment requested for the following outstanding work:

  1. Update this branch with the latest main using make merge-main, resolving any conflicts and preserving the intended changes.
  2. Review (pkg/workflow/checkout_ref_warning.go:36): address unresolved feedback - Review comment
  3. Review (pkg/workflow/checkout_ref_warning.go:30): address unresolved feedback - Review comment
  4. Review (pkg/workflow/checkout_ref_warning.go:36): address unresolved feedback - Review comment
  5. Review (pkg/parser/schemas/main_workflow_schema.json:17488): address unresolved feedback - Review comment

Keeping the changes focused; human review and CI get the final taste test. No merging or CI triggering.

Structured data
{
  "completed_task": 0,
  "head": "3982e84057a5839c2974553f782d20a954974b1d",
  "kind": "requested",
  "schema_version": "1",
  "state": "92ffda3623892d22b8d53c8542beabadd7f5c2f0c50931c1d79350880cfbfba0",
  "work": [
    "b3c292a66779b78b9bfec3bf5bf81bda5aababfbd8a0d1b6254e825ed72e3eda",
    "d1162f29093f0939e36e9146b1d66d48cf6a6e62bdd578a830a1124f3cc9ec98",
    "d642cf19c2d25456ada72f1f6b98bc76964b388aa26ef3a25a62ac1fd901f63d",
    "fca2f0c8bb46e9f667213072a921c9cc4a57d3d6a00af90459bc09301e78ce38"
  ],
  "workflow": "pr-sous-chef"
}

Generated by 👨🍳 PR Sous Chef · pi · routed: haiku55 xhigh · ~4 AIC · ⌖ ~2 AIC · ⊞ 6.9K · ◷
Comment /souschef to run again

Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>
Copilot AI and others added 2 commits November 11, 2026 09:49
…ing-tooling-data-repositories

Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>
Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>
@github-actions

Copy link
Copy Markdown
Contributor

Category: bug | Risk: medium | Score: 69/100 (impact 33 + urgency 20 + quality 16). Recommended action: auto_merge. Batch: pr-batch:copilot-runtime. Reason: narrow safety improvement for checkout warnings, with clear behavior and passing CI.

Generated by 🔧 PR Triage Agent · copilot · mai11 · ~5 AIC · ⌖ ~2 AIC · ⊞ 15.9K · ◷

@gh-aw-bot

Copy link
Copy Markdown
Collaborator

@copilot

🍳 A little prep to help this PR along! Copilot assignment requested for the following outstanding work:

  1. Update this branch with the latest main using make merge-main, resolving any conflicts and preserving the intended changes.
  2. Fix failing check Analyze (python) (FAILURE): Check run.
  3. Fix failing check impacted-go-tests (FAILURE): Check run.

Keeping the changes focused; human review and CI get the final taste test. No merging or CI triggering.

Structured data
{
  "completed_task": 0,
  "head": "4a124570f1110860b172f3738841aacfb7ad9fdf",
  "kind": "requested",
  "schema_version": "1",
  "state": "c3dd0f49c8b8dfb2020a7227a0c1b426d8eb30bdadcd2e9727c99e7222a1f8fc",
  "work": [
    "42e5d572d9d75b7168fe7d0aeda0717cc07e65ed240f6d299e6fec661214f1ac",
    "43282e7f5b3023b8c8cf9099eab79e7b99f650965f08b47d3df33cde5589879d"
  ],
  "workflow": "pr-sous-chef"
}

Generated by 👨🍳 PR Sous Chef · pi · routed: haiku55 xhigh · ~2 AIC · ⌖ ~2 AIC · ⊞ 6.9K · ◷
Comment /souschef to run again

…ing-tooling-data-repositories

Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>

Copilot AI commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor Author

@copilot

🍳 A little prep to help this PR along! Copilot assignment requested for the following outstanding work:...

Merged latest main in 993e9d3. The Analyze (python) job failed while GitHub CodeQL reported an installation API rate limit; impacted-go-tests timed out while running TestAzureDevOpsConnectorBundledNodeResponseIdentity, which passed when rerun with the checkout regressions locally. The final local gate's custom linters flagged existing checkout-manager function-length/indexing findings; no unrelated refactor was made.

@copilot

🍳 A little prep to help this PR along! Copilot assignment requested for the following outstanding work:...

Merged latest main in 993e9d3. The ref-warning ordering/regression coverage and syntax-reference documentation are in 4a12457.

@pelikhan
pelikhan merged commit e67faf8 into main Nov 11, 2026
51 checks passed
@pelikhan
pelikhan deleted the copilot/checkout-warning-tooling-data-repositories branch November 11, 2026 15:20
@github-actions

Copy link
Copy Markdown
Contributor

🎉 This pull request is included in a new release.

Release: v1.2.3

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Checkout warning cannot distinguish tooling and data repositories

4 participants