Repository navigation
Show detector-defined threat categories in generated warnings - #75774
Conversation
|
❌ PR Specification Reviewer failed to deliver outputs. Please review the logs for details.
|
|
🧠 Matt Pocock Skills Reviewer has completed the skills-based review. ✅
|
|
✅ Design Decision Gate 🏗️ completed the design decision gate check. See the comment below for the result and any generated ADR draft. Design Decision Gate 🏗️Outcome: The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible. Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.
|
|
✅ Test Quality Sentinel completed test quality analysis. Test Quality SentinelOutcome: The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible. Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.
|
|
✅ PR Code Quality Reviewer completed the code quality review.
|
|
✅ Ponytail Reviewer completed successfully!
|
| @@ -553,6 +557,9 @@ async function main() { | |||
| * @param {string} sourceLabel | |||
| */ | |||
| function evaluateAndReportVerdict(verdict, sourceLabel) { | |||
| const threatKinds = STRUCTURED_OUTPUT_REQUIRED_FIELDS.filter(field => verdict[field] === true).join(","); | |||
There was a problem hiding this comment.
Review complete: no actionable issues found in the changed lines. Static analysis was limited to the pre-fetched diff and targeted source reads; the advisory specialist pass was unavailable because its tool invocation was rejected.
🔎 Code quality review by PR Code Quality Reviewer · copilot · gpt50 · ~3 AIC · ⌖ ~1 AIC · ⊞ 1.6K
Comment /review to run again
There was a problem hiding this comment.
Reviewed head 5fdbd49 against Safe Outputs MCP Gateway Specification v1.29.20, Section 10.5 WTD1 and template/publication behavior. The implementation and tests support the intentional category-only publication behavior, explicit unavailable-category messaging, and separation of tooling failures. Synchronized docs/src/content/docs/specs/safe-outputs-specification.md to v1.29.21 (2026-11-08), updating WTD1 wording, template placeholder semantics, and the changelog; the commit was queued for publication to the same-repository PR branch (output so_78D3oa_qSdya). No blocking conformance findings. Static review only; tests were not run by this reviewer.
Generated by PR Specification Reviewer for #75774 · copilot · gpt50 · ~5 AIC · ⌖ ~2 AIC · ⊞ 1.7K
Comment /review-specs to run again
There was a problem hiding this comment.
What this change does: It carries the three schema-defined threat categories through inline and external detection jobs into public warnings and report blocks. It also replaces the review-PR and completion-comment warning paths with the typed report renderer.
Should fix:
-
Keep one warning presentation path (
actions/setup/js/push_to_pull_request_branch.cjs:L35)- What this is: The PR removes the last runtime call to
getThreatWarningPresentationand usesgetDetectionWarningBlocksinstead. - Problem: The old helper is now referenced only by tests, so the repository keeps a second warning API that cannot affect production output.
- Fix: Delete
getThreatWarningPresentation, its export, and its tests, or keep a real caller if the object is still needed. - If we skip it: Future warning-copy changes must be checked against dead presentation code and can leave two descriptions out of sync.
- What this is: The PR removes the last runtime call to
-
Remove the unused string failure renderer (
actions/setup/js/messages_run_status.cjs:L111)- What this is:
getDetectionFailureBlocksis the new runtime path, whilegetDetectionFailureMessageremains beside it. - Problem: A repository search leaves the string helper in the preview and tests only; both helpers select the same template and construct the same context.
- Fix: Remove the string helper and its preview/tests, or derive the preview from the block renderer so there is one failure-message implementation.
- If we skip it: The two failure-message paths can drift while one of them is never used by workflows.
- What this is:
Verdict: fix 1 and 2 first.
Lean: -40 lines possible.
Generated by ✂️ Ponytail Reviewer for #75774 · codex · routed: gpt56 max · ~8 AIC · ⌖ ~1 AIC · ⊞ 118
Comment /ponytail to run again
| @@ -31,7 +31,10 @@ const gitHelpers = require("./git_helpers.cjs"); | |||
| const { ensureFullHistoryForBundle, importLocalBundle, offlineGitOptions, linearizeRangeAsCommit, ensureSafeDirectoryTrust } = gitHelpers; | |||
| const { extractPatchBaseCommit } = require("./commit_sha_helpers.cjs"); | |||
| const { findRepoCheckout } = require("./find_repo_checkout.cjs"); | |||
| const { getThreatWarningPresentation, getEffectiveDetectionConclusion } = require("./threat_detection_warning.cjs"); | |||
| const { getEffectiveDetectionConclusion } = require("./threat_detection_warning.cjs"); | |||
| const { getDetectionWarningBlocks } = require("./messages_run_status.cjs"); | |||
There was a problem hiding this comment.
Lean: Removing the call here leaves getThreatWarningPresentation with no production callers; rg finds it only in tests. Delete the helper/export and those tests so the template-backed getDetectionWarningBlocks path is the single warning presentation and cannot drift.
There was a problem hiding this comment.
Removed getThreatWarningPresentation, its export, and the remaining test/mocked usage. Template-backed warning rendering is now the sole presentation path. Commit 3e48d5b33b.
| * @param {DetectionFailureContext} ctx | ||
| * @returns {import("./report_ast.cjs").ReportBlock[]} | ||
| */ | ||
| function getDetectionFailureBlocks(ctx) { |
There was a problem hiding this comment.
Lean: This new block renderer leaves getDetectionFailureMessage used only by the preview/tests, while both functions select the same template and build the same context. Remove the unused string API and its preview/tests (or derive the preview from this renderer) to avoid maintaining two failure-message paths.
There was a problem hiding this comment.
Removed the unused getDetectionFailureMessage string API and its preview. Failure message coverage now exercises the typed report-block renderer. Commit 3e48d5b33b.
There was a problem hiding this comment.
🟡 Changes recommended
The test relies on GH_AW_DETECTION_THREAT_KINDS leaking from an earlier case, causing it to fail when run independently.
1 open issue
What changed in this PR
This PR propagates schema-defined threat categories (prompt_injection, secret_leak, and malicious_patch) into generated warnings and reviews, using framework-authored descriptions rather than detector-generated explanations.
Changes:
- Export threat categories and pass them to safe-output and conclusion jobs.
- Include category descriptions in warnings, tracking comments, blocked-output messages, and request-changes reviews.
- Update templates, documentation, regression assertions, and generated workflow wiring.
This pull request requires changes.
Outstanding review comments:
- moderate, 8 votes —
actions/setup/js/generate_footer.test.cjs:526: The assertion depends onGH_AW_DETECTION_THREAT_KINDSleaking from an earlier test and fails independently. Set the variable explicitly in this test. - nit, 3 votes —
actions/setup/js/parse_threat_detection_results.cjs:560: The export assertions are inside a skipped suite, so neither warn nor strict mode exercises category extraction/export. Add an enabled regression using a real result-file fixture or injectable filesystem. - nit, 3 votes —
actions/setup/js/parse_threat_detection_results.test.cjs:1007: The parameterized category-export regression and related assertions never execute. Move them into an active suite to cover the inline producer’s output and environment propagation. - nit, 1 vote —
actions/setup/js/safe_output_handler_manager.cjs:1112: Optional: extend the blocked-output test to verify the framework-authored category text and exclusion of unknown tokens, rather than checking onlyerrorCode. - nit, 2 votes —
docs/src/content/docs/specs/safe-outputs-specification.md:5948: The documenteddetection-warningsetting is unsupported by the schema and Go configuration, while the schema’sdetection-failuredescription still lists old placeholders and default text. Align the documentation and configuration support.
| File | Description |
|---|---|
pkg/workflow/threat_detection_job.go |
Exposes threat categories as a detection-job output. |
pkg/workflow/notify_comment_conclusion_helpers.go |
Passes categories to conclusion handlers. |
pkg/workflow/detection_success_test.go |
Checks category output and environment wiring. |
pkg/workflow/compiler_safe_outputs_envvars.go |
Passes categories to safe-output handlers. |
docs/src/content/docs/specs/safe-outputs-specification.md |
Documents category disclosure and message placeholders. |
actions/setup/md/threat_warning_request_changes_review.md |
Includes framework-authored threat details. |
actions/setup/md/detection_runs_issue.md |
Explains category reporting in tracking comments. |
actions/setup/js/threat_detection_warning.test.cjs |
Tests category allowlisting and descriptions. |
actions/setup/js/safe_output_handler_manager.cjs |
Uses framework-authored blocked-output descriptions. |
actions/setup/js/parse_threat_detection_results.test.cjs |
Adds category-export assertions within a skipped suite. |
actions/setup/js/parse_threat_detection_results.cjs |
Exports boolean findings and clears unavailable categories. |
actions/setup/js/messages_core.cjs |
Documents detection-message placeholders. |
actions/setup/js/issue_body_report_builder.test.cjs |
Checks trusted warnings remain separate from agent prose. |
actions/setup/js/handle_detection_runs.test.cjs |
Tests allowlisted categories in tracking comments. |
actions/setup/js/handle_detection_runs.cjs |
Includes category descriptions in tracked findings. |
actions/setup/js/generate_footer.test.cjs |
Adds expired-warning assertions with an environment-state dependency. |
actions/setup/js/create_pull_request.cjs |
Supplies safe details to request-changes reviews. |
.github/workflows/workflow-skill-extractor.lock.yml |
Generated category wiring. |
.github/workflows/workflow-normalizer.lock.yml |
Generated category wiring. |
.github/workflows/workflow-health-manager.lock.yml |
Generated category wiring. |
.github/workflows/workflow-generator.lock.yml |
Generated category wiring. |
.github/workflows/windows-grower.lock.yml |
Generated category wiring. |
.github/workflows/weekly-otel-reference.lock.yml |
Generated category wiring. |
.github/workflows/weekly-issue-summary.lock.yml |
Generated category wiring. |
.github/workflows/weekly-editors-health-check.lock.yml |
Generated category wiring. |
.github/workflows/weekly-blog-post-writer.lock.yml |
Generated category wiring. |
.github/workflows/visual-regression-checker.lock.yml |
Generated category wiring. |
.github/workflows/video-analyzer.lock.yml |
Generated category wiring. |
.github/workflows/update-astro.lock.yml |
Generated category wiring. |
.github/workflows/ubuntu-image-analyzer.lock.yml |
Generated category wiring. |
.github/workflows/typist.lock.yml |
Generated category wiring. |
.github/workflows/terminal-stylist.lock.yml |
Generated category wiring. |
.github/workflows/technical-doc-writer.lock.yml |
Generated category wiring. |
.github/workflows/super-linter.lock.yml |
Generated category wiring. |
.github/workflows/step-name-alignment.lock.yml |
Generated category wiring. |
.github/workflows/static-analysis-report.lock.yml |
Generated category wiring. |
.github/workflows/stale-pr-cleanup.lock.yml |
Generated category wiring. |
.github/workflows/squad-implement-worker.lock.yml |
Generated category wiring. |
.github/workflows/squad-game-planner.lock.yml |
Generated category wiring. |
.github/workflows/spec-librarian.lock.yml |
Generated category wiring. |
.github/workflows/spec-extractor.lock.yml |
Generated category wiring. |
.github/workflows/spec-enforcer.lock.yml |
Generated category wiring. |
.github/workflows/smoke-workflow-call.lock.yml |
Generated category wiring. |
.github/workflows/smoke-security-alerts.lock.yml |
Generated category wiring. |
.github/workflows/smoke-pi-routed.lock.yml |
Generated category wiring. |
.github/workflows/smoke-issues.lock.yml |
Generated category wiring. |
.github/workflows/smoke-factory.lock.yml |
Generated category wiring. |
.github/workflows/smoke-atomic.lock.yml |
Generated category wiring. |
.github/workflows/slide-deck-maintainer.lock.yml |
Generated category wiring. |
.github/workflows/sighthound-security-scan.lock.yml |
Generated category wiring. |
.github/workflows/sergo.lock.yml |
Generated category wiring. |
.github/workflows/semantic-function-refactor.lock.yml |
Generated category wiring. |
.github/workflows/security-compliance.lock.yml |
Generated category wiring. |
.github/workflows/schema-feature-coverage.lock.yml |
Generated category wiring. |
.github/workflows/schema-consistency-checker.lock.yml |
Generated category wiring. |
.github/workflows/safe-output-health.lock.yml |
Generated category wiring. |
.github/workflows/research.lock.yml |
Generated category wiring. |
.github/workflows/repository-quality-improver.lock.yml |
Generated category wiring. |
.github/workflows/repo-tree-map.lock.yml |
Generated category wiring. |
.github/workflows/refiner.lock.yml |
Generated category wiring. |
.github/workflows/refactoring-cadence.lock.yml |
Generated category wiring. |
.github/workflows/python-data-charts.lock.yml |
Generated category wiring. |
.github/workflows/purelock.lock.yml |
Generated category wiring. |
.github/workflows/prompt-clustering-analysis.lock.yml |
Generated category wiring. |
.github/workflows/pr-finisher.lock.yml |
Generated category wiring. |
.github/workflows/pr-description-caveman.lock.yml |
Generated category wiring. |
.github/workflows/portfolio-analyst.lock.yml |
Generated category wiring. |
.github/workflows/org-health-report.lock.yml |
Generated category wiring. |
.github/workflows/objective-impact-report.lock.yml |
Generated category wiring. |
.github/workflows/metrics-collector.lock.yml |
Generated category wiring. |
.github/workflows/lockfile-stats.lock.yml |
Generated category wiring. |
.github/workflows/linter-miner.lock.yml |
Generated category wiring. |
.github/workflows/lint-monster.lock.yml |
Generated category wiring. |
.github/workflows/layout-spec-maintainer.lock.yml |
Generated category wiring. |
.github/workflows/jsweep.lock.yml |
Generated category wiring. |
.github/workflows/issue-triage-agent.lock.yml |
Generated category wiring. |
.github/workflows/issue-monster.lock.yml |
Generated category wiring. |
.github/workflows/issue-arborist.lock.yml |
Generated category wiring. |
.github/workflows/instructions-janitor.lock.yml |
Generated category wiring. |
.github/workflows/hourly-ci-cleaner.lock.yml |
Generated category wiring. |
.github/workflows/gpclean.lock.yml |
Generated category wiring. |
.github/workflows/go-pattern-detector.lock.yml |
Generated category wiring. |
.github/workflows/go-logger.lock.yml |
Generated category wiring. |
.github/workflows/go-fan.lock.yml |
Generated category wiring. |
.github/workflows/glossary-maintainer.lock.yml |
Generated category wiring. |
.github/workflows/github-mcp-tools-report.lock.yml |
Generated category wiring. |
.github/workflows/front-page-copy-guard.lock.yml |
Generated category wiring. |
.github/workflows/firewall-escape.lock.yml |
Generated category wiring. |
.github/workflows/feature-grower.lock.yml |
Generated category wiring. |
.github/workflows/example-workflow-analyzer.lock.yml |
Generated category wiring. |
.github/workflows/example-failure-category-filter.lock.yml |
Generated category wiring. |
.github/workflows/evoskill-evolver.lock.yml |
Generated category wiring. |
.github/workflows/eslint-refiner.lock.yml |
Generated category wiring. |
.github/workflows/eslint-monster.lock.yml |
Generated category wiring. |
.github/workflows/eslint-miner.lock.yml |
Generated category wiring. |
.github/workflows/eslint-factory-dispatcher.lock.yml |
Generated category wiring. |
.github/workflows/enterprise-docs-audit.lock.yml |
Exposes categories to safe outputs. |
.github/workflows/duplicate-code-detector.lock.yml |
Generated category wiring. |
.github/workflows/draft-pr-cleanup.lock.yml |
Generated category wiring. |
.github/workflows/docs-noob-tester.lock.yml |
Generated category wiring. |
.github/workflows/dictation-prompt.lock.yml |
Generated category wiring. |
.github/workflows/dev-hawk.lock.yml |
Generated category wiring. |
.github/workflows/detection-analysis-report.lock.yml |
Generated category wiring. |
.github/workflows/designer-drift-audit.lock.yml |
Generated category wiring. |
.github/workflows/deployment-incident-monitor.lock.yml |
Generated category wiring. |
.github/workflows/delight.lock.yml |
Generated category wiring. |
.github/workflows/deep-report.lock.yml |
Generated category wiring. |
.github/workflows/dead-code-remover.lock.yml |
Generated category wiring. |
.github/workflows/daily-workflow-updater.lock.yml |
Generated category wiring. |
.github/workflows/daily-workflow-security-model.lock.yml |
Generated category wiring. |
.github/workflows/daily-work-queues-report.lock.yml |
Generated category wiring. |
.github/workflows/daily-work-queue-formal-verification.lock.yml |
Exposes categories to safe outputs. |
.github/workflows/daily-token-consumption-report.lock.yml |
Generated category wiring. |
.github/workflows/daily-team-status.lock.yml |
Generated category wiring. |
.github/workflows/daily-team-evolution-insights.lock.yml |
Generated category wiring. |
.github/workflows/daily-syntax-error-quality.lock.yml |
Generated category wiring. |
.github/workflows/daily-storify.lock.yml |
Generated category wiring. |
.github/workflows/daily-spending-forecast.lock.yml |
Generated category wiring. |
.github/workflows/daily-security-red-team.lock.yml |
Generated category wiring. |
.github/workflows/daily-security-observability.lock.yml |
Generated category wiring. |
.github/workflows/daily-secrets-analysis.lock.yml |
Generated category wiring. |
.github/workflows/daily-safe-output-optimizer.lock.yml |
Generated category wiring. |
.github/workflows/daily-safe-output-integrator.lock.yml |
Generated category wiring. |
.github/workflows/daily-report-dispatcher.lock.yml |
Generated category wiring. |
.github/workflows/daily-reliability-review.lock.yml |
Generated category wiring. |
.github/workflows/daily-regulatory.lock.yml |
Generated category wiring. |
.github/workflows/daily-pr-review-cursor.lock.yml |
Generated category wiring. |
.github/workflows/daily-performance-summary.lock.yml |
Generated category wiring. |
.github/workflows/daily-news.lock.yml |
Generated category wiring. |
.github/workflows/daily-model-resolution.lock.yml |
Generated category wiring. |
.github/workflows/daily-model-inventory.lock.yml |
Generated category wiring. |
.github/workflows/daily-issues-report.lock.yml |
Generated category wiring. |
.github/workflows/daily-graft-intelligence.lock.yml |
Generated category wiring. |
.github/workflows/daily-grader-audit.lock.yml |
Generated category wiring. |
.github/workflows/daily-geo-optimizer.lock.yml |
Generated category wiring. |
.github/workflows/daily-function-namer.lock.yml |
Generated category wiring. |
.github/workflows/daily-firewall-report.lock.yml |
Generated category wiring. |
.github/workflows/daily-file-diet.lock.yml |
Generated category wiring. |
.github/workflows/daily-fact.lock.yml |
Generated category wiring. |
.github/workflows/daily-experiment-report.lock.yml |
Generated category wiring. |
.github/workflows/daily-evals-report.lock.yml |
Generated category wiring. |
.github/workflows/daily-documentation-diagram.lock.yml |
Generated category wiring. |
.github/workflows/daily-doc-updater.lock.yml |
Generated category wiring. |
.github/workflows/daily-doc-healer.lock.yml |
Generated category wiring. |
.github/workflows/daily-credit-limit-test.lock.yml |
Generated category wiring. |
.github/workflows/daily-compiler-quality.lock.yml |
Generated category wiring. |
.github/workflows/daily-community-attribution.lock.yml |
Generated category wiring. |
.github/workflows/daily-code-metrics.lock.yml |
Generated category wiring. |
.github/workflows/daily-code-debt-aider.lock.yml |
Generated category wiring. |
.github/workflows/daily-cli-tools-tester.lock.yml |
Generated category wiring. |
.github/workflows/daily-cli-performance.lock.yml |
Generated category wiring. |
.github/workflows/copilot-session-insights.lock.yml |
Generated category wiring. |
.github/workflows/copilot-opt.lock.yml |
Generated category wiring. |
.github/workflows/copilot-agent-analysis.lock.yml |
Generated category wiring. |
.github/workflows/contribution-check.lock.yml |
Generated category wiring. |
.github/workflows/constraint-solving-potd.lock.yml |
Generated category wiring. |
.github/workflows/code-simplifier.lock.yml |
Generated category wiring. |
.github/workflows/code-scanning-fixer.lock.yml |
Generated category wiring. |
.github/workflows/cli-version-checker.lock.yml |
Generated category wiring. |
.github/workflows/cli-consistency-checker.lock.yml |
Generated category wiring. |
.github/workflows/ci-coach.lock.yml |
Generated category wiring. |
.github/workflows/changeset.lock.yml |
Generated category wiring. |
.github/workflows/breaking-change-checker.lock.yml |
Generated category wiring. |
.github/workflows/blog-auditor.lock.yml |
Generated category wiring. |
.github/workflows/avenger.lock.yml |
Generated category wiring. |
.github/workflows/auditor.lock.yml |
Generated category wiring. |
.github/workflows/audit-workflows.lock.yml |
Generated category wiring. |
.github/workflows/artifacts-summary.lock.yml |
Generated category wiring. |
.github/workflows/architecture-guardian.lock.yml |
Generated category wiring. |
.github/workflows/api-consumption-report.lock.yml |
Generated category wiring. |
.github/workflows/agent-persona-explorer.lock.yml |
Generated category wiring. |
.github/workflows/agent-performance-analyzer.lock.yml |
Generated category wiring. |
.github/workflows/agent-job-health.lock.yml |
Generated category wiring. |
.github/workflows/ab-testing-advisor.lock.yml |
Generated category wiring. |
Review effort: Max (auto)
| @@ -522,6 +523,8 @@ describe("generate_footer.cjs", () => { | |||
| expect(result).toContain("agentic threat detected"); | |||
| expect(result).toContain("<!-- gh-aw-threat-detected -->"); | |||
| expect(result).toContain("Potential security threats were detected"); | |||
| expect(result).toContain("Detected threat types: secret leak."); | |||
There was a problem hiding this comment.
Skills-Based Review 🧠
Applied /tdd and /grill-with-docs. Requesting changes on one test-isolation defect and on docs that describe a placeholder users cannot reach.
📋 Key Themes & Highlights
Key Themes
- Test isolation:
generate_footer.test.cjssetsGH_AW_DETECTION_THREAT_KINDSwithout clearing it, so the value leaks into later tests. - Documentation accuracy: the spec and
messages_core.cjsdocument{reason_text}and{threat_kinds}fordetection-warning, but Go only mapsdetection-failure(pkg/workflow/safe_outputs_messages_config.go:40), and the schema lists placeholders only fordetection-failure. - Schema drift:
pkg/parser/schemas/main_workflow_schema.json(detection-failure, around line 13434) still lists only{workflow_name}, {run_url}.frontmatter-full.mdis generated from it, so it is stale too.
Positive Highlights
- ✅ Threat-kind handling is allowlisted (
THREAT_KIND_LABELS), and private detector text never reaches public output. Tests cover this with aPRIVATE_TEXTnon-leakage check. - ✅
conclude_threat_detection.shvalidates all three booleans withjq -erand fails closed to an empty category list on invalid input. - ✅
notify_comment_error.cjsassignsmessagein every branch, so the block-based refactor cannot crash on an unset value. - ✅
getDetectionReasonTextonly adds category text forthreat_detected, so tooling failures never show threat types.
Uncertainty
- Lock-file consistency could not be verified: the local worktree has about 324 uncommitted
.lock.ymledits, which do not match HEAD. I did not treat them as a finding. - The usage-artifact collector in
pkg/workflow/notify_comment.go(~line 428) omitsGH_AW_DETECTION_THREAT_KINDS. This may be intentional, but no test documents that.
@copilot please address the review comments above.
🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer · copilot · routed: haiku55 xhigh · ~8 AIC · ⌖ ~2 AIC · ⊞ 5.4K
Comment /matt to run again
| @@ -488,6 +488,7 @@ describe("generate_footer.cjs", () => { | |||
| it("should NOT include caution alert (caution is now injected by callers at the top)", () => { | |||
| process.env.GH_AW_DETECTION_CONCLUSION = "warning"; | |||
| process.env.GH_AW_DETECTION_REASON = "threat_detected"; | |||
| process.env.GH_AW_DETECTION_THREAT_KINDS = "secret_leak PRIVATE_TEXT"; | |||
There was a problem hiding this comment.
[/tdd] Test isolation: GH_AW_DETECTION_THREAT_KINDS is set here but never cleared. The beforeEach at lines 55-56 only deletes GH_AW_DETECTION_CONCLUSION and GH_AW_DETECTION_REASON, so the value persists into later tests and can change footer text in an order-dependent way.
Add delete process.env.GH_AW_DETECTION_THREAT_KINDS; alongside the other deletes.
@copilot please address this.
There was a problem hiding this comment.
Added GH_AW_DETECTION_THREAT_KINDS to per-test cleanup and set it explicitly in the category assertion test. Commit 3e48d5b33b.
| @@ -5943,6 +5943,9 @@ When threat detection executes in `warn` mode and reports a threat signal for a | |||
|
|
|||
| 2. A visible threat label string: `agentic threat detected`. | |||
| 3. An XML comment marker in emitted markdown content: `<!-- gh-aw-threat-detected -->`. | |||
| 4. Detector-defined threat types from the boolean verdict fields: `prompt_injection`, `secret_leak`, and `malicious_patch`. Public warnings, failure notifications, tracking comments, and review messages MUST describe only the fields set to `true`, using framework-authored labels. They MUST NOT publish detector-generated explanations or agent text as threat details. If the types are unavailable, the message MUST say they were not reported rather than infer a finding. Tooling failures MUST remain distinct from threat findings. | |||
|
|
|||
| Configured `detection-failure` and `detection-warning` templates can use `{reason_text}` for a framework-authored description and `{threat_kinds}` for the comma-separated category identifiers (`unknown` when unavailable). | |||
There was a problem hiding this comment.
[/grill-with-docs] This sentence says detection-warning is a configurable template, but no user-facing key reads it. pkg/workflow/safe_outputs_messages_config.go:40 maps only detection-failure, and the schema defines only detection-failure. So {reason_text} and {threat_kinds} are unreachable for warnings in practice.
Either drop detection-warning from this sentence, or wire the key through the Go config and the schema.
@copilot please address this.
There was a problem hiding this comment.
Updated the specification to document only detection-failure as configurable; warn-mode warning templates are framework-provided. Commit 3e48d5b33b.
| * @property {string} [pullRequestCreated] - Custom template for pull request creation link. Placeholders: {item_number}, {item_url} | ||
| * @property {string} [issueCreated] - Custom template for issue creation link. Placeholders: {item_number}, {item_url} | ||
| * @property {string} [commitPushed] - Custom template for commit push link. Placeholders: {commit_sha}, {short_sha}, {commit_url} | ||
| * @property {string} [detectionWarning] - Custom detection warning message template (continue-on-error mode). Placeholders: {workflow_name}, {run_url}, {reason_text} | ||
| * @property {string} [detectionWarning] - Custom detection warning message template (continue-on-error mode). Placeholders: {workflow_name}, {run_url}, {reason_text}, {threat_kinds} |
There was a problem hiding this comment.
[/grill-with-docs] This JSDoc advertises {reason_text} and {threat_kinds} for detectionWarning, but nothing maps a user-facing key to it (see the spec comment above). Keep the placeholder list in line with what users can actually configure, or add the mapping.
@copilot please address this.
There was a problem hiding this comment.
Removed the unsupported detectionWarning JSDoc configuration field and placeholder claim; only detectionFailure is user-configurable. Commit 3e48d5b33b.
|
Copilot assignment requested for the following outstanding work:
Once you start work, post this work summary as a PR comment without the leading Copilot mention, including the Sous-chef head, work, and state lines below, to record that the assignment is running. Apply the necessary fixes on this PR's existing branch, reply to each listed review thread and resolve it when addressed. Ignore feedback already answered or resolved; stop when only human review or CI remains. Do not merge or trigger CI. Sous-chef requested-head: 5fdbd49
|
…eat-category-messages Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>
Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>
…mports, polished README with logo - receipts.py: tasks, streamed events per step, paste mode, recheck, low-thinking Gemini - server.py: POST /api/run streams each sponsor step; rate-limited for the public site - index.html: four-step pipeline with live clocks, claim highlights in the message and the log, gallery lies-first - tools/import_pr.py: judge a real agent-written PR against its CI logs - runs_evidence: 6 live Gemini runs + 2 real GitHub Copilot PRs (github/gh-aw#75774: contradicted) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
🎉 This pull request is included in a new release. Release: |

Summary
prompt_injection,secret_leak,malicious_patch) from both inline and external detector conclusion paths, and propagate them to safe-output and conclusion jobs.Validation
make agent-report-progresspassed, including Go/custom/JavaScript/shell lint, impacted tests, schema freshness, and full workflow drift checking.make recompile: 361/361 workflows compiled successfully.npm run typecheckpassed.go test ./pkg/workflow -run '^(TestConcludeThreatDetectionScript_.*|TestDetectionJobHasSuccessOutput|TestSafeOutputJobsCheckDetectionSuccess|TestDetectionRunsStepInConclusionJob)$' -count=1passed.No workflows were dispatched.