Skip to content

Fix safe-output URL redaction corrupting HTML tag boundaries - #76719

Merged
lpcox merged 4 commits into
mainfrom
lpcox-fix-safe-output-url-tag-boundaries
Nov 11, 2026
Merged

lpcox merged 4 commits into
mainfrom
lpcox-fix-safe-output-url-tag-boundaries

Conversation

@lpcox

@lpcox lpcox commented Nov 11, 2026

Copy link
Copy Markdown
Collaborator

Summary

Resolves githubnext/gh-aw-security#3379.

  • Bound protocol and domain URL redaction at attribute quotes and angle brackets so later URL filtering cannot delete boundaries established by HTML attribute sanitization.
  • Cover single-colon blocked schemes, non-HTTPS schemes, disallowed HTTPS/protocol-relative URLs, and scheme-less www. candidates. Preserve the earlier www-autolink authority validation outside HTML attribute candidates.
  • Add exact-output regressions covering the reported minimal reproduction, quoted/unquoted attributes, multiline prose, dangerous-attribute stripping, adjacent tags, encoded schemes, all sanitizer entry paths, both URL policies, and idempotence.

This is an independent change based on current target main (9c2d7ee7386ba26cfa035aa929eb4682adef4cb2), not a stack on #76708. Existing PRs were checked; no fix for this specific mechanism was found.

Verification

Reproduced on main:

Input:  <img src="data:image/png;base64,AAAA=="> text "onerror=alert(1)>
Before: <img src="(redacted) text "onerror=alert(1)>
After:  <img src="(redacted)"> text "onerror=alert(1)>

The original tag now remains closed; the later onerror text remains outside it rather than becoming an attribute.

  • Eight targeted sanitizer/safe-output suites: 1,029 tests passed, including the existing www-autolink regressions.
  • make agent-report-progress: passed, including build, scoped JavaScript lint/type checks, and impacted tests. No Go files or workflow sources changed.

Scope and limits

This fixes a reproduced sanitizer string-transformation defect. It does not establish live GitHub XSS: GitHub's renderer may apply additional sanitization. No live rendering probe or pentest workflow dispatch was performed. No security-repository workflow pins or unrelated findings were changed.

Bound protocol and domain URL matches so redaction cannot remove attribute quotes or tag delimiters after dangerous attributes have been stripped. Preserve GFM www authority checks outside HTML attribute candidates.

Resolves githubnext/gh-aw-security#3379

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI dynamic review requested due to automatic review settings November 11, 2026 02:36
// fully general single-colon pattern produces too many false positives
// (e.g. "key:value" in YAML, "std::vector" in C++, "C:\path" on Windows).
return result.replace(/(?:data|javascript|vbscript|about|mailto|tel|magnet):[^\s]+/gi, match => {
return result.replace(/(?:data|javascript|vbscript|about|mailto|tel|magnet):[^\s"'<>]+/gi, match => {

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Critical URL-boundary and attribute-detection defects leave blocked or disallowed URLs unredacted and can pull outside text into image tags after dangerous-attribute stripping.

3 open issues
What changed in this PR

This PR targets HTML-boundary corruption in gh-aw’s shared safe-output sanitizer, preventing URL redaction from undoing earlier HTML attribute sanitization.

Changes:

  • Bound protocol and domain URL matches at quotes and angle brackets.
  • Adjust scheme-less www. scanning around HTML attribute candidates.
  • Add exact-output regressions across sanitizer entry paths, URL policies, adjacent tags, and idempotence.
File Description
actions/​setup/​js/​sanitize_content_url_boundaries.test.cjs Adds URL-redaction boundary and sanitizer-path regressions.
actions/​setup/​js/​sanitize_content_core.cjs Updates URL matchers and www. candidate boundary handling.

Review effort: Max (auto)

// fully general single-colon pattern produces too many false positives
// (e.g. "key:value" in YAML, "std::vector" in C++, "C:\path" on Windows).
return result.replace(/(?:data|javascript|vbscript|about|mailto|tel|magnet):[^\s]+/gi, match => {
return result.replace(/(?:data|javascript|vbscript|about|mailto|tel|magnet):[^\s"'<>]+/gi, match => {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated the blocked-scheme matcher to redact a non-whitespace value even when its bounded suffix is empty, leaving the quote or angle delimiter intact. Immediate-delimiter and prose regressions pass; included in b72ba7759f.

const hasAuthorityUserinfo = authorityUserinfoAt[candidateStart] === 1;
let beforeCandidate = candidateStart - 1;
while (beforeCandidate >= 0 && /\s/.test(s[beforeCandidate])) beforeCandidate--;
const isAttributeCandidate = /["'=]/.test(s[beforeCandidate] || "");

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added tag-aware www. boundary tracking so a bare attribute candidate stops at the enclosing tag rather than consuming later userinfo-like prose. The bare-attribute regression passes in b72ba7759f.

if (character === "\\" && end + 1 < s.length) {
// GFM permits ">" inside a userinfo authority, but HTML attribute
// candidates must stop there even when later prose contains "@".
if (/[\s"'|\[\]]/.test(character) || (/[<>]/.test(character) && (!inAuthority || !hasAuthorityUserinfo || isAttributeCandidate))) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The scanner now distinguishes actual HTML-tag context from prose while preserving full-authority URL parsing. Prose and in-tag userinfo regressions pass in b72ba7759f.

@pelikhan

Copy link
Copy Markdown
Collaborator

@copilot run pr-finisher skill

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
@pelikhan

Copy link
Copy Markdown
Collaborator

/matt

@pelikhan

Copy link
Copy Markdown
Collaborator

/review

@github-actions

github-actions Bot commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

🧠 Matt Pocock Skills Reviewer has completed the skills-based review. ✅

Matt Pocock Skills Reviewer

Outcome: reported_action

The agent completed and reported one or more actions. The no-op output is a completion signal, not a claim that no action was taken.

Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.

Generated from Matt Pocock Skills Reviewer · routed: haiku55 xhigh · ~8 AIC · ⌖ ~2 AIC · ⊞ 6.6K · ◷

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer

@github-actions

github-actions Bot commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

🧠 Matt Pocock Skills Reviewer has completed the skills-based review. ✅

Matt Pocock Skills Reviewer

Outcome: reported_action

The agent completed and reported one or more actions. The no-op output is a completion signal, not a claim that no action was taken.

Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.

Generated from Matt Pocock Skills Reviewer · routed: haiku55 xhigh · ~8 AIC · ⌖ ~2 AIC · ⊞ 6.6K · ◷

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔎 Code quality review by PR Code Quality Reviewer · copilot · gpt50 · ~3 AIC · ⌖ ~2 AIC · ⊞ 1.6K
Comment /review to run again

let htmlQuote = "";
for (let i = 0; i < s.length; i++) {
const character = s[i];
if (!inHtmlTag && character === "<" && /[a-z]/i.test(s[i + 1] || "")) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The scanner enters inHtmlTag for any <letter... prefix and never recovers when the tag is malformed or has an unterminated quote. For example, <b don't leaves htmlQuote set through the remainder of the string, so later (www/redacted). candidates are parsed with HTML-tag termination rules and can have their URL boundary changed by unrelated prose. Since this state is attacker-controlled input, please bound/reset malformed tag state (for example at a newline or a conservative maximum) and add a regression for an unclosed tag before a later URL.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added recovery for unterminated quoted tags at line breaks and a 4096-character cap for tag state, with regressions for both recovery paths. The later URL boundary is preserved in b72ba7759f.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Found one correctness issue in the new HTML-boundary scanner: malformed or unterminated tags can leave its state active across the rest of the input and alter later URL redaction boundaries. Inline details and a regression suggestion are provided above. The specialist pass completed; other candidate concerns were not actionable or were intentional boundary-preservation behavior.

🔎 Code quality review by PR Code Quality Reviewer · copilot · gpt50 · ~3 AIC · ⌖ ~2 AIC · ⊞ 1.6K
Comment /review to run again

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Skills-Based Review 🧠

Applied /diagnosing-bugs and /tdd. Requesting changes: the new in-tag quote break lets a (www/redacted). URL leak its real host in HTML tag attributes, which the base commit redacts.

📋 Key Themes & Highlights

Key Themes

  • Security regression (blocking): <img alt="www.github.com"@evil.example/x"> leaks evil.example on head. Base redacts it. Inline comment on sanitize_content_core.cjs line 684.
  • Test gap: the new tests cover "www.github.com"@ only in prose, not inside a tag.
  • Existing thread on line 412: (redacted)"blank" is still redacted on head (leftover "blank" is plain text), so that concern does not reproduce.

Positive Highlights

  • ✅ The new expectations pass on head (459/459 in a plain-Node harness) and fail on the reconstructed base (11/459), so the tests catch the original bug.
  • ✅ Differential fuzzing found no newly introduced dangerous-scheme survivors.

Validation Limits

  • The vitest suite could not run in this sandbox (npm registry TLS certificate error). Validation used a plain-Node harness that runs the test expectations directly. Please run the full vitest suite in CI.

@copilot please address the inline review comment on line 684.

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer · copilot · routed: haiku55 xhigh · ~8 AIC · ⌖ ~2 AIC · ⊞ 6.6K
Comment /matt to run again

// must stop there even when later prose contains "@".
if (
/[\s|\[\]]/.test(character) ||
(isHtmlTagCandidate && /["']/.test(character)) ||

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[/diagnosing-bugs] In-tag host leak: the new quote break on this line ends the (www/redacted). candidate at the first quote, so the allowlist checks only www.github.com and the host after @ stays visible. On this head, <img alt="www.github.com"@evil.example/x"> becomes <img alt="www.github.com" @evil .example/x">, which leaks evil.example. The base commit redacts it to <img alt="(evil.example/redacted)>.

💡 Suggested fix and regression test

Before accepting the allowed prefix, check the full authority (the text up to the next delimiter, including the @ and the host after it) and redact the whole candidate when that host is not allowlisted. Add an in-tag case to sanitize_content_url_boundaries.test.cjs, for example <img alt="www.github.com"@evil.example/x">, expecting no evil.example in the output. The existing test only covers `"(redacted) in prose (lines 105 to 106).

@copilot please address this.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tag candidates with a following @ now continue across the first quote to validate the complete userinfo authority, while stopping at the final quote/tag boundary. Added the reported in-tag leak regression; fixed in b72ba7759f.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Skills-Based Review 🧠

Applied /diagnosing-bugs and /tdd. Requesting changes on one regression that the new tests do not cover.

📋 Key Themes & Highlights

Key Themes

  • Regression in unclosed-tag redaction: <b www.github.com"@evil.invalid/x is redacted at the base commit but passes through at HEAD with the host visible. The inline comment on sanitize_content_core.cjs has a one-line guard that restores base behaviour in a scratch copy.
  • Test gap: sanitize_content_url_boundaries.test.cjs has no unclosed-tag (www/redacted). plus @ case, which is how this slipped through.

Positive Highlights

  • ✅ The repro from the PR body (<img src="(redacted)"> ... onerror=alert(1)>) is fixed and the (redacted)"blank" thread is resolved.
  • ✅ The forward HTML-tag pre-pass is a clearer model than the old per-candidate lookahead.

Limits

  • I could not run the vitest suite here because the npm registry rejected the sandbox TLS certificate, so the PR test results are unverified by me. The differential fuzz covered 2,304 inputs.

@copilot please address the review comments above.

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer · copilot · routed: haiku55 xhigh · ~8 AIC · ⌖ ~2 AIC · ⊞ 6.6K
Comment /matt to run again

// must stop there even when later prose contains "@".
if (
/[\s|\[\]]/.test(character) ||
(isHtmlTagCandidate && /["']/.test(character)) ||

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[/diagnosing-bugs] Regression: unclosed-tag redaction now leaks the outside host. <b www.github.com"@evil.invalid/x is redacted to <b (evil.invalid/redacted) at the base commit, but HEAD returns it unchanged, so evil.invalid is never checked against the allowlist. The new quote terminator here stops the (www/redacted). candidate before the @ userinfo branch runs. The same root cause affects <b www.github.com"@-evil.invalid/x and a <b www.github.com'@_evil.invalid/x.

💡 Candidate fix and repro

Skip the quote terminator when the candidate already has @ userinfo ahead, mirroring the non-tag branch:

(isHtmlTagCandidate && /["']/.test(character) && !hasAuthorityUserinfo) ||

In a scratch copy, this restores base output for the three cases above. Across 2,304 fuzzed inputs it changed only the one output that regressed, and it does not affect the existing "www.github.com"@evil.invalid/x case. I could not run the vitest suite in this sandbox, so please add an unclosed-tag (www/redacted). plus @ case to sanitize_content_url_boundaries.test.cjs.

@copilot please address this.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated quote handling to retain authority validation when @ follows the candidate and added the reported unclosed-tag/host variants. The malformed-tag state is also bounded; fixed in b72ba7759f.

@gh-aw-bot

Copy link
Copy Markdown
Collaborator

@copilot

🍳 A little prep to help this PR along! Copilot assignment requested for the following outstanding work:

  1. Update this branch with the latest main using make merge-main, resolving any conflicts and preserving the intended changes.
  2. Review (actions/setup/js/sanitize_content_core.cjs:615): address unresolved feedback - Review comment
  3. Review (actions/setup/js/sanitize_content_core.cjs:684): address unresolved feedback - Review comment
  4. Review (actions/setup/js/sanitize_content_core.cjs:684): address unresolved feedback - Review comment
  5. Fix failing check agent (FAILURE): Check run.

Keeping the changes focused; human review and CI get the final taste test. No merging or CI triggering.

Structured data
{
  "completed_task": 0,
  "head": "5ea7e38c614a4fafa846a7eeca54baebc7572406",
  "kind": "requested",
  "schema_version": "1",
  "state": "0e4a76a1a5086dd4be0c5d12d9d4fb598f90f8ec81e0e0eadf6c138346f68455",
  "work": [
    "21d4bd39367ba4e168dfe3432a649bf92736b544e0dae4a3e81eb0ba8ac9d534",
    "34f38971738f6a752a646bd7797a6c8d47fb096e4eb6f91e946a91255433c8c4",
    "53810549110f381add47161658a8dc3e0abd1231bfeb494acbe0d498c6058687",
    "64ce6f708a9d89c9014a979951dfbfe04edd8058bfb6730db93e36da665dc091"
  ],
  "workflow": "pr-sous-chef"
}

Generated by 👨🍳 PR Sous Chef · pi · routed: haiku55 xhigh · ~3 AIC · ⌖ ~2 AIC · ⊞ 6.9K · ◷
Comment /souschef to run again

Copilot AI and others added 2 commits November 11, 2026 05:11
…-url-tag-boundaries

Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>
Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>

Copilot AI commented Nov 11, 2026

Copy link
Copy Markdown
Contributor

@copilot

🍳 A little prep to help this PR along! Copilot assignment requested for the following outstanding work:...

Merged latest main via make merge-main (c7efe24e) and pushed the review fixes in b72ba7759f. Formatting, lint, final build/impacted tests (184 passed), and secret scan pass. The cited Ponytail agent failure was AWF model-routing completion validation (Codex exited 0; router failed with exit 78), not a sanitizer/test failure, so no unrelated workflow workaround was added. CodeQL timed out without results; CI on this head needs a maintainer re-trigger.

Copilot AI requested a review from gh-aw-bot November 11, 2026 05:33
@lpcox
lpcox merged commit 74fb78d into main Nov 11, 2026
84 checks passed
@lpcox
lpcox deleted the lpcox-fix-safe-output-url-tag-boundaries branch November 11, 2026 06:24
@github-actions

Copy link
Copy Markdown
Contributor

🎉 This pull request is included in a new release.

Release: v1.2.3

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants