Skip to content

Fix #4007: lab-ops-hub: Worker-fallback extractor — complete HTML character-referen #5115

Closed as not planned

Description

@developing-today

Implement GitHub issue #4007 in developing-today/code: lab-ops-hub: Worker-fallback extractor — complete HTML character-reference decoding parity

Problem

The lab-ops-hub Worker-fallback extractor (pkgs/lab-ops-hub/src/lib.ts, decodeHtmlEntities) only decodes a small named-entity table and requires a trailing semicolon, so its output diverges from the container Extractor (Python html.parser + html.unescape) on standard references — stored titles/text then depend on which crawling path ran.

Found by Codex review round 4 on PR #3918 (comment 4437512013), dispositioned there as a follow-up because the fix (complete HTML5 named-reference data + semicolon-omission rules) is larger than a review-fix round. Sibling of #569.

Evidence (verified against the container Extractor, Python 3.13.13)

Input: <title>&Alpha; &AMP;</title><p>&rarr; &#65 &amp B</p>

  • Container: title Α &, text → A & B
  • Worker fallback: every reference preserved literally

Scope

  • Decode the complete HTML5 named character references (the data behind Python's html.entities.html5), including the legacy semicolon-less forms and the spec's semicolon-omission rules for text content (e.g. &#65 -> A, &amp -> & in the cases html.unescape expands).
  • Keep the existing numeric-reference normalization (C1 remap, U+FFFD rules) intact.
  • Add shared parity fixtures (test/extractor-fixtures.json) pinning the cases above plus a spread of named references; expected values verified against the container suite before pinning. Both suites (node + python) must stay green.

Acceptance

  • The example above produces identical {title, links, text} from the Worker fallback and the container Extractor.
  • New fixtures cover: a non-Latin-1 named reference (→), an uppercase legacy reference (&), a semicolon-less numeric reference, and a semicolon-less named reference.

Acceptance criteria

  • The Worker-fallback extractor (pkgs/lab-ops-hub/src/lib.ts, decodeHtmlEntities) decodes the complete HTML5 named character references, including legacy semicolon-less forms, with the spec's semicolon-omission rules for text content matching Python html.unescape.
  • Filed evidence case: input <title>&Alpha; &AMP;</title><p>&rarr; &#65 &amp B</p> yields title Α & and text → A & B from the Worker fallback, identical to the container Extractor.
  • A differential test corpus (named references with/without semicolons, numeric references with/without semicolons, ambiguous ampersands in text and in attribute-like contexts) matches html.unescape output case for case.
  • Existing extractor test suites pass; no change to the container Extractor.

Rules from the repo owner: (1) Deliver the finished work as a pull request in this run; put "Fixes #4007" in the PR description so the issue closes on merge. (2) Cover the whole set in the one PR: if the issue names a set (files, subcommands, cases), fix every member, and batch any directly related dependency updates into the same PR — never one PR per item or per dependency. (3) Run the relevant tests and checks before opening the PR. (4) Merging is permitted but never required: merge only when checks pass and you are reasonably certain the PR is correct and low-risk — otherwise leave it open, and state your confidence and what a human should check. (5) If you hit a genuine blocker (missing credentials, a decision only the owner can make), stop and say exactly what is needed in your final summary instead of guessing.


Launched by pm-dispatch for developing-today/code. Queue task and OpenCode session ids follow in a comment below.

Activity

  1. developing-today commented on Nov 3, 2026

    @developing-today
    ContributorAuthor

    Dispatch launch FAILED at enqueue before a queue task was created, so this dispatch never became real. Closing this tracking issue as not planned so it does not linger as an orphan (#4917).

    Error: getaddrinfo ENOTFOUND nltgxorqwxhgfzseiirt.supabase.co

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions