Repository navigation
aggregator(charges-collect): parseChargeCandidates accepts impossible calendar dates — dedupe breaks and hand-fed rows are silently dropped #2194
Description
Bug
parseChargeCandidates (pm-stack/aggregator/src/charges-collect.ts:42), the structural validation boundary for the charges candidates state file the daily collector consumes, checks a candidate's date with a shape-only regex:
if (typeof c.at !== "string" || !/^\d{4}-\d{2}-\d{2}/.test(c.at)) continue;That accepts impossible calendar dates — 2026-13-45, 2026-02-30, 2026-00-00, 2026-10-00 all pass as "an ISO date" (the function's own doc comment, charges-collect.ts:25-29). The existing test (test/charges-collect.test.ts:24) only covers the non-date string "not-a-date", which the regex does reject.
The bad dates then corrupt both consumers in opposite directions, because both daysBetween implementations (charges-collect.ts:63-66, charges-match.ts:118-121) are Math.abs(Date.parse(a) - Date.parse(b)):
- Month 13+ (
Date.parse→NaN): inmatchCharges(charges-match.ts:196)NaN <= DEDUPE_WINDOW_DAYSis false, so the same real charge seen by Plaid and Gmail is never deduped — it publishes as two rows. InmergeChargeRows/sameCharge(charges-collect.ts:79)NaN > DEDUPE_WINDOW_DAYSis also false, so the window check passes vacuously — a hand-fed row is dropped as a "duplicate" of an auto row whose date is actually years away. - Feb 30 / day-00 (
Date.parserolls over):Date.parse("2026-02-30")is 2026-03-02, so window arithmetic silently uses a date different from the one in the file and in the published row'sat.
The producer writes this file with the same shape-only checks (scripts/produce-charge-candidates.ts:111 for Plaid, :207 for Gmail), so the collector's validator is the only place a real calendar check can be enforced for the file as persisted state (hand edits, an older/buggier producer, a partially migrated file).
Evidence (verified this run, main 478797c)
Verbatim copies of the shipped charges-collect.ts + charges-match.ts (sha256 identical to main) run under Bun 1.4.2 on the laptop — probe: ~/git/fleet/probe-2026-10-29-laneA-chargedates/:
parse "2026-10-28" -> ACCEPTED at=2026-10-28 (control)
parse "2026-13-45" -> ACCEPTED at=2026-13-45
parse "2026-02-30" -> ACCEPTED at=2026-02-30
parse "2026-00-00" -> ACCEPTED at=2026-00-00
parse "2026-10-00" -> ACCEPTED at=2026-10-00
parse "not-a-date" -> dropped (existing test case)
Date.parse 2026-13-45 = NaN 2026-02-30 = 2026-03-02T00:00:00.000Z
matchCharges rows (plaid 2026-10-28 + gmail 2026-13-45, same merchant/amount): 2 (control, both valid: 1)
mergeChargeRows (auto at=2026-13-45 vs hand-fed at=2020-01-01, same merchant/amount): hand-fed row DROPPED (control: kept, 2 rows)
Collector wiring: scripts/collect-fleet-feed.ts:421-427 — parseChargeCandidates(JSON.parse(...)) → matchCharges(...) → mergeChargeRows(auto.rows, charges ?? []).
Note: the at.slice(0, 10) truncation of a full timestamp ("2026-10-27T14:03:00Z" → "2026-10-27", test line 22) is intended behavior and is not part of this issue — only the calendar validity of the resulting 10 chars is.
Why it matters
This file is the charges feed's source of record for a period. A single impossible date silently (a) double-publishes a charge in the hub, or (b) suppresses a legitimate hand-fed row — the exact hand-fed rows the merge exists to preserve — with no log line either way (parseChargeCandidates drops malformed entries silently by design, and these entries are not even seen as malformed). Validation that only checks shape is what the function's doc promises it does not do.
Acceptance criteria
-
parseChargeCandidatesrejects candidates whoseat(after the existing 10-char slice) is not a real calendar date: month 01-12, day valid for that month/year (leap years included) —2026-13-45,2026-02-30,2026-00-00,2026-10-00are dropped;2026-02-29is dropped but2024-02-29is kept. - Regression tests in test/charges-collect.test.ts cover the impossible-date cases above alongside the existing
"not-a-date"case, and the intended full-timestamp slice case (test line 22) keeps passing. - With only real dates able to enter,
matchChargesdedupe andmergeChargeRowshand-fed preservation behave as in the controls above (a same-charge Plaid+Gmail pair yields 1 row; a years-apart hand-fed row is kept). - Triage pins whether the same calendar check is also applied at the producer's two shape-only sites (scripts/produce-charge-candidates.ts:111, :207) in this fix or as a follow-up.
Dedup
Checked open + closed issues and the repo backlog this run: #2130 covers the matcher's clustering rules (same-source repeats inside the window; empty-normalized merchants merging) — different defect, in charges-match.ts clustering, with all-valid dates. #1057 (closed) is the feature. #2161/#2172 are config env-var validation in other components. No existing issue covers calendar validity of charge candidate dates.
Activity
developing-today commentedon Oct 29, 2026
CODE-1082
dezren39 commentedon Oct 29, 2026
🤖 Agent note · filed
23 remaining items
dezren39 commentedon Nov 2, 2026
agent-note:v1
event: picked_up
task: 5223 (repo-health-code-pilot, OpenChamber scheduled lane; attempt 2)
state: Resumed from the existing PR #4136 and checkpoint; issue #2194 and PR #4136 are open, no hard-stop labels found, and the PR head matches the recorded branch/commit. Required test check is still pending.
next: Recheck PR checks/review state; park if CI remains pending per the five-minute rule.
dezren39 commentedon Nov 2, 2026
agent-note:v1
event: parked
task: 5223 (repo-health-code-pilot, OpenChamber scheduled lane; attempt 2)
state: PR #4136 is open at d4e7a33; fresh checks show the required test check pending (CodeRabbit skipped). Task parked for 20 minutes per the CI rule; no claim is held.
resume: After the park delay, recheck PR #4136 checks and review state. If green, revalidate diff and targeted tests; merge only if all standing criteria pass.
dezren39 commentedon Nov 2, 2026
agent-note:v1
task: 5223
state: parked
reason: PR #4136 rechecked fresh 2026-11-02T04:49Z — head d4e7a33, Codex review completed with no findings, CodeRabbit neutral, but the required test check is still QUEUED (runner queue delay, not a failure). Prior checkpoint: targeted tests pass (charges-collect 12, charges-match+collect 25), producer build pass.
resume: merge PR #4136 when the test check goes green, then complete task 5223 fixed_merged.
dezren39 commentedon Nov 2, 2026
agent-note:v1 — escalated — OpenChamber scheduled lane
escalated-to-user; issue aggregator(charges-collect): parseChargeCandidates accepts impossible calendar dates — dedupe breaks and hand-fed rows are silently dropped #2194 remains OPEN.dezren39(Drew-owned). Its checks are PENDING (teststill pending; one check passed); the recorded targeted tests pass from prior checkpoints. The PR is linked to this issue.