Repository navigation
feat(aggregator): wire charges auto-matcher into collector + Plaid/Gmail producer (#1057) - #2149
Conversation
|
Warning Review limit reachedYou've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Next included review available in 40 minutes. View limit details
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: bf15a77e56
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| try { | ||
| const candidates = parseChargeCandidates(JSON.parse(await readFile(chargeCandidatesPath, "utf8"))); | ||
| if (candidates.length) { | ||
| const auto = matchCharges(candidates, (saas ?? []).map((s) => ({ name: s.name }))); |
There was a problem hiding this comment.
For existing Anthropic or OpenAI subscriptions, this catalog always classifies their charges as unrecognized: extractSaasServices() returns only the eight infrastructure services, while Anthropic, OpenAI, and the other AI providers are represented in extractQuotaPools() instead. Consequently, an ANTHROPIC transaction produces an unmatched-charge attention flag even though the hub already tracks that provider. Include the hub's AI quota providers and their service aliases in the matching catalog rather than using only saas.
Useful? React with 👍 / 👎.
| skipped++; | ||
| continue; | ||
| } | ||
| candidates.push({ source: "gmail", at, merchant, amountUsd: Number(amount.replaceAll(",", "")) }); |
There was a problem hiding this comment.
For a card-alert subject such as Your $20.00 purchase at OpenAI was declined, the Gmail query includes the message, the amount parser extracts $20, and merchantFromSubject() extracts OpenAI before was. This unconditional insertion therefore records a payment that never occurred, which the matcher subsequently displays as a real charge and may flag as unrecognized SaaS spending. Reject declined or failed purchase notices before creating a candidate.
Useful? React with 👍 / 👎.
| await writeFile( | ||
| chargesSurfacedStatePath, | ||
| JSON.stringify({ version: 1, surfaced: keys, updatedAt: now.toISOString() }, null, 2), | ||
| ); |
There was a problem hiding this comment.
If feed publication or the laptop transfer fails, these charges are permanently suppressed without ever reaching the attention strip. In the inspected scripts/push-fleet-feed.sh, collection runs at line 62 before the transfer at lines 185–195; this state write therefore happens before either local feed publication or successful delivery. On the next five-minute collection, the persisted keys remove those flags and the replacement feed omits them. Keep the notifications pending until successful delivery, or otherwise preserve them across failed publication attempts.
Useful? React with 👍 / 👎.
| if (handFed.amountUsd !== auto.amountUsd) return false; | ||
| if (daysBetween(handFed.at, auto.at) > DEDUPE_WINDOW_DAYS) return false; |
There was a problem hiding this comment.
When a hand-fed incident and an auto row have the same merchant and amount within three days but identify different cards, this predicate still treats them as the same purchase and deletes the hand-fed incident. For example, an auto $20 METAPAY charge on Visa •7535 removes a separate $20 Meta PAYG spill on Visa •9545. The existing extractCharges() incident deduplication explicitly includes the card in its identity, so this merge loses evidence that was previously preserved. Reject duplicate matches when both rows carry known, conflicting card masks.
Useful? React with 👍 / 👎.
| }); | ||
| const text = (JSON.parse(body) as { body_text?: string }).body_text ?? ""; | ||
| amount = | ||
| /(?:amount paid|amount charged|total charged|total)[:\s]*\$([\d,]+\.\d{2})/i.exec(text)?.[1] ?? AMOUNT_RE.exec(text)?.[1]; |
There was a problem hiding this comment.
For an amount-less receipt subject whose body contains Subtotal $20.00, followed by tax and Total $21.60, the unbounded total alternative matches the suffix of Subtotal and records a $20 charge rather than the $21.60 payment. This was reproduced through the producer; even a later explicit Amount paid $21.60 does not help because exec() selects the first textual match. Since the matcher requires equal amounts to deduplicate sources, the corresponding $21.60 Plaid transaction also remains as a separate charge. Require a standalone total label and prefer explicit paid or charged totals.
Useful? React with 👍 / 👎.
4e78f7e to
1adfa43
Compare
…ail producer (#1057) Collector: read the producer candidates file, run matchCharges, merge auto rows over hand-fed charges (auto wins on duplicate), surface unmatched SaaS/AI charges in feed chargesAttention exactly once via persisted surfaced-state. Live source maps chargesAttention to unmatched-charge attention flags. Producer: Plaid transactions-get (purchase-like only) + Gmail receipt/alert parse, charge evidence only.
1adfa43 to
7d064e5
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 1adfa438f8
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| amount ??= | ||
| /(?:amount paid|amount charged|total charged|transaction amount|total)[:\s]*\$([\d,]+\.\d{2})/i.exec( | ||
| text, | ||
| )?.[1] ?? AMOUNT_RE.exec(text)?.[1]; |
There was a problem hiding this comment.
For an email titled Your OpenAI invoice whose body says Amount due $20.00\nDue Nov 1, the subject filter accepts the message, the merchant parser returns OpenAI, and this fallback extracts $20 without any evidence of payment. The producer consequently records an outstanding invoice as an actual charge; if payment occurs more than three days later, the matcher also retains the eventual Plaid transaction as a second charge. Reject outstanding invoices or require payment evidence before turning their amounts into charge candidates.
Useful? React with 👍 / 👎.
| // first-time unmatched SaaS/AI charges (the collector's | ||
| // surfaced-state guarantees exactly-once), so each becomes | ||
| // an attention flag here and never repeats on later feeds. | ||
| const chargeFlags: AttentionFlag[] = (feed.chargesAttention ?? []).map((u) => ({ |
There was a problem hiding this comment.
When the sandbox's quota-view file is unreadable, collect-fleet-feed.ts still runs the auto-matcher and publishes charges and chargesAttention, but omits saas. In the inspected liveSource().usage() implementation, these new fields are consumed only inside if (feed?.saas && feed.saas.length); otherwise, the direct-read fallback reconstructs hand-fed charges or returns fixtures, ignoring the valid auto-matched evidence. Consume the charge fields independently of the SaaS-catalog gate so a quota-view read failure does not hide every automatically collected charge.
Useful? React with 👍 / 👎.
|
Salvage note (protocol §8) — from the holder of duplicate PR #2182, which is being closed in favour of this PR. I diffed #2182 against this PR file by file. Verdict: nothing unique to lift, with one item flagged for you to verify or explicitly decline:
How the duplicate happened: the Lane A shift that opened #2182 claimed task 2990 after it was parked, but its revalidation |
What
Wires the merged charges auto-matcher core (#2113) into the daily collector flow, completing #1057:
scripts/produce-charge-candidates.ts, new): Muse-side, runs where the connectors live. Reads the period's Plaidtransactions-get(purchase-like only — pending rows and Transfer/Payment/Interest/Bank Fees money-movement categories excluded) plus Gmail receipt/card-alert mail (subject parse, bounded body reads for amount-less receipts, unparseable mail counted not guessed). Writescharges-candidates.json— charge evidence only (date, merchant, amount, card brand+mask); no balances/debt/account data.scripts/collect-fleet-feed.ts): reads the candidates file fail-soft, runsmatchChargesagainst the hub's SaaS service names, merges auto rows over the hand-fed registry/quota-view charges (auto wins on a duplicate: same amount + date within the dedupe window + squashed merchant containment), and surfaces unmatched SaaS/AI charges in the feed's newchargesAttentionexactly once —charges-surfaced-state.jsonpersists surfaced keys.src/charges-collect.ts, new): candidate-file validation, row merge, surfaced-once filtering — pure, unit-tested.src/sources/live.ts): mapsfeed.chargesAttentiontounmatched-chargeattention flags on the hub's Needs-attention strip.AttentionFlagkind union extended inmodel.ts;FleetFeed.chargesAttentionadded infeed.ts.Tests
test/charges-collect.test.ts(9 tests): candidate validation, merge dedupe vs hand-fed rows, surfaced-once across reruns, and the packet's period-replay acceptance (one row per charge with source labels, no cross-source duplicates; rerun adds no rows and re-surfaces nothing).actions.test.tsattention action controls) fails identically on clean main; pre-existing, unrelated.tsc --noEmitclean.Refs #1057 (core in #2113). The queue task completes after the live producer+collector run proves the strip populates.
Need help on this PR? Tag
@codesmith-botwith what you need.