Skip to content

Give one-day Magic Compose drafts a Date column, so their reminders go out - #693

Open
richshaw wants to merge 1 commit into
mainfrom
claude/issue-685-one-day-date-field
Open

richshaw wants to merge 1 commit into
mainfrom
claude/issue-685-one-day-date-field

Conversation

@richshaw

Copy link
Copy Markdown
Owner

Fixes #685.

The problem

When a Magic Compose description covers one day ("bake sale on Saturday, November 14, 9am to 1pm…"), the draft usually puts the date in the description and gives the slots no Date column. A signup with no date field has no reminderFromFieldRef and its slots have no slot_at, so nobody who signs up gets the day-before reminder, and nothing tells the organizer. The rule that would have prevented this, USE_DATE_AND_TIME_FIELDS, only went to connected AI assistants. #475 kept it out of the Magic Compose prompt until it had an eval run.

The fix

  • Shared rule. USE_DATE_AND_TIME_FIELDS moves into RULES_IN_BOTH (src/lib/signup-rules.ts), so the drafter and assistants read the same words, and the tests on both sides check that.

  • One day is named. The rule now reads: "When slots happen on a day or at a time, add a date field, even if all are on one day, and a time field if needed. Reminders are timed from the date field: with a text field like "Sat 9am" nobody gets one." The issue said to add the one-day clause only if the model still left the date out. I added it up front, because the reported failure is exactly the one-day case and I had no model to measure with (see Before merging).

  • Byte budget. My first wording put the MCP instructions at 2101 bytes before the tools line, over the 2048 limit route.test.ts checks. "Reminder emails" became "Reminders" and the clause got shorter, so the instructions are now 2045 bytes. A comment by the constant says every word there counts against that budget.

  • The prompt (src/lib/magic-compose/prompt.ts) gets a DATES AND TIMES — <rule> See Example F. line right after the field type guide. Example F is new: a one-day car wash with two jobs, where every slot carries a Date (2026-06-06) and a start Time (09:00). It's a list of jobs on one day, the same shape as the failing case, and it uses its own words, so the test that compares worked examples with eval prompts still passes. As the issue asked, a time range gets only a start time.

  • Golden file regenerated. Its diff is just the new line and Example F.

  • Evals. Two made-up cases in evals/magic-compose/dev-evals.json. Each needs a date field and slot_dates.set_must_equal on the one date, which means that date is in every slot:

    • ev_038: a one-day spring fair with roles and a 10am-to-3pm range. It also needs the start time.
    • ev_039: a one-day block-party dish list.

    Both use existing harness checks, so the harness itself doesn't change.

  • CHANGELOG has an entry under Fixed.

Tests

  • prompt.test.ts now expects worked examples A–F. A new test checks that Example F gives every slot the same date and start time.
  • Ran and passed: pnpm lint, pnpm typecheck, pnpm knip, pnpm test (157 files, 1996 tests, including the instructions byte-budget test and the word-for-word checks on both sides), and pnpm test:db src/mcp.

Before merging

This session had no LLM endpoint, so I haven't run the model-panel eval that the issue's "Done when" asks for and that the golden-file test's comment requires before a prompt change. Please run it on this branch and on main (3 runs per side, across the OpenRouter panel used for #526 and #577):

pnpm eval:magic-compose

Watch for:

  • whether ev_038 and ev_039 now get a date field;
  • the date-field rate on one-day prompts like the issue's example;
  • drift on the item-list cases, which should still have no date field: ev_001, ev_016 and ev_032;
  • drift on the "TBD:" placeholder cases, which should invent no dates: ev_034, ev_035 and ev_037.

Generated by Claude Code

A one-day description usually came back with the date in the
description only. With no date field the signup has no reminder anchor,
so nobody who signs up gets the day-before email.

The date-and-time rule connected assistants already had now goes to the
drafter too, through RULES_IN_BOTH, and says it holds when every slot is
on the same day. The prompt gains a DATES AND TIMES line and a worked
example of a one-day event with Date and Time columns, the golden file
is regenerated, and two single-day eval cases are added. The rule is
reworded to keep the MCP instructions inside their 2048-byte budget.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016YZ9SU29EH5yo7dguphwJD
@richshaw richshaw added pipeline:ux Changes what a person using OpenSignup sees or does pipeline:working A session is working on it right now labels Oct 27, 2026 — with Claude
@richshaw
richshaw requested a balanced review from Copilot October 27, 2026 11:05

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.

@richshaw richshaw added pipeline:tophat pipeline:working A session is working on it right now and removed pipeline:working A session is working on it right now labels Oct 27, 2026 — with Claude

Copy link
Copy Markdown
Owner Author

Tophat: passed

Ran b49e9e0 locally with pnpm dev, Postgres 16, and Chromium at 1280×900 and 390×844. I didn't run the worker. This change only alters what the drafting model reads, and reminders are anchored when the signup is created. There's no real model in this session, so I pointed LLM_BASE_URL at a small local mock of the Chat Completions API. The mock saved each request it got and answered with a one-day draft that has Date and Time fields. So this tophat shows what the drafter is now told and what happens to a one-day draft with a Date column. It can't show how often a real model now adds one. That's the model-panel eval the PR's "Before merging" asks the owner to run. I used made-up data: pat@example.com, and "Bake sale on Saturday, November 14, 2030, 9am to 1pm. We need 2 people to bring baked goods, any number to work a sales shift, and 3 to bring drinks."

Check Result What I saw
Before (main at e8ab9b1): what the drafter reads ✅ gap confirmed main's pinned prompt (__golden__/system-prompt.txt) has no date-field rule and no Example F. Its only worked example of a list is Example E, which has no date field
After: the system prompt the drafter receives ✅ The request the mock got through Draft my signup carries DATES AND TIMES — When slots happen on a day or at a time, add a date field, even if all are on one day, and a time field if needed. Reminders are timed from the date field: with a text field like "Sat 9am" nobody gets one. See Example F. It also has Example F, the one-day car wash with 2026-06-06 / 09:00 on both slots
A one-day draft with a Date column gets its reminder anchor ✅ The created signup has reminderFromFieldRef: date and sendReminders: true. All 3 slots have slot_at set to 2030-11-14 09:00. The Build tab shows "Thu, Nov 14 · 9:00 AM" on each row, and the banner reads "Here's your draft, 3 columns and 3 slots" (693-after-desktop-build)
Connected AI assistants (MCP instructions) ✅ buildInstructions(TOOLS) carries the same new sentence, and everything before the tools line is 2044 bytes, within the 2048 limit
Phone, 390×844 ✅ The same flow drafts and opens Build, with no horizontal scroll (scroll width 390) (693-after-phone-build)
Unit tests ✅ pnpm test src/lib/magic-compose src/lib/signup-rules src/mcp: 18 files, 271 tests pass. That covers the golden prompt, the worked-example checks, RULES_IN_BOTH word for word on both sides, and the instructions' byte budget
Browser console and server log ✅ No console errors. The only server-log errors are the health check's pgboss.job probe, which fails because the worker wasn't running

The mock's date, 2030-11-14, is a Thursday, so the rows read "Thu". That comes from my test data, not from this PR.

For the owner (UX): nothing on screen changes. What changes is what Magic Compose and connected AI assistants are told: give a signup a Date column even when everything is on one day. When a draft follows that rule, people who sign up get the day-before reminder, as the third row shows. This tophat can't tell you how often real models now follow it. Please run the model-panel eval (pnpm eval:magic-compose on this branch and on main) as the PR's "Before merging" asks, before you approve. See 693-after-desktop-build and 693-after-phone-build.

Screenshots are in https://claude.ai/code/session_016YZ9SS8d7RXpGsMtTDVwGd.


Generated by Claude Code

@richshaw richshaw added pipeline:review pipeline:working A session is working on it right now and removed pipeline:tophat pipeline:working A session is working on it right now labels Oct 27, 2026 — with Claude

@richshaw richshaw left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review: passed (round 1)

The PR moves USE_DATE_AND_TIME_FIELDS into RULES_IN_BOTH, adds the one-day clause to it, puts it in the Magic Compose prompt under the field type guide, and adds Example F (a one-day car wash where every slot has a Date and a start Time). It also regenerates the golden prompt and adds two eval cases. The change is small and does what it says. The shared constant reaches both readers word for word: prompt.test.ts:60 and instructions.test.ts:24 both loop over RULES_IN_BOTH, and the MCP instructions still use the same constant at src/mcp/instructions.ts:45, so the wording can't drift. June 6, 2026 really is a Saturday, and Example F's prompt isn't copied from any eval case. The new eval keys (fields.must_include_field_type, slot_dates.set_must_equal, slot_values_must_cover_all) are ones the harness already handles.

Blocking (0)
Optional (1):

  • evals/magic-compose/dev-evals.json (ev_038): the description says this case "also needs the start time". But slot_times_or_description_must_mention searches the whole draft, description included, and passes when any one of the strings appears. So a draft with no Time field still passes as long as its description says "10am". Requiring a time field would need a new harness check. The eval is a smoke signal, so it's fine as it stands.

Earlier threads: none. Copilot couldn't review (its quota was used up).
Checks: CI is green on b49e9e0 (lint-type-test, e2e, CodeQL). The tophat passed on b49e9e0, the same commit, so no runtime code has changed since. Locally I only read the diff, the eval harness's handlers for the new keys, and src/mcp/instructions.ts.

Why it waits for the owner: it's UX (it changes what Magic Compose drafts and what connected assistants are told), and the PR asks the owner to do something before merging. Issue #685's "Done when" asks for a model-panel eval run, PR vs main, and that run hasn't happened yet: this pipeline has no LLM endpoint. Before approving, please run pnpm eval:magic-compose on this branch and on main (3 runs each, on the OpenRouter panel). Check that ev_038 and ev_039 now get a date field, and look for drift on the item-list cases (ev_001, ev_016, ev_032) and the TBD cases (ev_034, ev_035, ev_037). Nothing changes on screen. The tophat's 693-after-desktop-build shows a one-day draft that has its reminder anchor.

https://claude.ai/code/session_016YZ9SSy8e2nEbatBBDPC4L


Generated by Claude Code

@richshaw richshaw added pipeline:owner Passed; waits for the owner (UX or high-risk) and removed pipeline:review pipeline:working A session is working on it right now labels Oct 27, 2026 — with Claude
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pipeline:owner Passed; waits for the owner (UX or high-risk) pipeline:ux Changes what a person using OpenSignup sees or does

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Magic Compose puts a one-day signup's date in the description, so its reminders never go out

3 participants