[richshaw]

Part of #570, the second of five stacked PRs. It's based on #572 (friendly limit messages). It's dormant on its own: the prompt doesn't ask questions until #577, so nothing an organizer sees changes.

What it adds

Magic Compose can now answer a description with a short list of what's missing instead of a draft. The organizer's answers can come back in a second request. That second request must end in a draft or a refusal, never more questions.

The model's questions reply

{ "questions": { "topics": ["what", "when", "how_many"], "whatOptions": ["Bake something to sell", "Staff the table"] } }

• QuestionsSchema in prompt.ts is .strict() at both levels.
  • topics: one to three of what, when and how_many, each once, always with what.
  • whatOptions: two to four distinct options of up to 40 characters, tidied the way the answers are.
• It's the middle arm of MagicComposeDraftSchema.
  • A refusal still wins, as before.
  • The draft arm refuses a questions key, so a reply that mixes questions with half a draft fails.
  • Null top-level keys are dropped first, as models in json_object mode send them, so a draft with "questions": null is still a draft.
  • FullDraftSchema, which the MCP create_signup tool uses, is unchanged.
• A first request the model answers with questions gets 200 { data: { questions } }, with no signup and no links. It's logged at info with the topics and the number of options, never the options.

The second request

POST /api/signups/magic-compose takes an optional followUp, one key per topic that was asked:

{
  what: { picked: string[]; added: string[] };   // ≤ 4 picked, ≤ 12 in total, 1–80 chars each, no repeats
  when?: { kind: 'one_day', date, from?, to? } | { kind: 'several', text } | { kind: 'not_decided' };
  howMany?: (number | null)[];                    // one per item, picked then added; whole 1–9999, null = no limit
  placeholders?: true;                            // Draft with placeholders: every answer blank
}

• Checks. FollowUpSchema (src/lib/magic-compose/follow-up.ts) requires:

  • a real date;
  • times that exist, with the end after the start;
  • whole-number counts lined up with the items;
  • no repeated items, ignoring case;
  • no answers alongside placeholders.

  Text goes through tidyText (shared with the page). A bad followUp is invalid_input on field: 'followUp', before the quota is charged.

• The message. buildFollowUpMessages puts the description and the answers under fixed labels in one user message, with the organizer's own words JSON-quoted:

  Bake sale for the PTA
  
  Answers to your questions:
  - What people sign up for: "Bake something to sell", "Staff the table"
  - When: 2026-11-14, 09:00 to 13:00
  - How many people for each: "Bake something to sell": no limit, "Staff the table": 2
  
  Use these exact values. Draft the signup now. Don't ask questions.
  

  A skipped topic reads "not given, use a TBD: placeholder". A topic that wasn't asked has no line.

• If the model asks again. The route retries once with a firmer last line and no extra quota.

  • If the retry asks too, it answers invalid_input with details.reason: 'asked_again' and "Couldn't draft from these answers. Add a little more, or draft with placeholders." After Draft with placeholders, the message is "Couldn't draft this. Try again, or fill it in by hand."
  • A reply counts as asking again whatever its shape (asksQuestions in prompt.ts). After the answers, a model most likely asks about less than the first time.

• Time. The whole request gets one LLM_TIMEOUT_MS. generateDraft takes an optional timeoutMs, so the retry gets what's left, and running out is still a timeout. With under 15 seconds left (or half the budget, if that's less), or the organizer gone, it skips the retry. That keeps a follow-up inside the browser's 210-second watchdog.

• Record link copies, failed AI drafts and refused publishes #525's failure telemetry. Every failed path still records magic_compose.failed, and asked_again joins its codes (aborted when the organizer had left). A first call answered with questions isn't a failure and records none. docs/telemetry.md lists the new code.

• Privacy. The privacy page now says any answers go to the AI provider too.

Where this differs from the issue's wording

• what arrives as picked and added rather than one items list. Record when Magic Compose asks questions, and how they were answered #575 records how many of each without the browser sending the questions back.
• howMany is an array aligned with the items, not an object keyed by the organizer's text.
• placeholders: true marks Draft with placeholders.

Tests

• prompt.test.ts:
  • the issue's schema cases (accepts a questions reply; rejects one without what, a repeated topic, one or five options, a reply mixing questions and draft keys, and a reply with none of the three);
  • null keys;
  • tidied options;
  • asksQuestions.
• follow-up.test.ts: every check above, and the message word for word.
• llm-client.test.ts: a shorter timeoutMs times out as timeout and can't lengthen LLM_TIMEOUT_MS.
• route.questions.db.test.ts, with the real route, quota and database and a stubbed model:
  • a name-only prompt gets questions and creates no signup;
  • a follow-up drafts;
  • questions twice give asked_again after exactly one retry and one quota charge;
  • a partial or mixed questions reply is retried;
  • a draft with "questions": null drafts in one call;
  • a retry that times out;
  • a refusal after answering;
  • an aborted request;
  • the retry floor, with the clock moved on;
  • a bad date, times out of order and a fractional count are rejected before the quota or the model.
• route.failed.test.ts (Record link copies, failed AI drafts and refused publishes #525's) gains asked_again, and "no failure for a first-call questions reply".
• pnpm lint && pnpm typecheck && pnpm knip && pnpm test (1423) and pnpm test:db (555) pass.

Tophat

This branch on its own, against a local fake OpenAI-compatible endpoint, through the API:

• "Bake sale" gets the questions;
• answers create a signup;
• a model that asks twice gets one firm retry, then asked_again;
• a bad date, or placeholders with answers, is refused before any model call;
• the server log holds topics and lengths only, no answer text.

On this branch alone, the existing page shows "AI drafting failed (HTTP 200)" if a model ever answers with questions. That's expected: the screen comes in #576, and this branch's prompt never asks.

Reviews

• The first review, fixed in 06981ec: the route only noticed a re-ask that passed the strict schema.
• A second review (/code-review), fixed in a83a270:
  • a draft with "questions": null failed (and was retried as a re-ask);
  • options and answers were tidied differently.
• Left as is:
  • overnight times (the issue asks for times in order);
  • both calls counting against the hourly limit (the issue's decision; the answers stay on screen if the limit is hit);
  • the existing log of malformed model output on a schema failure (it predates this PR, and it's how operators debug).

main (with #525 and #526) is merged in.

🤖 Generated with Claude Code

https://claude.ai/code/session_01UW9ZpEu4MWkeqTv2Qe14ks