Skip to content

Proposal: graceful restart (drain-and-restart) from the memory observability card #563

Closed

Description

@dev-shinyu

Context

Since v2.7.41 the dashboard memory card (from #427) lets users watch RSS grow in
real time. A process restart remains the only reliable way to reclaim that memory:
live response-store references and allocator page retention survive GC.

Restart already exists today:

  • ocx restart
  • Windows tray Restart Proxy
  • POST /api/stop → existing drainAndShutdown → ensure/restart

Important clarification: these paths are not a blind immediate kill.
/api/stop already performs a short graceful drain via drainAndShutdown, using
config.shutdownTimeoutMs (default 5s), then aborts leftover turns.

What users still lack is an informed, longer, dashboard-visible restart from
the memory card:

  • show the current in-flight count before confirming
  • wait longer for active turns to finish
  • make the drain/restart tradeoff explicit in the UI

Proposal

Add a memory-card action for a longer drain-and-restart, reusing the existing
lifecycle rather than inventing a second shutdown system.

Suggested behavior:

  1. Confirm dialog shows current in-flight count and the drain timeout.
  2. On confirm, stop accepting new data-plane requests.
    Reuse the existing draining response: 503 + Retry-After + JSON (503 from /v1/responses via Codex even though direct proxy curl to same endpoint returns 200 OK (Xiaomi MiMo provider, openai-chat adapter) #452).
    Do not queue requests.
  3. Wait for registered active turns (getActiveTurnCount() / existing turn
    tracking) up to a longer fixed timeout (60s for v1).
  4. On timeout, proceed with restart anyway (same abort-remaining-turns model as
    today's drainAndShutdown).
  5. Restart through the existing stop → ensure / restart infrastructure, including
    current port-reclaim behavior.
  6. While draining, the card shows progress such as
    Draining N request(s)… restarting when complete.

Wording

Use different words for the issue vs the button:

Surface Wording
Issue / design discussion graceful restart (drain-and-restart)
English UI button Drain & restart
English confirm copy Wait for N in-flight request(s), then restart (up to 60s; remaining requests are cut on timeout)
Korean UI button 작업 완료 후 재시작

Avoid these UI labels:

  • Graceful Restart as the button text — "graceful" already names today's short
    /api/stop drain, so the new action looks like a duplicate
  • Safe restart / 안전 재시작 — collides with tray Restart safety, which
    is a startup-health label, not a restart action

Positioning for reviewers:

This is not "add graceful restart where none exists".
It is "expose a longer, informed drain-and-restart on the memory card, on top of
the short drain that /api/stop already performs".

Why this is not a tray duplicate

  • Tray Restart Proxy stays the existing restart entrypoint.
  • Tray Restart safety is unrelated health labeling and must not be confused
    with this action.
  • The memory-card action is cross-platform dashboard UX with in-flight visibility
    and a longer drain window. That is the product difference.

Deliberately out of scope (v1)

  • Threshold auto-restart — remains deferred per 램 누수가 있어요 #314
  • Cancel-during-drain / resume normal traffic mid-drain
  • New user-facing timeout config key — keep 60s fixed for v1
  • Changing tray menu behavior
  • Changing watchdog warn-only policy
  • Queuing new requests during drain

Implementation sketch

Prefer reuse over new machinery:

  • Reuse isDraining, getActiveTurnCount, drainAndShutdown, and the existing
    503 draining response
  • Expose in-flight count on /api/system/memory or an adjacent scalar field
  • Add one management restart route that performs longer drain then existing
    restart/ensure path
  • GUI: in-flight metric + Drain & restart button + confirm dialog + draining
    progress
  • i18n for all six locales under dash.mem.*
  • Tests: lifecycle/route coverage + GUI render regression from day one
  • docs-site update; English source first, no locale contradictions

Privacy / safety

Scalar-only: counts, timestamps, statuses.
No request bodies, tokens, or account identifiers.
Confirm-gated to prevent misclicks.
bun run privacy:scan stays green.

Open defaults I am proposing

Unless maintainers prefer otherwise:

  1. Drain timeout: 60s fixed in v1
  2. On timeout: still restart
  3. Cancel-during-drain: not in v1
  4. No supervisor detected: allow with warning, do not hide the button
  5. New requests during drain: keep existing 503 behavior

If this direction sounds right, I would implement it as one focused full-stack PR
(management route + reused drain/restart path + GUI + i18n + tests + docs).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions