You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[BUG] grpc server job selection algorithm can starve clients when multiple clients continuously submit #2210
Workaround (26.10 proxy and direct gRPC in 26.08+). The server runs jobs by scanning slots from slot 0, not in arrival order. A client that submits again as soon as its job finishes can take the lowest free slot and run ahead of jobs already waiting. Those waiting jobs can stay queued while other clients complete many times.
Keep at most one job in flight per worker. Use a single submitter, or a shared lock or FIFO in your
application, and submit the next job only after the current result is retrieved. With one worker,
extra jobs sitting in the server queue do not add throughput.
A one-shot batch of at most 100 jobs is fine if nothing else is submitted until that batch finishes
. The server rejects anything beyond 100 in flight with Job queue full (HTTP 500 through the proxy,
RESOURCE_EXHAUSTED on gRPC).
The legacy REST server is unaffected: it dispatches from a FIFO.
Activity
tmckayus commentedon Oct 24, 2026
Addressed in #2209