Skip to content

[BUG] grpc server job selection algorithm can starve clients when multiple clients continuously submit #2210

Open

Description

@tmckayus

Workaround (26.10 proxy and direct gRPC in 26.08+). The server runs jobs by scanning slots from slot 0, not in arrival order. A client that submits again as soon as its job finishes can take the lowest free slot and run ahead of jobs already waiting. Those waiting jobs can stay queued while other clients complete many times.

Keep at most one job in flight per worker. Use a single submitter, or a shared lock or FIFO in your
application, and submit the next job only after the current result is retrieved. With one worker,
extra jobs sitting in the server queue do not add throughput.

A one-shot batch of at most 100 jobs is fine if nothing else is submitted until that batch finishes
. The server rejects anything beyond 100 in flight with Job queue full (HTTP 500 through the proxy,
RESOURCE_EXHAUSTED on gRPC).

The legacy REST server is unaffected: it dispatches from a FIFO.

Activity

  1. tmckayus commented on Oct 24, 2026

    @tmckayus
    ContributorAuthor

    Addressed in #2209

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions