SearchService: bound ListFilterValues response size for unbounded fields #2124

Summary

SearchService.ListFilterValues (added in #2121) returns every distinct value for a field in one unary response. Four of its five fields are naturally bounded, but AUTHOR is not: a registry can hold up to one distinct author per record. A large enough registry therefore produces a response that exceeds the gRPC message limit, and the RPC fails outright with ResourceExhausted instead of returning its facets.

Raised by Copilot in review of #2121 and deferred to keep that PR focused.

Why the limit is effectively hard

The server-side ceiling is configurable — MaxSendMsgSize defaults to 4 MiB (server/config/config.go:55-57) and is settable via max_send_msg_size or DIRECTORY_SERVER_MAX_SEND_MSG_SIZE, applied at server/server.go:90.

The client-side ceiling is not. client.Client dials with transport credentials only (client/options.go); nothing sets grpc.WithDefaultCallOptions(grpc.MaxCallRecvMsgSize(...)) anywhere in client/, cli/, or utils/. Every consumer in this repo, dirctl included, therefore sits on gRPC's built-in 4 MB receive default with no way to raise it.

So an operator who hits this cannot configure their way out. Raising the server limit only moves the rejection to the client.

Scale

Roughly 4 MiB / ~40 bytes ≈ 100k distinct authors before the response becomes unreceivable. Not near-term for any current deployment, which is why this is filed rather than fixed. SKILL_NAME and DOMAIN_NAME are bounded by the OASF taxonomy, and MODULE_NAME and SCHEMA_VERSION are small, so AUTHOR is the only field with this property.

Options

Cap per field plus a truncated flag. Bounds the response by construction and degrades to a partial list rather than a hard failure. Small change, additive to the proto, and the only option requiring no deployment configuration. This is the suggested default.

Per-field streaming does not fix it. A request carries at most five fields, so a stream emits at most five messages and AUTHOR is still one message carrying every value. The limit is enforced per message, so it fails identically. Only chunking within a field would help, which needs the same magic constant as a cap while giving up the one-entry-per-field response shape and offering no resume.

Page tokens per field. Most complete and most machinery, for a ceiling nobody has reached.

Note that everything except a unary-to-streaming switch is a backward-compatible proto change, so deferring forecloses nothing but that one option.

Related

client.Client never sets MaxCallRecvMsgSize. That is a general SDK gap rather than a ListFilterValues problem — it will constrain any large response — and may deserve separate treatment.