[ishandhanani]

Stacked on #99 (SGLang backend), which stacks on #102 (infrastructure). Only the last commits are this PR's.

Summary

Adds vLLM to the frontend's direct mode: python -m dynamo.frontend --direct --direct-backend vllm --engine-url http://host:30001 ... routes to stock vllm-rs serve --grpc-port servers over their native gRPC API with no sidecar.

• dynamo-vllm-sidecar::direct: probe over the existing VllmClient discovery (GetModelInfo, GetServerInfo, GetKvEventSources, including the replay socket) and a VllmDirectDispatch whose stream body is shared with LLMEngine::generate through prepare_generate / open_and_consume. On the direct path the dispatcher sends Control.Abort on cancellation itself, keeping the decode-leg deferral until the KV transfer completes.
• vLLM does not report its prefill/decode role over gRPC (its ServerInfo has no such field), so roles come from the operator: --prefill-engine-url lists prefill engines, --engine-url the rest.
• Parser names stay the operator's call, as with the sidecar's --dyn-* flags: vLLM names its parsers differently from Dynamo (qwen3_xml where Dynamo has qwen3), so the probe leaves the card's parser fields empty and reports the engine's names in its probe log. Found live: the first start against a stock vLLM engine failed the card's parser check on qwen3_xml.
• --direct-backend vllm; the direct integration tests run for both engine families; docs cover the vLLM recipe and the operations surface that still needs the sidecar (dynamic LoRA, RL controls, encode role).

Validation

• cargo clippy -D warnings --tests clean on dynamo-sidecar-common, dynamo-sglang-sidecar, dynamo-vllm-sidecar, dynamo-llm, dynamo-sidecar-testkit; bindings cargo check; cargo fmt --check; pre-commit clean.
• dynamo-vllm-sidecar 133 unit tests (one new: the probed card leaves parsers to the operator). direct_mocker_integration.rs 6/6, the three scenarios for both the SGLang and vLLM mockers. test_frontend_direct_args.py 6/6.

Live vLLM 0.28.0 (vllm-rs) on 2x H100: aggregated with KV routing and disaggregated over NIXL, see the comment below (recorded on the pre-split branch; the code is the same).

[ishandhanani]

Live validation: vLLM 0.28.0 (vllm-rs serve --grpc-port), 2x H100, Qwen3-0.6B

Branch head 2b63548681. Engines started with vllm-rs serve ... --grpc-port 18701|18711 --kv-events-config '{"enable_kv_cache_events":true,"publisher":"zmq",...}' --reasoning-parser none; no Dynamo process beside them. Frontend: python -m dynamo.frontend --direct --direct-backend vllm ... --router-mode kv --discovery-backend mem --dyn-tool-call-parser hermes.

Aggregated, two engines, KV routing

Probe (one line per URL; the engine's own parser names are reported, not adopted):

probed vLLM engine for direct routing endpoint=http://127.0.0.1:18701 mode=Aggregated model=/data/models/Qwen/Qwen3-0.6B kv_event_sources=1 vllm_tool_call_parser="qwen3_xml" vllm_reasoning_parser="none"
probed vLLM engine for direct routing endpoint=http://127.0.0.1:18711 mode=Aggregated model=/data/models/Qwen/Qwen3-0.6B kv_event_sources=1 vllm_tool_call_parser="qwen3_xml" vllm_reasoning_parser="none"

Smoke: 4 chat completions returned finish_reason=length with 40 completion tokens each; streaming returned 33 chunks. Both workers took traffic. The same-prefix pair pinned to one worker on KV overlap read from vLLM's ZMQ events:

[ROUTING] Best: worker_3458080756468540 dp_rank=0 with 0/1 blocks overlap
[ROUTING] Best: worker_3627132050719741 dp_rank=0 with 0/4 blocks overlap
[ROUTING] Best: worker_3627132050719741 dp_rank=0 with 3/4 blocks overlap

The first start, before 2b63548, failed the card's parser check with tool_call_parser 'qwen3_xml' is not supported; that is what the last commit fixes.

Prefill/decode, one prefill engine and one decode engine

Engines: --kv-transfer-config '{"kv_connector":"NixlConnector","kv_role":"kv_producer"}' on GPU 2 (port 18701) and kv_consumer on GPU 3 (port 18711). Frontend: --prefill-engine-url http://127.0.0.1:18701 --engine-url http://127.0.0.1:18711 --router-mode kv.

Probe: 18701 mode=Prefill, 18711 mode=Aggregated from the engine, then converted to decode by the fleet because a prefill engine exists (serving directly dialed engines decode_engines=1 prefill_engines=1). Smoke: 4 chat completions finish_reason=length with 40 tokens each, streaming 33 chunks. Every routed request went through both legs:

     13 prefill_worker_id=3458080756468540
     13 decode_worker_id=3627132050719741

The decode engine's log confirms the KV pull from prefill over NIXL (NIXL compatibility check passed, Transfer plan: TransferTopology(tp_ratio=1.0, ...)). Three streaming requests with 48 output tokens:

request	TTFT	mean ITL
0	133.7 ms	16.84 ms
1	118.0 ms	16.79 ms
2	121.2 ms	16.69 ms

ITL reflects this vLLM build (FlashInfer sampler disabled because the host CUDA toolkit does not match the wheel), not the routing path; the SGLang run on the same GPUs in #99 saw ~1.4 ms.

No sidecar, etcd or NATS ran at any point; the frontend used --discovery-backend mem.