[ishandhanani]

Summary

Makes the two-tier cost function (dynamo-two-tier-cost-fn) the default worker-selection policy for --router-mode kv, and logs a startup notice so users know about the change. Closes DYN-5308.

• lib/router-plugins/builtin/src/two_tier_cost_fn.rs: new default_factory() that hosts install as the default. It uses the upstream sgl-router thresholds (cache_threshold 0.5, balance_abs_threshold 32, balance_rel_threshold 1.1) and keeps session-affinity targets exclusive, as the previous default did.
• Where the previous default read no cache overlap (plain disagg decode, the --load-aware preset, overlap_score_credit: 0), the default two-tier picker declares only WorkerInputs::LOAD and selects the worker with the fewest active requests. The host therefore runs no KV indexer or event subscription for those routers, as before. Configured dynamo-two-tier-cost-fn instances are unchanged.
• lib/llm/src/kv_router/plugins.rs: the installed registry and the fallback registry both use the two-tier default factory.
• lib/llm/src/entrypoint/input/http.rs: logs a WARN at frontend startup when --router-mode kv runs without a configured worker-selection policy:

  Starting with Dynamo 1.8, --router-mode kv defaults to the two-tier cost function (dynamo-two-tier-cost-fn). To keep the previous cost function, select a dynamo-default-cost-fn instance in --router-policy-config.

• Docs: the router configuration page describes the new default and how to opt out.

To opt out, configure an instance of the previous cost function:

worker_selection:
  aggregated: previous
  prefill: previous
  instances:
    - name: previous
      type: dynamo-default-cost-fn

Behavior changes for default --router-mode kv

• Cost-function flags have no effect. overlap_score_weight/overlap_score_credit (except 0, which turns off the cache tier), router_temperature, prefill_load_scale, and the host, disk, and shared cache weights all tune the previous cost function. Two-tier reads only its three thresholds.
• PATCH /router_policy returns 422 for default pools, because two-tier has no runtime parameters. GET /router_policy lists them with configuration: null.
• The load signal is the active request count, not prefill tokens or decode blocks. This also applies to plain decode pools. Ties go to the first candidate row, where the previous default sampled randomly among ties.

Open items

• Threshold values: per the ticket, confirm with Guan before merge. The defaults are the sgl-router cache_aware_zmq values.
• Release timing: main is still 1.7.0 and release/1.7.0 is not cut yet. Merge after the cut so this ships in 1.8 as the notice says.

Validation

• cargo test -p dynamo-custom-policy-builtin: all pass.
• cargo test -p dynamo-llm --lib: 3920 passed, 0 failed. Two existing tests changed:
  • selection_matches_frozen_frontend_trace now pins dynamo_custom_policy_builtin::default_factory(), so the frozen trace keeps covering the additive cost function.
  • kv_chooser_ignores_stale_presence now asserts that the default pool reports no runtime parameters.
• Not run: the Python bindings crate and e2e router tests. No end-to-end routing benchmark yet comparing two-tier against the previous default.