[chienchunhung]

Summary

Existing KV-transfer deadlines include readiness and queue waiting; they do not provide an independent limit for a backend-accepted operation. Add a default-off submitted-operation timeout while preserving existing wait budgets and physical-resource retirement rules.

Scope

• Add kv_transfer_operation_timeout_ms, limited to the Python transceiver's qualified no-retry, generation-first FP4 MLA ownership profile; startup validation rejects unsupported profiles and inconsistent executor-rank budgets.
• Start one operation deadline immediately after the backend returns an accepted handle, shared by the existing KV/AUX submission path; post-acceptance bookkeeping and backend queueing consume it, while readiness, sender queueing and the submission call remain outside it.
• Use the C++ NIXL agent's exact wait result to distinguish timeout from backend failure; actual service start remains unknown, so this is not an on-wire timeout.
• Preserve request/KV/safety/grace deadlines and reuse existing IN_DOUBT retention and late settlement; expiry closes subsequent sender submission, and late completion does not clear that poison, so an aggressive budget can require executor recovery.
• Add configuration, schema, timing, cross-rank activation and real KV/AUX delivery regressions; no new cancellation protocol, ownership ledger or timeout thread.

Verification

Draft head: 695e9bba704a1626131b85c89eebd4c5229c84a1 (base 6fc7f47a322d88037ffe9eca6913526f1f05a4fb). This draft is incomplete and not ready for approval.

• Full source self-review found no actionable correctness/safety issue; git diff --check, Python 3.13 syntax checks, applicable pre-commit checks and DCO checks passed. The advisory Pyrefly hook did not execute type checking because Pyrefly is unavailable.
• Required generated artifacts are still pending: regenerate and inspect tensorrt_llm/usage/llm_args_golden_manifest.json and tensorrt_llm/schemas/*.json in a matching runtime before this PR is ready. They have not been hand-edited or waived.
• Runtime tests have not executed. Local collection and generators stop on missing Torch; remote execution is currently authorization-blocked. No GPU/profile qualification or performance claim is made. Required native-wrapper cases must execute rather than skip.
• Full CI has not been requested; it remains gated on generated-artifact completion and successful targeted verification of the final head.

Targeted verification (still required):

python scripts/generate_llm_args_golden_manifest.py
python scripts/generate_trtllm_serve_schemas.py
python scripts/generate_llm_args_golden_manifest.py --check
python scripts/generate_trtllm_serve_schemas.py --check
python -m pytest -m cpu_only -ra \
  tests/unittest/disaggregated/test_agent.py::TestBindingsNixlTransferStatus \
  tests/unittest/disaggregated/test_retirement_deadline.py \
  tests/unittest/disaggregated/test_lifecycle_activation.py \
  tests/unittest/disaggregated/test_kv_transfer_fail_stop.py \
  tests/unittest/disaggregated/test_disagg_transfer_diagnostics_wiring.py \
  tests/unittest/llmapi/test_llm_args.py::TestTransferOperationTimeout \
  tests/unittest/llmapi/test_trtllm_serve_config_schemas.py

Notes

Related #21734 adds a separate request-wide deadline; this PR targets main and does not reset or replace that budget. The new configuration field requires telemetry/privacy CODEOWNER approval. A tracking issue has not yet been assigned.

[chienchunhung]

/bot run --disable-fail-fast

[tensorrt-cicd]

PR_Github #83582 [ run ] triggered by Bot. Commit: 695e9bb Link to invocation

[tensorrt-cicd]

PR_Github #83582 [ run ] completed with state SUCCESS. Commit: 695e9bb
/LLM/main/L0_MergeRequest_PR pipeline #69096 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation