[curnane-lab]

Summary

Ascend NPU enablement for the native sglang-spec-capture plugin (host-publication route), plus a condensed native-plugin section in the existing Ascend NPU tutorial and short Ascend notes in the plugin README. Two implementation files, two test files, two existing-doc edits.

Plugin changes

• gpu_put_enabled(device): platform decisions follow the model runner's actual device instead of CUDA-only detection. Auto mode selects host publication on non-CUDA runners; explicit SGLANG_SPEC_CAPTURE_GPU_PUT=1 or MOONCAKE_PROTOCOL=nvlink on a non-CUDA runner fails at install time with a clear error, instead of crashing on torch.cuda.Event inside the first capture.
• Logical card resolution: _model_device() fills a missing device index from the runner's gpu_id (the NPU fork reports device="npu" and gpu_id separately), so the sink worker binds the actual card instead of defaulting to index 0. An explicit device index always wins.
• SpecCaptureSink(device=...): Ascend CPU-segment mounting (setup(0,0) + allocate_and_mount_segment(size, protocol, "cpu")) is keyed off the model device, with ASCEND_RT_VISIBLE_DEVICES kept only as a fallback hint. Mount semantics unchanged.
• Sink writer thread device context: the thread binds its NPU device context (logical index) before Mooncake setup, fixing ACL_ERROR_RT_CONTEXT_NULL for capture producers that never ran init_distributed. No-op on CUDA/CPU.

CUDA/CPU/ROCm behavior is preserved. Auto mode keeps the historical torch.version.cuda backend check (ROCm stays on host publication), and an explicit GPU_PUT=1 on CUDA-type devices (ROCm included) keeps its old semantics — only a known non-CUDA runner is rejected. No public env or protocol renames.

Tests

New unit tests in tests/test_runtime/test_sglang_capture_plugin.py and tests/test_runtime/test_sglang_capture_plugin_sink.py: 60 passed, 6 skipped under torch 2.10 + torch_npu, and 59 passed, 7 skipped with TORCH_DEVICE_BACKEND_AUTOLOAD=0 (no NPU backend registered). NPU devices in tests are stand-ins/local mocks, so the suites also pass on plain CPU/CUDA hosts; one real-NPU parse test is backend-gated. black/isort clean.

Testing on hardware (8x 910B3, Qwen3.5-4B GDN hybrid + DFlash, host publication + TCP)

Validated combination: SpecForge pr/npu/spec-capture-ascend @ 67bb423 (this PR); SGLang = NPU fork npu/capture-forward-observer @ b462f44827 (fork head 88a0368750 + the already-merged #43450 cherry-pick; #43449 was already in the fork); torch 2.10.0+cpu + torch_npu 2.10.0, transformers 5.12.1, CANN 9.0.0, mooncake-transfer-engine 0.3.13.post1 (TCP host path).

Full verification log and migration guide live on the fork's archive branch, pinned:

• Validation log + SHA inventory
• Migration guide + verification scripts (scripts under scripts/npu/ on the same branch)

Highlights:

• Capture → Mooncake → separate-process client read-back: aux payload bit-identical to the engine's own response rows; passthrough input_ids byte-exact.
• Final-head nonzero logical-card check (67bb423, visible 3,4, base_gpu_id=1): trace shows model_runner.device='npu', gpu_id=1, _model_device() resolving npu:1; the sink is configured with npu:1 and its worker reports current_device=1. Publication and read-back pass (VERIFY_E2E_OK).
• Actual published target numerics: the Mooncake-read-back target matches a capture-off post-norm reference on the same logical card for 9/9 rows bit-exactly (max_abs=0, every per-row relative error 0), rather than merely checking finite values.
• Trainer-side MooncakeFeatureStore: CPU get bit-identical; npu:0 receive + H2D intact; release() frees objects and a later get() raises KeyError.
• Numerics vs an HF fp32 CPU reference at TP1 and TP2: aux chunk for layer L is that layer's output residual (cos ≥ 0.9999), final hidden post-norm, greedy next token matches HF.
• Prefill graph lifecycle through the real plugin chain (real spec_capture requests, test-only delayed sink): other requests kept executing (including graph replays) while capture A's publication was still pending, and A's read-back is consistent — published aux bit-identical to the eager reference, isolated repeat A2 bit-identical to A. The plain return_hidden_states graph path is verified separately.
• DFlash training-step smoke (managed-local: capture server + producer + single-rank trainer): finite loss, clean control-plane lifecycle.
• Offline round-trip (the route [2/4] offline: prepare target features through the capture plugin #1064 proposes, exercised with a verification script, not a public CLI): 8/8 samples byte-exact on reload; checkpoint resume bit-exact; online vs offline per-step losses bit-identical once feature bytes, init, sample/batch order and optimizer/RNG state are aligned. This surfaced an upstream ordering asymmetry: online consumes in random.Random(seed + epoch) order, offline in torch.randperm(seed + epoch) order.

Known limits (documented in the guide)

• Host publication + TCP only; device-direct publication is rejected at install time on NPU pending an Ascend Mooncake transport probe.
• Trainer receive is pageable host memory + explicit H2D; no pending-bytes budget yet.
• Validated on the DFlash route only; EAGLE3/DSPARK routes are not yet gated on NPU.

Notes

• Server side was validated on an NPU SGLang fork carrying #43449 + #43450 (both already merged upstream); this PR changes only the SpecForge plugin/tests/docs.
• [2/4] offline: prepare target features through the capture plugin #1064/[3/4] capture: remove legacy SGLang patch support #1065 (legacy offline-backend and server-patch removal) are still open; the guide describes them as in-flight and marks the two-patch instructions as the legacy 0.5.18 route.

Related: #1063 #1064 #1065 #1066.

Checklist

• Format your code according to the Code Formatting with Pre-Commit.
• Add unit tests as outlined in the Running Unit Tests.
• Update documentation / docstrings / example tutorials as needed, according to Writing Documentation.
• Provide throughput / latency benchmark results and accuracy evaluation results as needed, according to Benchmark and Profiling and Accuracy Results.
• For reviewers: If you haven't made any contributions to this PR and are only assisting with merging the main branch, please remove yourself as a co-author when merging the PR.
• Please feel free to join our Slack channel at https://sgl-fru7574.slack.com/archives/C09784E3EN6 to discuss your PR.