[Feature]: Opt-in BF16 norm–SiLU fusion for Wan VAE decoding #19309

Feature, motivation and proposed scope

The native BF16 Wan VAE decoder computes channel normalization, affine scaling and SiLU in separate operations. Add an initially opt-in internal path using the existing Apache-2.0 SGLang norm–SiLU kernel, preserving its attribution and BF16 rounding boundaries.

The proposed scope is decoder residual norm1/norm2 and decoder norm_out. Encoder and attention normalization remain native. Current FP4 normalization/SiLU absorption, fused convolution residuals, temporal caching and checkpoint quantization paths must retain their behavior. Unsupported inputs take the native path; the draft keeps the feature disabled by default while production validation is pending. No public VisualGen API field is proposed.

Prototype evidence and limits

A separate same-process prototype on one B200 with TensorRT-LLM 1.3.0rc26 measured FastWan2.2 TI2V 5B, BF16, 704×1280×121 frames, three DMD steps, guidance 1, seed 42 and three prompts. All 36 timed draws from six repetitions per arm/prompt were retained. Equal-prompt mean of medians: 4.284661 s native → 3.471258 s candidate, an 18.98% warmed generation latency reduction. The conservative comparison using candidate maxima and native minima reduced latency by 18.56%; every prompt exceeded 18.47%. Worst-prompt LPIPS over all 121 frames was 0.01636 against a predeclared 0.25 limit. Separate profiles showed 899 selected fused calls and 31 unchanged attention normalization calls.

Those measurements cover the prototype's actual executor generation interval. Startup, per-request prototype installation/restoration, encoding, report I/O and scoring were outside that interval. They do not validate a new permanent integration on current main. Operator tests passed their declared absolute-error contract, but approximate square-root/exponential lowering and native instruction equivalence still need numerical-policy review.

Alternatives and related work

Retain eager normalization plus SiLU, or investigate another kernel after measuring the same workload. This proposal credits the existing SGLang implementation; it makes no new-kernel novelty claim. Current-main FP4 changes in #18931 overlap the source file and must be preserved. #17844 concerns a separate Cosmos3 VAE path. Gains from these changes are not additive claims.

Draft readiness gates

• Decoder-only dispatch, native fallbacks and checkpoint/buffer lifecycle tests.
• GPU operator numerical coverage, including supported layouts and clamp-edge inputs.
• Current-main integration, launch engagement, full-generation timing and LPIPS validation.
• Maintainer review of this proposal and the numerical acceptance policy before the draft is marked ready.

The draft PR is intended for implementation discussion while these gates remain open.