Repository navigation
[BugFix][ONNX] Fix Round op to use ties-to-even - #19367
Conversation
The ONNX Round operator specification requires ties-to-even rounding: "For cases where number is exactly halfway between two integers, it rounds to the nearest even integer." Previously, `topi.round()` lowered to `te.round` -> `tir.round` -> `llvm::round`, which uses ties-away-from-zero. This caused TVM to return wrong results for midpoint values (e.g., round(0.5) = 1 instead of 0, round(2.5) = 3 instead of 2). Fix by switching `topi.round()` to `te.nearbyint`, which lowers to `tir.nearbyint` -> `llvm::nearbyint`. The `nearbyint` intrinsic respects the IEEE 754 default rounding mode (ties-to-even), matching the ONNX spec and onnxruntime behavior. Also register `tir.nearbyint` for the WebGPU backend, mapping to WGSL `round()` which is already ties-to-even per the WGSL spec. Add a targeted test with midpoint inputs to prevent regression. Fixes apache#18590
There was a problem hiding this comment.
Code Review
This pull request updates the round operation in TVM TOPI to use ties-to-even (banker's rounding) by switching the underlying implementation to te.nearbyint, ensuring alignment with ONNX and IEEE 754 standards. It also adds the necessary intrinsic registration for WebGPU and a new test case to verify rounding behavior. Feedback indicates that the existing tirx.round registration on WebGPU is now inconsistent because it also maps to the ties-to-even round function, whereas it typically implies ties-away-from-zero; a follow-up is suggested to address this discrepancy.
|
As noted in the comment above, the same semantics is intentional. The actual pre-existing inconsistency runs in the opposite direction: LLVM/ROCm/Hexagon backends lower Worth a follow-up issue to align all backends — |
|
@swjng Thanks for the contribution! Could you send a follow-up PR to fix the problem you said? |
|
@tlopex Yes, will make a follow-up |
tir.round constant-folds using std::nearbyint (ties-to-even), but backends lowered it to platform round() (ties-away-from-zero). This inconsistency meant compiled code could produce different results from constant-folded code for midpoint values like 0.5, 2.5, etc. Fix all backends to use ties-to-even intrinsics: - LLVM/ROCm/Hexagon: llvm::Intrinsic::round -> nearbyint - NVPTX: __nv_round -> __nv_nearbyint - CUDA: round/roundf -> nearbyint/nearbyintf (f16/bf16 already used hrint) - Metal/OpenCL: round -> rint - Vulkan/SPIR-V: GLSLstd450Round -> GLSLstd450RoundEven - OpenCL codegen: fix nearbyint mapping from round() to rint() Also update op.h documentation to explicitly state ties-to-even semantics and add test_round_ties_to_even regression test. Follow-up to apache#19367. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
tir.round constant-folds using std::nearbyint (ties-to-even), but backends lowered it to platform round() (ties-away-from-zero). This inconsistency meant compiled code could produce different results from constant-folded code for midpoint values like 0.5, 2.5, etc. Fix all backends to use ties-to-even intrinsics: - LLVM/ROCm/Hexagon: llvm::Intrinsic::round -> nearbyint - NVPTX: __nv_round -> __nv_nearbyint - CUDA: round/roundf -> nearbyint/nearbyintf (f16/bf16 already used hrint) - Metal/OpenCL: round -> rint - Vulkan/SPIR-V: GLSLstd450Round -> GLSLstd450RoundEven - OpenCL codegen: fix nearbyint mapping from round() to rint() Also update op.h documentation to explicitly state ties-to-even semantics and add test_round_ties_to_even regression test. Follow-up to apache#19367. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
## Problem `tir.round` constant-folds using `std::nearbyint` (IEEE 754 ties-to-even), but all backends lower it to platform `round()` which uses ties-away-from-zero. This means compiled code can produce different results from constant-folded code for midpoint values: | Input | Constant-fold (ties-to-even) | Compiled (ties-away) | |-------|-----|------| | 0.5 | 0.0 | 1.0 | | 2.5 | 2.0 | 3.0 | | -0.5 | 0.0 | -1.0 | This was identified as a follow-up to #19367 — see [this comment](#19367 (comment)). ## Fix Align all backends to use ties-to-even intrinsics, matching the constant-folding behavior: | Backend | Before | After | |---------|--------|-------| | LLVM/ROCm/Hexagon | `llvm::Intrinsic::round` | `llvm::Intrinsic::nearbyint` | | NVPTX | `__nv_round[f]` | `__nv_nearbyint[f]` | | CUDA | `round`/`roundf` | `nearbyint`/`nearbyintf` (f16/bf16 already used `hrint`) | | Metal/OpenCL | `round` | `rint` | | Vulkan/SPIR-V | `GLSLstd450Round` | `GLSLstd450RoundEven` | Also fixes OpenCL codegen where `tir.nearbyint` was incorrectly mapped to OpenCL `round()` instead of `rint()`. Updates `op.h` documentation to explicitly state ties-to-even semantics for both `round()` and `nearbyint()`. ## Testing ``` python -m pytest tests/python/tirx-base/test_tir_intrin.py -xvs ``` New `test_round_ties_to_even` verifies midpoint inputs `[0.5, 1.5, 2.5, 3.5, -0.5, -1.5, -2.5, -3.5]` produce ties-to-even results on the LLVM backend. All 12 tests pass (10 passed, 2 skipped for CUDA). --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…ported_program and fix negative-decimals rounding torch.export lowers torch.round(x, decimals) (any explicit decimals, including decimals=0) to aten.round.decimals, but the exported-program convert map only registered round.default. Any explicit decimals made from_exported_program fail with "Unsupported function types ['round.decimals']". Additionally, BaseFXGraphImporter._round scaled every non-zero decimals by round(x * 10**decimals) / 10**decimals. For negative decimals this multiplies by 0.1 / 0.01 / ..., which is inexact in floating point: in float64, torch.round(torch.tensor(25.0), decimals=-1) computed 25 * 0.1 == 2.5000000000000004 and rounded up to 30 instead of 20. Register "round.decimals" in ExportedProgramImporter.create_convert_map and branch the decimals != 0 scale in _round to use an exact integer power of 10: multiply for positive decimals, divide for negative ones (round(x / 10**|d|) * 10**|d|). The ties-to-even inner rounding is already provided by upstream apache#19367 / apache#19368 (tir.round -> nearbyint across backends) and is not changed here. Validated by the verify_patch.py differential harness on the locked build: Part B (fix + ties-to-even inner round, = latest semantics) matches PyTorch for all 28 combinations (from_exported_program / from_fx x decimals {0,1,2,3,-1,-2,-3} x float32/float64), including the previously-failing round(25, -1) == 20 and round(2.25, 1) == 2.2. Co-Authored-By: Claude <noreply@anthropic.com>
…ported_program and fix negative-decimals rounding torch.export lowers torch.round(x, decimals) (any explicit decimals, including decimals=0) to aten.round.decimals, but the exported-program convert map only registered round.default. Any explicit decimals made from_exported_program fail with "Unsupported function types ['round.decimals']". Additionally, BaseFXGraphImporter._round scaled every non-zero decimals by round(x * 10**decimals) / 10**decimals. For negative decimals this multiplies by 0.1 / 0.01 / ..., which is inexact in floating point: in float64, torch.round(torch.tensor(25.0), decimals=-1) computed 25 * 0.1 == 2.5000000000000004 and rounded up to 30 instead of 20. Register "round.decimals" in ExportedProgramImporter.create_convert_map and branch the decimals != 0 scale in _round to use an exact integer power of 10: multiply for positive decimals, divide for negative ones (round(x / 10**|d|) * 10**|d|). The ties-to-even inner rounding is already provided by upstream apache#19367 / apache#19368 (tir.round -> nearbyint across backends) and is not changed here. Validated by the verify_patch.py differential harness on the locked build: Part B (fix + ties-to-even inner round, = latest semantics) matches PyTorch for all 28 combinations (from_exported_program / from_fx x decimals {0,1,2,3,-1,-2,-3} x float32/float64), including the previously-failing round(25, -1) == 20 and round(2.25, 1) == 2.2.
…ported_program and fix negative-decimals rounding torch.export lowers torch.round(x, decimals) (any explicit decimals, including decimals=0) to aten.round.decimals, but the exported-program convert map only registered round.default. Any explicit decimals made from_exported_program fail with "Unsupported function types ['round.decimals']". Additionally, BaseFXGraphImporter._round scaled every non-zero decimals by round(x * 10**decimals) / 10**decimals. For negative decimals this multiplies by 0.1 / 0.01 / ..., which is inexact in floating point: in float64, torch.round(torch.tensor(25.0), decimals=-1) computed 25 * 0.1 == 2.5000000000000004 and rounded up to 30 instead of 20. Register "round.decimals" in ExportedProgramImporter.create_convert_map and branch the decimals != 0 scale in _round to use an exact integer power of 10: multiply for positive decimals, divide for negative ones (round(x / 10**|d|) * 10**|d|). The ties-to-even inner rounding is already provided by upstream apache#19367 / apache#19368 (tir.round -> nearbyint across backends) and is not changed here. Validated by the verify_patch.py differential harness on the locked build: Part B (fix + ties-to-even inner round, = latest semantics) matches PyTorch for all 28 combinations (from_exported_program / from_fx x decimals {0,1,2,3,-1,-2,-3} x float32/float64), including the previously-failing round(25, -1) == 20 and round(2.25, 1) == 2.2.
…m_exported_program` and negative-decimals rounding (#20239) Fixes: #20231 ## Summary `torch.export` lowers `torch.round(x, decimals)` (any explicit `decimals`, including `decimals=0`) to `aten.round.decimals`, while plain `torch.round(x)` lowers to `aten.round.default`. The Relax Torch frontend registered only `round.default` in the exported-program convert map, so **any** explicit `decimals` made `from_exported_program` fail outright with `"AssertionError: Unsupported function types ['round.decimals']"`. In addition, the `decimals != 0` path in `BaseFXGraphImporter._round` always scaled by `round(x * 10**decimals) / 10**decimals`. For **negative** `decimals` this multiplies by `0.1 / 0.01 / ...`, which loses the exact power-of-10 scale and, in float64, breaks e.g. `torch.round(torch.tensor(25.0, dtype=float64), decimals=-1)` (`25 * 0.1 == 2.5000000000000004` rounds up to `30` instead of the correct `20`). This PR registers `round.decimals` in the exported-program convert map and makes the `decimals != 0` scaling use an exact integer power of 10: multiply for positive decimals, **divide** for negative ones. > Note: the round-half-to-even (ties-to-even) semantics themselves are already > provided on latest by upstream #19367 / #19368 (`tir.round` → `nearbyint` > across all backends); they are **not** changed by this PR. The fixes here are > the `round.decimals` dispatch gap and the negative-decimals scale precision. ## Root cause 1. **`from_exported_program` rejects `torch.round(x, decimals)`.** In `exported_program_translator.py`, `ExportedProgramImporter.create_convert_map` maps `"round.default": self._round` but no `round.decimals` entry. Since `torch.export` always emits `aten.round.decimals` when `decimals` is passed explicitly — even `decimals=0` — every such call hits the `"Unsupported function types ['round.decimals']"` assert in dispatch. 2. **Negative decimals round incorrectly.** `_round` computed `scale = relax.const(10**decimals, dtype)` and emitted `divide(round(multiply(arg, scale)), scale)` for every non-zero `decimals`. For `decimals = -1` the scale is `0.1`; multiplying by a non-integer power of 10 is inexact in floating point, so `torch.round(torch.tensor([25.0], dtype=torch.float64), decimals=-1)` produced `30` instead of `20`. (`from_fx` shares the same `_round`.) ## Fix - `exported_program_translator.py` — add `"round.decimals": self._round,` right after `"round.default": self._round,` in `ExportedProgramImporter.create_convert_map`, so any explicit-`decimals` `torch.round` converts through the existing `_round`. - `base_fx_graph_translator.py` — in `BaseFXGraphImporter._round`, keep the `decimals == 0` fast path, and branch the `decimals != 0` scale: - `decimals > 0`: `divide(round(multiply(arg, 10**d)), 10**d)` (unchanged). - `decimals < 0`: `multiply(round(divide(arg, 10**-d)), 10**-d)` — divide by the exact integer power of 10 and multiply back, avoiding the inexact `× 0.1` path. ## Validation ### In-tree regression tests (added) - `test_round_decimals` in `tests/python/relax/test_frontend_from_exported_program.py` — runs `verify_model_numerically` (Relax vs PyTorch) for `decimals in (0, 1, -1, -2)` over a value set that exercises ties-to-even half values (`0.5, 1.5, 2.5, 4.5, -0.5, -2.5`) and the negative-decimals path (`25.0, 125.0, 165.0` → `20, 120, 160` at `decimals=-1`, and `2.25 → 2.2`). Before the fix, `decimals=0` alone fails to import with `"Unsupported function types ['round.decimals']"`. - `test_round_decimals` in `tests/python/relax/test_frontend_from_fx.py` — same values through `from_fx`, asserting TVM output matches `torch.round` for the same decimals set (this path already dispatched to `_round`, but produced the wrong negative-decimals result before the fix). ### Differential test `verify_patch.py` was run on the locked pre-#19368 build (rounds half values away from zero). The real fix code is monkey-patched in; the ties-to-even inner round is reproduced with `te.nearbyint` to stand in for latest `relax.op.round` semantics (#19368): | stage | export + fx × decimals {0,1,2,3,-1,-2,-3} | matched | rejected | diff elements | |-------|--------------------------------------------|---------|----------|---------------| | Part 0 — before fix | `export(decimals=-1)` | import fails: `AssertionError: Unsupported function types ['round.decimals']` | | | `fx(decimals=-1)` | `[30,130,170]` vs torch `[20,120,160]` | | Part A — fix, ties-away inner round (locked build) | 14 | 0 | 22 (all half-value ties — the #19368 ties-to-even gap, unrelated to this PR) | | Part B — fix + ties-to-even inner round (= latest) | 28 | 0 | 0 | Part B matches PyTorch for **all** 28 combinations — both frontends (`from_exported_program`, `from_fx`) × 7 `decimals` × `float32`/`float64` — including the previously-failing `round(25, -1) == 20` and `round(2.25, 1) == 2.2` cases. Run: ```bash export PATH=/home/shenqingchao/miniconda3/envs/tvm23/bin:$PATH export PYTHONPATH=/tmp/tvmffi019:/data/shenqingchao/enwei/familyfuzz/tvm/python export TVM_LIBRARY_PATH=/data/shenqingchao/enwei/familyfuzz/tvm/build python results/TVM/deepseek-v4-flash/prove_hum/torch_round/verify_patch.py ``` ## Files changed - `python/tvm/relax/frontend/torch/base_fx_graph_translator.py` — `_round`: negative `decimals` now divide by the exact integer power of 10 (`round(x / 10^|d|) * 10^|d|`) instead of multiplying by `10**decimals` (`× 0.1`). - `python/tvm/relax/frontend/torch/exported_program_translator.py` — register `round.decimals` in the exported-program convert map. - `tests/python/relax/test_frontend_from_exported_program.py` — add `test_round_decimals`. - `tests/python/relax/test_frontend_from_fx.py` — add `test_round_decimals`.
## Problem The ONNX `Round` operator specification requires **ties-to-even** (banker's) rounding: > "For cases where number is exactly halfway between two integers, it rounds to the nearest even integer." > — https://onnx.ai/onnx/operators/onnx__Round.html However, the current TVM implementation produces **ties-away-from-zero** results on midpoint values: | Input | Expected (ties-to-even) | Actual (ties-away) | |-------|------------------------|--------------------| | 0.5 | 0.0 | 1.0 | | 1.5 | 2.0 | 2.0 | | 2.5 | 2.0 | 3.0 | | -0.5 | 0.0 | -1.0 | | -2.5 | -2.0 | -3.0 | This was reported in issue #18590. ## Root Cause The lowering chain for `relax.op.round`: ``` relax.op.round -> (LegalizeOps) -> topi.round() -> te.round -> tir.round -> llvm::round ``` `llvm::round` is defined as ties-away-from-zero (C99 `round()`), while `llvm::nearbyint` uses the IEEE 754 default rounding mode (ties-to-even). ## Fix **`python/tvm/topi/math.py`**: Switch `topi.round()` from `te.round` to `te.nearbyint`. This lowers to `tir.nearbyint` -> `llvm::nearbyint`, which respects IEEE 754 ties-to-even. **`src/target/source/intrin_rule_webgpu.cc`**: Register `tir.nearbyint` for the WebGPU backend. WGSL `round()` is already ties-to-even per the WGSL spec, so `tir.nearbyint` -> `round` is the correct mapping. **`tests/python/relax/test_frontend_onnx.py`**: Add `test_round_ties_to_even()` with explicit midpoint inputs to prevent regression. ## Testing ``` python -m pytest tests/python/relax/test_frontend_onnx.py::test_round_ties_to_even -xvs python -m pytest "tests/python/relax/test_frontend_onnx.py::test_unary[Round]" -xvs ``` Both pass. The new test compares TVM output against onnxruntime (which correctly implements ties-to-even) for inputs `[0.5, 1.5, 2.5, -0.5, -1.5, -2.5]`. Fixes #18590
## Problem `tir.round` constant-folds using `std::nearbyint` (IEEE 754 ties-to-even), but all backends lower it to platform `round()` which uses ties-away-from-zero. This means compiled code can produce different results from constant-folded code for midpoint values: | Input | Constant-fold (ties-to-even) | Compiled (ties-away) | |-------|-----|------| | 0.5 | 0.0 | 1.0 | | 2.5 | 2.0 | 3.0 | | -0.5 | 0.0 | -1.0 | This was identified as a follow-up to #19367 — see [this comment](#19367 (comment)). ## Fix Align all backends to use ties-to-even intrinsics, matching the constant-folding behavior: | Backend | Before | After | |---------|--------|-------| | LLVM/ROCm/Hexagon | `llvm::Intrinsic::round` | `llvm::Intrinsic::nearbyint` | | NVPTX | `__nv_round[f]` | `__nv_nearbyint[f]` | | CUDA | `round`/`roundf` | `nearbyint`/`nearbyintf` (f16/bf16 already used `hrint`) | | Metal/OpenCL | `round` | `rint` | | Vulkan/SPIR-V | `GLSLstd450Round` | `GLSLstd450RoundEven` | Also fixes OpenCL codegen where `tir.nearbyint` was incorrectly mapped to OpenCL `round()` instead of `rint()`. Updates `op.h` documentation to explicitly state ties-to-even semantics for both `round()` and `nearbyint()`. ## Testing ``` python -m pytest tests/python/tirx-base/test_tir_intrin.py -xvs ``` New `test_round_ties_to_even` verifies midpoint inputs `[0.5, 1.5, 2.5, 3.5, -0.5, -1.5, -2.5, -3.5]` produce ties-to-even results on the LLVM backend. All 12 tests pass (10 passed, 2 skipped for CUDA). --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
## Problem The ONNX `Round` operator specification requires **ties-to-even** (banker's) rounding: > "For cases where number is exactly halfway between two integers, it rounds to the nearest even integer." > — https://onnx.ai/onnx/operators/onnx__Round.html However, the current TVM implementation produces **ties-away-from-zero** results on midpoint values: | Input | Expected (ties-to-even) | Actual (ties-away) | |-------|------------------------|--------------------| | 0.5 | 0.0 | 1.0 | | 1.5 | 2.0 | 2.0 | | 2.5 | 2.0 | 3.0 | | -0.5 | 0.0 | -1.0 | | -2.5 | -2.0 | -3.0 | This was reported in issue #18590. ## Root Cause The lowering chain for `relax.op.round`: ``` relax.op.round -> (LegalizeOps) -> topi.round() -> te.round -> tir.round -> llvm::round ``` `llvm::round` is defined as ties-away-from-zero (C99 `round()`), while `llvm::nearbyint` uses the IEEE 754 default rounding mode (ties-to-even). ## Fix **`python/tvm/topi/math.py`**: Switch `topi.round()` from `te.round` to `te.nearbyint`. This lowers to `tir.nearbyint` -> `llvm::nearbyint`, which respects IEEE 754 ties-to-even. **`src/target/source/intrin_rule_webgpu.cc`**: Register `tir.nearbyint` for the WebGPU backend. WGSL `round()` is already ties-to-even per the WGSL spec, so `tir.nearbyint` -> `round` is the correct mapping. **`tests/python/relax/test_frontend_onnx.py`**: Add `test_round_ties_to_even()` with explicit midpoint inputs to prevent regression. ## Testing ``` python -m pytest tests/python/relax/test_frontend_onnx.py::test_round_ties_to_even -xvs python -m pytest "tests/python/relax/test_frontend_onnx.py::test_unary[Round]" -xvs ``` Both pass. The new test compares TVM output against onnxruntime (which correctly implements ties-to-even) for inputs `[0.5, 1.5, 2.5, -0.5, -1.5, -2.5]`. Fixes #18590
## Problem `tir.round` constant-folds using `std::nearbyint` (IEEE 754 ties-to-even), but all backends lower it to platform `round()` which uses ties-away-from-zero. This means compiled code can produce different results from constant-folded code for midpoint values: | Input | Constant-fold (ties-to-even) | Compiled (ties-away) | |-------|-----|------| | 0.5 | 0.0 | 1.0 | | 2.5 | 2.0 | 3.0 | | -0.5 | 0.0 | -1.0 | This was identified as a follow-up to #19367 — see [this comment](#19367 (comment)). ## Fix Align all backends to use ties-to-even intrinsics, matching the constant-folding behavior: | Backend | Before | After | |---------|--------|-------| | LLVM/ROCm/Hexagon | `llvm::Intrinsic::round` | `llvm::Intrinsic::nearbyint` | | NVPTX | `__nv_round[f]` | `__nv_nearbyint[f]` | | CUDA | `round`/`roundf` | `nearbyint`/`nearbyintf` (f16/bf16 already used `hrint`) | | Metal/OpenCL | `round` | `rint` | | Vulkan/SPIR-V | `GLSLstd450Round` | `GLSLstd450RoundEven` | Also fixes OpenCL codegen where `tir.nearbyint` was incorrectly mapped to OpenCL `round()` instead of `rint()`. Updates `op.h` documentation to explicitly state ties-to-even semantics for both `round()` and `nearbyint()`. ## Testing ``` python -m pytest tests/python/tirx-base/test_tir_intrin.py -xvs ``` New `test_round_ties_to_even` verifies midpoint inputs `[0.5, 1.5, 2.5, 3.5, -0.5, -1.5, -2.5, -3.5]` produce ties-to-even results on the LLVM backend. All 12 tests pass (10 passed, 2 skipped for CUDA). --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Problem
The ONNX
Roundoperator specification requires ties-to-even (banker's) rounding:However, the current TVM implementation produces ties-away-from-zero results on midpoint values:
This was reported in issue #18590.
Root Cause
The lowering chain for
relax.op.round:llvm::roundis defined as ties-away-from-zero (C99round()), whilellvm::nearbyintuses the IEEE 754 default rounding mode (ties-to-even).Fix
python/tvm/topi/math.py: Switchtopi.round()fromte.roundtote.nearbyint. This lowers totir.nearbyint->llvm::nearbyint, which respects IEEE 754 ties-to-even.src/target/source/intrin_rule_webgpu.cc: Registertir.nearbyintfor the WebGPU backend. WGSLround()is already ties-to-even per the WGSL spec, sotir.nearbyint->roundis the correct mapping.tests/python/relax/test_frontend_onnx.py: Addtest_round_ties_to_even()with explicit midpoint inputs to prevent regression.Testing
Both pass. The new test compares TVM output against onnxruntime (which correctly implements ties-to-even) for inputs
[0.5, 1.5, 2.5, -0.5, -1.5, -2.5].Fixes #18590