Repository navigation
GEMM INT8 large shapes at group-0 BOs should EINVAL, not wedge the NPU (driver) #176
Description
Severity: HIGH (system-level). Companion to the XRTHostRuntime group_id issue.
An INT8 GEMM with large buffers (2048³) using group-0 BOs against a kernel that declares 0x10000 data groups does not just return garbage — it wedges the NPU: dmesg logs repeated amdxdna ... IO_PAGE_FAULT and aie2_set_cmd_timeout, /dev/accel/accel0 stays enumerated but every kernel dispatch silently no-ops until the PCI device is removed and rescanned (or the driver reloaded).
Even if the runtime is fixed to bind correct groups, the driver should reject a submit whose BO group layout does not match the kernel ABI (EINVAL) instead of faulting and taking the whole device down. Also worth checking whether a wedged context can be reset per-context rather than requiring a full device rescan.
Activity
bong-water-water-bong commentedon Aug 31, 2026
Still open (driver-side; companion amd/xdna-driver#1645). The runtime-side group binding (174) plus the INT8 dtype fix (171) eliminate the trigger on the current stack; driver should still EINVAL mismatched submits instead of wedging the device.
hunhoffe commentedon Oct 28, 2026
Re-checked the "already eliminated" claim above against the current stack, and it only holds half true.
#171 (INT8 GEMM dtype support) is genuinely fixed — closed, bit-exact verified on Strix Halo. But #174 (runtime-side group binding) is not fixed: re-audited it directly, XRTTensor still defaults every host BO to group_id=0 with no kernel-group matching, and the actual fix is tracked upstream at mlir-aie#3653 (open, labeled bug/help wanted/host-runtime, no PR attached yet). So the trigger for this wedge is still live.
Also checked the companion driver issue, amd/xdna-driver#1645 (open): the maintainers declined to add BO-group validation at the driver level, since the driver treats command contents as untrusted and relies on firmware isolation — they place validation responsibility on userspace (XRT/mlir-aie), not the driver. So the "driver should EINVAL instead of wedge" ask here isn't likely to land at that layer; mlir-aie#3653 is where this actually gets closed out.
Keeping open, now cross-linked to both: mlir-aie#3653 (the real fix) and xdna-driver#1645 (confirms the driver-side ask won't be picked up there).