finegrained-fp8: MXFP4 recipes by IlyasMoutawwakil · Pull Request #923 · huggingface/kernels-community

IlyasMoutawwakil · 2026-06-04T08:23:39Z

Related issue

smae as #867

What does this PR do?

Adds support for DSv4-like fp4 experts

Motivation

DSv4 support across multiple accelerators

Testing

Extended the testing suite

Checklist

This PR is linked to an issue that was discussed and approved
I have tested these changes locally
New/changed functionality has test coverage
LLM disclosure:
- I did not use an LLM to create this PR.
- I used and LLM for assistance while creating this PR.
- This PR was mostly or completely generated by an LLM.

Signed-off-by: Wang, Yi <yi.a.wang@intel.com>

IlyasMoutawwakil · 2026-06-04T10:32:23Z

    "xpu",
 ]
-version = 1
+version = 2


danieldk · 2026-06-04T13:14:19Z

/kernel-bot build finegrained-fp8

github-actions · 2026-06-04T13:14:40Z

Build request processed.

Command: /kernel-bot build finegrained-fp8
Mode: build only
Target branch: pr-923
PR head SHA: 953cd77e7e5cd4be9573cd01716f58e15a069ea6
Workflows: build.yaml, build-mac.yaml, build-windows.yaml

Dispatched (1):

finegrained-fp8 (build.yaml): https://github.com/huggingface/kernels-community/actions/runs/26954040210

danieldk · 2026-06-04T14:31:00Z

/kernel-bot build finegrained-fp8

github-actions · 2026-06-04T14:31:18Z

Build request processed.

Command: /kernel-bot build finegrained-fp8
Mode: build only
Target branch: pr-923
PR head SHA: eb0f3ccf3e32c3ce4277773bca4c912ba3d962b3
Workflows: build.yaml, build-mac.yaml, build-windows.yaml

Dispatched (1):

finegrained-fp8 (build.yaml): https://github.com/huggingface/kernels-community/actions/runs/26958381576

danieldk · 2026-06-04T14:37:53Z

/kernel-bot build finegrained-fp8

github-actions · 2026-06-04T14:38:09Z

Build request processed.

Command: /kernel-bot build finegrained-fp8
Mode: build only
Target branch: pr-923
PR head SHA: ce8efbc720d2b2171549c80991cdb2606fe0bb15
Workflows: build.yaml, build-mac.yaml, build-windows.yaml

Dispatched (1):

finegrained-fp8 (build.yaml): https://github.com/huggingface/kernels-community/actions/runs/26958776598

This breaks tests in kernel-builder devshells

danieldk · 2026-06-04T15:13:13Z

/kernel-bot build finegrained-fp8

github-actions · 2026-06-04T15:13:34Z

Build request processed.

Command: /kernel-bot build finegrained-fp8
Mode: build only
Target branch: pr-923
PR head SHA: 5a401cf694946794a55d4833b43f78c38d33c851
Workflows: build.yaml, build-mac.yaml, build-windows.yaml

Dispatched (1):

finegrained-fp8 (build.yaml): https://github.com/huggingface/kernels-community/actions/runs/26960884594

danieldk · 2026-06-04T17:44:11Z

/kernel-bot merge-and-upload finegrained-fp8

github-actions · 2026-06-04T17:44:26Z

Build request processed.

Command: /kernel-bot merge-and-upload finegrained-fp8
Mode: merge, build and upload
Target branch: ``
PR head SHA: 5a401cf694946794a55d4833b43f78c38d33c851
Workflows: `build.yaml, build-mac.yaml, build-windows.yaml`

Merge result: PR is already merged. Continuing with build/upload.

Dispatched (1):

finegrained-fp8 (build.yaml): https://github.com/huggingface/kernels-community/actions/runs/26969155489

* add fp4 matmul kernels for deepseek v4 flash Signed-off-by: Wang, Yi <yi.a.wang@intel.com> * add missing source code.. Signed-off-by: Wang, Yi <yi.a.wang@intel.com> * update Signed-off-by: Wang, Yi <yi.a.wang@intel.com> * update to w4a8 Signed-off-by: Wang, Yi <yi.a.wang@intel.com> * update Signed-off-by: Wang, Yi <yi.a.wang@intel.com> * perf optimization add add benchmark_perf.py script Signed-off-by: Wang, Yi <yi.a.wang@intel.com> * structure * standarize all kernels and testing around the same api * better testing framework * fix batched with sentinels * tests with sentinels as well * test static and support ue8m0 * specify v2 as the version * cleanup * naming convention * update readme * remove bench * format * update readme * trigger * skip tests on < sm89 * drop sm89 testing, too error prone * Use triton-rocm-fixes branch to pass build * Update build.toml * Fix Torch ops * Remove custom path insertion This breaks tests in kernel-builder devshells --------- Signed-off-by: Wang, Yi <yi.a.wang@intel.com> Co-authored-by: Wang, Yi <yi.a.wang@intel.com> Co-authored-by: Daniël de Kok <me@danieldk.eu>

This reverts commit 9f9d5e1.

* feat: allow kernel-bot to run security analysis. * include security in the build. * megablocks for rocm (#916) * feat: update merge-and-upload permissions (#930) * [Liger Kernels] Adding TiledMLP kernels (#918) * chore: adding liger tiled mlp kernels * liger-kernels: test with Triton ROCm binary wheels * chore: align layer to pure --------- Co-authored-by: Daniël de Kok <me@danieldk.eu> * finegrained-fp8: MXFP4 recipes (#923) * add fp4 matmul kernels for deepseek v4 flash Signed-off-by: Wang, Yi <yi.a.wang@intel.com> * add missing source code.. Signed-off-by: Wang, Yi <yi.a.wang@intel.com> * update Signed-off-by: Wang, Yi <yi.a.wang@intel.com> * update to w4a8 Signed-off-by: Wang, Yi <yi.a.wang@intel.com> * update Signed-off-by: Wang, Yi <yi.a.wang@intel.com> * perf optimization add add benchmark_perf.py script Signed-off-by: Wang, Yi <yi.a.wang@intel.com> * structure * standarize all kernels and testing around the same api * better testing framework * fix batched with sentinels * tests with sentinels as well * test static and support ue8m0 * specify v2 as the version * cleanup * naming convention * update readme * remove bench * format * update readme * trigger * skip tests on < sm89 * drop sm89 testing, too error prone * Use triton-rocm-fixes branch to pass build * Update build.toml * Fix Torch ops * Remove custom path insertion This breaks tests in kernel-builder devshells --------- Signed-off-by: Wang, Yi <yi.a.wang@intel.com> Co-authored-by: Wang, Yi <yi.a.wang@intel.com> Co-authored-by: Daniël de Kok <me@danieldk.eu> * address review feedback * Revert "megablocks for rocm (#916)" This reverts commit e06068f. * Revert "feat: update merge-and-upload permissions (#930)" This reverts commit a913728. * Revert "[Liger Kernels] Adding TiledMLP kernels (#918)" This reverts commit 3d9bb2f. * Revert "finegrained-fp8: MXFP4 recipes (#923)" This reverts commit 9f9d5e1. * security-and-build --------- Signed-off-by: Wang, Yi <yi.a.wang@intel.com> Co-authored-by: BADAOUI Abdennacer <106801897+Abdennacer-Badaoui@users.noreply.github.com> Co-authored-by: drbh <david.richard.holtz@gmail.com> Co-authored-by: Aritra Roy Gosthipaty <aritra.born2fly@gmail.com> Co-authored-by: Daniël de Kok <me@danieldk.eu> Co-authored-by: Ilyas Moutawwakil <57442720+IlyasMoutawwakil@users.noreply.github.com> Co-authored-by: Wang, Yi <yi.a.wang@intel.com>

sywangyi and others added 23 commits May 18, 2026 16:11

add fp4 matmul kernels for deepseek v4 flash

99c182a

Signed-off-by: Wang, Yi <yi.a.wang@intel.com>

add missing source code..

0225c79

Signed-off-by: Wang, Yi <yi.a.wang@intel.com>

update

ccd9d65

Signed-off-by: Wang, Yi <yi.a.wang@intel.com>

update to w4a8

fa8bd0c

Signed-off-by: Wang, Yi <yi.a.wang@intel.com>

update

f6ebbb4

Signed-off-by: Wang, Yi <yi.a.wang@intel.com>

perf optimization add add benchmark_perf.py script

702b7e9

Signed-off-by: Wang, Yi <yi.a.wang@intel.com>

Merge branch 'main' into deepseek_v4_fp4

1b19639

structure

f4f7012

standarize all kernels and testing around the same api

28c34df

better testing framework

bb0fad9

fix batched with sentinels

90f7b91

tests with sentinels as well

f20c018

Merge branch 'main' into pr/sywangyi/867

48de8cd

test static and support ue8m0

366285b

specify v2 as the version

3b5c219

cleanup

8117fc8

naming convention

30ab31a

update readme

e90cbab

remove bench

c1e8d0c

format

7951038

update readme

075c69a

Merge branch 'main' into deepseek_v4_fp4

60e0f8a

trigger

a5b4cfc

IlyasMoutawwakil requested review from danieldk and drbh as code owners June 4, 2026 08:23

IlyasMoutawwakil added 2 commits June 4, 2026 02:11

skip tests on < sm89

52adb7d

drop sm89 testing, too error prone

953cd77

IlyasMoutawwakil closed this Jun 4, 2026

IlyasMoutawwakil reopened this Jun 4, 2026

IlyasMoutawwakil commented Jun 4, 2026

View reviewed changes

Comment thread finegrained-fp8/build.toml Outdated

"xpu",

]

version = 1

version = 2

IlyasMoutawwakil Jun 4, 2026

Copy link
Copy Markdown

Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

v2

Use triton-rocm-fixes branch to pass build

eb0f3cc

Update build.toml

ce8efbc

danieldk added 2 commits June 4, 2026 15:12

Fix Torch ops

5a54bce

Remove custom path insertion

5a401cf

This breaks tests in kernel-builder devshells

danieldk approved these changes Jun 4, 2026

View reviewed changes

danieldk merged commit dd0ec0d into main Jun 4, 2026
5 checks passed

sayakpaul added a commit that referenced this pull request Jun 5, 2026

Revert "finegrained-fp8: MXFP4 recipes (#923)"

46fa552

This reverts commit 9f9d5e1.

IlyasMoutawwakil mentioned this pull request Jun 8, 2026

Triton finegrained fp8/fp4 huggingface/transformers#46407

Merged

6 tasks

Navigation Menu

Repository navigation

finegrained-fp8: MXFP4 recipes - #923

finegrained-fp8: MXFP4 recipes#923
danieldk merged 29 commits into
mainfrom
fp4-matmuls

IlyasMoutawwakil commented Jun 4, 2026 •

edited

Loading

Uh oh!

IlyasMoutawwakil Jun 4, 2026

Uh oh!

danieldk commented Jun 4, 2026

Uh oh!

github-actions Bot commented Jun 4, 2026 •

edited

Loading

Uh oh!

danieldk commented Jun 4, 2026

Uh oh!

github-actions Bot commented Jun 4, 2026 •

edited

Loading

Uh oh!

danieldk commented Jun 4, 2026

Uh oh!

github-actions Bot commented Jun 4, 2026 •

edited

Loading

Uh oh!

danieldk commented Jun 4, 2026

Uh oh!

github-actions Bot commented Jun 4, 2026 •

edited

Loading

Uh oh!

Uh oh!

danieldk commented Jun 4, 2026

Uh oh!

github-actions Bot commented Jun 4, 2026 •

edited

Loading

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

IlyasMoutawwakil commented Jun 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Related issue

What does this PR do?

Motivation

Testing

Checklist

Uh oh!

IlyasMoutawwakil Jun 4, 2026

Choose a reason for hiding this comment

Uh oh!

danieldk commented Jun 4, 2026

Uh oh!

github-actions Bot commented Jun 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

danieldk commented Jun 4, 2026

Uh oh!

github-actions Bot commented Jun 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

danieldk commented Jun 4, 2026

Uh oh!

github-actions Bot commented Jun 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

danieldk commented Jun 4, 2026

Uh oh!

github-actions Bot commented Jun 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

danieldk commented Jun 4, 2026

Uh oh!

github-actions Bot commented Jun 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

IlyasMoutawwakil commented Jun 4, 2026 •

edited

Loading

github-actions Bot commented Jun 4, 2026 •

edited

Loading

github-actions Bot commented Jun 4, 2026 •

edited

Loading

github-actions Bot commented Jun 4, 2026 •

edited

Loading

github-actions Bot commented Jun 4, 2026 •

edited

Loading

github-actions Bot commented Jun 4, 2026 •

edited

Loading