Repository navigation
build(deps): move to flashlib 0.4.0 and drop the numpy<2.5 cap - #669
Merged
Merged
Conversation
trcwebdesign
pushed a commit
to trcwebdesign/FreeToken
that referenced
this pull request
Oct 14, 2026
pretor
pushed a commit
to pretor/FreeToken
that referenced
this pull request
Oct 14, 2026
KarrAcaRn
pushed a commit
to KarrAcaRn/FreeToken-ByAI
that referenced
this pull request
Oct 14, 2026
Brings in FlashML-org#669 (flashlib 0.4.0), FlashML-org#670 (cpu-moe executor split into weight formats and gpu sync backends) and FlashML-org#685 (hybrid route split in the ensure kernel). - cpu_moe_ext.cpp: upstream's split version; next's ROCm memop graph probe is superseded by upstream's gpu_sync_hip.cpp - cpu_executor.py: upstream structure (shared IO block, weight_formats, host_func_capture_safe) plus next's auto_pool_cores core reservation - offload_kernels.py: upstream's split ensure kernel (CPU reference dropped) plus next's disk-tier materialize_count - engine.py: max_tokens covers cuda_graph_bs and DFlash verify rows - benchbw.py: upstream's run() API with next's decode-shaped gathers - setup.py: upstream's split sources and nvcc gencodes, kept next's ROCm C++ std pick and Windows /O2 + no cpu_moe on Windows Assisted-by: Claude Opus 5.5
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Move to flashlib 0.4.0. It no longer depends on
numbaornvidia-cutlass-dsl: its CuTe DSL backends moved to the optionalflashlib[cutedsl]extra, and its HDBSCAN tree helpers no longer use numba. numba is what capped numpy below 2.5, so the cap goes.pyproject.toml:flashlib==0.4.0,numpy>=2.0,<3.flashlib.kernels.slot_cache(lru_ensure,N_STATS,Stat). That package is unchanged between flashlib v0.3.0 and v0.4.0.nvidia-cutlass-dslis still installed with[accel], because flashinfer depends on it.uv pip compile(Linux x86_64, Python 3.12, with thefiandsglextras) compared withmain: flashlib 0.3.0 -> 0.4.0 and numpy 2.4.6 -> 2.5.4, and numba 0.68.0 and llvmlite 0.50.0 are no longer installed. Nothing else changes. In therocm/pytorch:rocm10.1.0_ubuntu24.04_py3.12_pytorch_release_2.14.0image, the dependency list no longer installs numba,nvidia-cutlass-dslorcuda-bindingsand no longer downgrades the image's numpy 2.5.3.Tested on H100 80GB with
main95ac86f plus this change, torch 2.14.1+cu130 and Triton 3.8.0:pytest -m "not slow"withmain's resolution and with this branch's: the same result for each of the 2309 tests (2090 passed, 218 skipped, 1 failed). The failure is the localapodex/Apodex-1.1-mini-NVFP4checkpoint stub, which has an index but no weights.tests/e2e/test_aime.py(AIME25 problem 1) with Qwen3.6-35B-A3B-FP8 on the offload MoE backend andFREETOKEN_TEST_MOE_CACHE_SIZE=8192: passed. The model has 10240 experts, so the cache evicts.