Repository navigation
Add new_thread_unsafe_stream API - #3578
Merged
Merged
Conversation
zcbenz
force-pushed
the
thread-unsafe-stream
branch
from
May 22, 2026 01:55
188782d to
ddd7b95
Compare
zcbenz
force-pushed
the
thread-unsafe-stream
branch
from
June 17, 2026 07:32
ddd7b95 to
0f1191e
Compare
tpegolotti
pushed a commit
that referenced
this pull request
Jun 22, 2026
3 of 4 tasks
This was referenced Aug 8, 2026
andresy
pushed a commit
to CharlieTLe/mlx-c
that referenced
this pull request
Aug 26, 2026
Streams are thread affine as of mlx 0.31.2: a stream's GPU command encoder is registered in a thread_local map, so evaluating on a stream from a thread other than the one that created it raises "There is no Stream(gpu, N) in current thread." mlx added `new_thread_unsafe_stream` for language bindings that do their own locking (ml-explore/mlx#3578), but mlx-c does not expose it, so those bindings have no way to reach it. mlx-swift currently carries a local C shim for this. Requires mlx 0.32.0, so the pin moves from v0.31.2. The existing bindings build and the examples pass unchanged against it.
andresy
pushed a commit
to CharlieTLe/mlx-c
that referenced
this pull request
Aug 27, 2026
Streams are thread affine as of mlx 0.31.2: a stream's GPU command encoder is registered in a thread_local map, so evaluating on a stream from a thread other than the one that created it raises "There is no Stream(gpu, N) in current thread." mlx added `new_thread_unsafe_stream` for language bindings that do their own locking (ml-explore/mlx#3578), but mlx-c does not expose it, so those bindings have no way to reach it. mlx-swift currently carries a local C shim for this. Requires mlx 0.32.0, so the pin moves from v0.31.2. The existing bindings build and the examples pass unchanged against it.
andresy
pushed a commit
to CharlieTLe/mlx-c
that referenced
this pull request
Aug 27, 2026
Streams are thread affine as of mlx 0.31.2: a stream's GPU command encoder is registered in a thread_local map, so evaluating on a stream from a thread other than the one that created it raises "There is no Stream(gpu, N) in current thread." mlx added `new_thread_unsafe_stream` for language bindings that do their own locking (ml-explore/mlx#3578), but mlx-c does not expose it, so those bindings have no way to reach it. mlx-swift currently carries a local C shim for this. Requires mlx 0.32.0, so the pin moves from v0.31.2. The existing bindings build and the examples pass unchanged against it.
andresy
pushed a commit
to CharlieTLe/mlx-c
that referenced
this pull request
Aug 27, 2026
Streams are thread affine as of mlx 0.31.2: a stream's GPU command encoder is registered in a thread_local map, so evaluating on a stream from a thread other than the one that created it raises "There is no Stream(gpu, N) in current thread." mlx added `new_thread_unsafe_stream` for language bindings that do their own locking (ml-explore/mlx#3578), but mlx-c does not expose it, so those bindings have no way to reach it. mlx-swift currently carries a local C shim for this. Requires mlx 0.32.0, so the pin moves from v0.31.2. The existing bindings build and the examples pass unchanged against it.
andresy
pushed a commit
to ml-explore/mlx-c
that referenced
this pull request
Aug 27, 2026
Streams are thread affine as of mlx 0.31.2: a stream's GPU command encoder is registered in a thread_local map, so evaluating on a stream from a thread other than the one that created it raises "There is no Stream(gpu, N) in current thread." mlx added `new_thread_unsafe_stream` for language bindings that do their own locking (ml-explore/mlx#3578), but mlx-c does not expose it, so those bindings have no way to reach it. mlx-swift currently carries a local C shim for this. Requires mlx 0.32.0, so the pin moves from v0.31.2. The existing bindings build and the examples pass unchanged against it.
Gajesh2007
pushed a commit
to Layr-Labs/mlx-c
that referenced
this pull request
Oct 23, 2026
* Support MLX v0.32.2 (ml-explore#123) * bump to MLX v0.32.0 * fix generator (parser not supporting thread_local) * overloads for mlx_astype * regenerate * fix const char* return assignement * regenerate * bump to MLX v0.32.1 * fix overloads to maintain backward-compatibility * add support for mlx_compile_cache (std::weak_ptr<mlx::core::detail::CompileCache> aka CompileCacheWeakPtr) * regenerate C files * bump to MLX v0.32.2 * regenerate * add missing mlx_compile_cache new/free functions * fix mlx_string return type (again) * add support for some missing overloads * fix flip overloads * Bind new_thread_unsafe_stream (ml-explore#122) Streams are thread affine as of mlx 0.31.2: a stream's GPU command encoder is registered in a thread_local map, so evaluating on a stream from a thread other than the one that created it raises "There is no Stream(gpu, N) in current thread." mlx added `new_thread_unsafe_stream` for language bindings that do their own locking (ml-explore/mlx#3578), but mlx-c does not expose it, so those bindings have no way to reach it. mlx-swift currently carries a local C shim for this. Requires mlx 0.32.0, so the pin moves from v0.31.2. The existing bindings build and the examples pass unchanged against it. * example-ops (ml-explore#121) * Add examples for ops added in 0.32.0 * fix examples --------- Co-authored-by: Ronan Collobert <collobert@apple.com> * fix custom io reader/writer API (ml-explore#130) * Implement missing stream functions (ml-explore#132) * prefer forward-declaration to avoid circular references * support for missing stream functions * cosmetics * version 0.7.0 (ml-explore#135) * bump to v0.32.4 (ml-explore#140) * Pin the MLX core to the Layr-Labs/mlx upstream sync merge Move the FetchContent GIT_TAG from ad5966879ec5 to 3c3750f4ed40. That commit merges upstream ml-explore/mlx main c6abf0dbb into Layr-Labs/mlx main. Its parent on Layr-Labs/mlx main has the same tree as ad5966879ec5, so no fork core change is lost. The new revision adds the core functions that the upstream bindings call, for example new_thread_local_stream and get_array_buffer_size. GIT_REPOSITORY stays on Layr-Labs/mlx. fork.yaml describes the new pin. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Pin the MLX core to the repaired Layr-Labs/mlx sync head Move the FetchContent GIT_TAG from 3c3750f4ed40 to e5b6550d5898. e5b6550d5898 is 3c3750f4ed40 plus two commits on the Layr-Labs/mlx sync branch. They repair the merge in the quantized Metal kernels (qmm_t_impl template parameters and the fp_gather_qmv_fast_tail adjust_matrix_offsets call). At 3c3750f4ed40 quantized.metal and fp_quantized.metal did not compile. The pin stays provisional until the Layr-Labs/mlx sync PR is green. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Pin the MLX core to the Layr-Labs/mlx sync PR head Move the FetchContent GIT_TAG from e5b6550d5898 to 780b57ea90cc, the head of Layr-Labs/mlx#38. 780b57ea90cc adds one commit over e5b6550d5898. It changes only tests/allocator_tests.cpp and tests/gptoss_mxfp4_tests.cpp, so the core library sources are the same. The pin stays provisional until Layr-Labs/mlx#38 CI is green. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Pin the MLX core to Layr-Labs/mlx#38 head dbf463fc93ca Move the FetchContent GIT_TAG from 780b57ea90cc to dbf463fc93ca, the new head of Layr-Labs/mlx#38. dbf463fc93ca adds one commit over 780b57ea90cc. It changes the CPU SIMD headers base_simd.h and accelerate_simd.h (guard INT_MIN / -1 in floor_divide) and fork.yaml. The pin stays provisional until Layr-Labs/mlx#38 CI is green. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Pin the MLX core to Layr-Labs/mlx#38 head 089acfe1 Layr-Labs/mlx#38 moved from dbf463fc to 089acfe1. The new commit computes the JACCL ring wire offsets in 64 bits (CodeQL review fix). It changes only mlx/distributed/jaccl/lib/jaccl/ring.cpp, ring_impl.h and fork.yaml, so no mlx-c source changes. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * Pin the MLX core to Layr-Labs/mlx#38 head 4588237c Layr-Labs/mlx#38 moved from 089acfe1 to 4588237c. The new commit computes the vector SDPA query layout check in 64 bits (CodeQL alert 29). It changes only mlx/backend/metal/scaled_dot_product_attention.cpp and fork.yaml, so no mlx-c source changes. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * Pin the MLX core to Layr-Labs/mlx#38 head 1f7e9063 Layr-Labs/mlx#38 moved from 4588237c through aaec1b63 (tests and CI only) to 1f7e9063. The code change is in mlx/ops.cpp: qqmm raises "Global scale is only supported on the GPU" on a CPU stream, because the CPU kernel has no global-scale input and returned wrong results. No mlx-c source changes. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * Pin the MLX core to Layr-Labs/mlx#38 head cb73247a Layr-Labs/mlx#38 moved from 1f7e9063 to cb73247a. The code change: every CPU BLAS gemm call with beta 0 clears its output region first, because Accelerate can read C even when beta is 0 and a reused buffer can hold NaN (latent upstream bug). No mlx-c source changes. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * Report buffer uniqueness from reference counts, not donatability mlx_array_get_buffer_info set `unique` from is_donatable(). Since upstream mlx 5dbe9991 (#4634), is_donatable() also requires that MLX owns the buffer, so a wrapped external buffer always read as not unique. The field means "no other array references this array or its data", and mlx-swift-lm relies on that for paged KV segments, checkpoint memory accounting and MTP state. Use array::is_uniquely_referenced() from Layr-Labs/mlx 699649d3 and pin the core there. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * Pin the MLX core to Layr-Labs/mlx#38 head 36f043ec Layr-Labs/mlx#38 moved from 699649d3 to 36f043ec: async_eval no longer throws an error that an earlier CPU stream task stored. It fixes an upstream ordering race between #3742 and #4338, and synchronize still raises the error. No mlx-c source changes. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * Pin the MLX core to Layr-Labs/mlx#38 head 83dbdb04 Layr-Labs/mlx#38 merged 9 more upstream ml-explore/mlx commits, through afc03c511 (2026-10-23): large-offset gather/scatter and pad fixes, a ring send/recv data race fix, mx.take negative indices, normal loc/scale shapes, padding jvp/vjp, and wider affine qmv_wide row tiles. No mlx-c source changes. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Ronan Collobert <collobert@apple.com> Co-authored-by: Charlie Le <charlie_le@apple.com> Co-authored-by: Eric Saxby <77073+sax@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add an API to let users create streams that can be freely passed to and evaluated on any thread, but as the API name implies MLX is not applying any synchronization around the streams and it is users' responsibility to ensure there is no race conditions.
This provides a way for language bindings (like Swift) to evaluate graph asynchronously via threads, on the condition that they add locks on themselves. Hopefully after making streams fully thread-safe in future and we would be able to remove this API, but it is going to be very tricky and might not come any time soon.