[CUB] Take a block offset in `LoadDirectStripedVectorized` by miscco · Pull Request #12601 · NVIDIA/cccl

miscco · 2026-11-11T15:11:37Z

Summary

LoadDirectStripedVectorized loads the vectors block_offset items after the base pointer. The offset must be a multiple of VecSize, so the vectors keep that pointer's alignment.
The loads are unchanged: each vector is still a CubVector, and a pointer that is not aligned to it is still loaded item by item.
This is the first signature of the function, which is new in 3.6, so a later change of the vector width does not need an overload of the old signature.

Test plan

nvcc compiles a kernel that calls the new signature with LOAD_DEFAULT and LOAD_LDG
cub.cpp17.test.block_load covers a misaligned pointer and a block_offset that is a multiple of the vector size

coderabbitai · 2026-11-11T15:16:42Z

Summary

Summary by CodeRabbit

New Features
- Vectorized block loads can begin at an element offset within the input, so callers can select a tile without adjusting the input pointer themselves.
- Offsets aligned to the vector width use vectorized loading; other offsets retain the same striped arrangement through item-by-item loading. Loads support aligned and unaligned input addresses and both default and LOAD_LDG cache modifiers.

LoadDirectStripedVectorized now accepts an item offset and uses it for vectorized and item-by-item loads. BlockLoad::Load passes zero. Tests cover offset loads across input alignments and cache modifiers.

Changes: Striped vectorized load offsets

Layer / File(s)	Summary
Offset-aware load implementation `cub/cub/block/block_load.cuh`	The primitive accepts an integral `block_offset` and uses the offset pointer for vectorized loads and item-by-item fallback. `BlockLoad::Load` passes zero.
Offset load test coverage `cub/test/catch2_test_block_load.cu`	The test kernel and helper pass `block_offset` to the primitive. Tests check expected tiles for offsets 8 and three tile sizes, input alignment offsets 0 and 1, and default and `LOAD_LDG` modifiers.

Possibly related issues

NVIDIA/cccl#12419 — This change updates a striped vectorized load helper and its alignment and offset behavior, which relates to the issue’s vectorized global-load implementations.

Suggested reviewers: bernhardmgruber

Change: Feature

Merge Risk: Moderate · up to f41c3

The scalar fallback for misaligned inputs no longer applies the requested cache modifier. This can change load semantics, for example volatile loads, for existing BlockLoad users with striped vectorized loading. Restore the modifier in the fallback before merging.

Architecture risk: Low · up to f41c3

The change affects 1 system.

Changed systems: cub

Architecture concerns
No architecture-level concerns identified.

Review details

Systems and components

observed — cub (service) was modified; 2 changed files map to changed impact.

Before / after behavior

observed — Modified behavior in cub/test/catch2_test_block_load.cu: The kernel now accepts block_offset and forwards it to LoadDirectStripedVectorized; previously it passed no offset.
observed — Modified behavior in cub/test/catch2_test_block_load.cu: The test helper adds an optional block_offset parameter defaulting to zero and passes it through to the kernel launch.
observed — Modified behavior in cub/test/catch2_test_block_load.cu: Adds a test for loading a tile beginning at block_offset elements after the input. It generates alignment offsets 0 or 1 and block offsets 8 or three tile sizes, then checks the expected tile for default and LOAD_LDG modifiers.
observed — Modified behavior in cub/cub/block/block_load.cuh: The documentation now specifies that striped vectors are loaded from block_offset items after block_src_ptr.

- [ ] Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

The function is new in 3.6, so its first signature loads the vectors `block_offset` items after the base pointer. The offset is a multiple of the vector size, which keeps the vectors aligned with that pointer, and later changes can select the vector width without adding an overload.

miscco · 2026-11-11T16:23:53Z

  for (int i = 0; i < ItemsPerThread / vec_size; i++)
  {
-    if constexpr (Modifier != LOAD_DEFAULT)
-    {
-      _CUB::LoadDirectBlocked(i * ThreadsPerBlock,
-                              CacheModifiedInputIterator<Modifier, InputT, int>(thread_src_ptr),
-                              reinterpret_cast<T(&)[vec_size]>(dst_items[i * vec_size]));
-    }
-    else
-    {
-      _CUB::LoadDirectBlocked(
-        i * ThreadsPerBlock, thread_src_ptr, reinterpret_cast<T(&)[vec_size]>(dst_items[i * vec_size]));
-    }
+    _CUB::LoadDirectBlocked(
+      i * ThreadsPerBlock, thread_src_ptr, reinterpret_cast<T(&)[vec_size]>(dst_items[i * vec_size]));
  }


This is a bug compatible change to keep this as BlockLoadDirectVectorized, which drops the load modifier on fallback

coderabbitai

Actionable comments posted: 1

Review info

Run configuration

Source mode: Auto

Repository: NVIDIA/cccl

Branch: main

File: .coderabbit.yaml

Commit: e37713a

Layers: Repository YAML

Review profile: CHILL

Plan: Enterprise

Run ID: `af6c5aff-68a0-4dd1-9cc6-82a15f7662bc`

Commits

Reviewing files that changed from the base of the PR and between `c7da36f` and `f41c362`.

Files selected for processing: 1

`cub/cub/block/block_load.cuh`

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 10 remain after this review.

coderabbitai · 2026-11-11T16:28:51Z

+    _CUB::LoadDirectBlocked(
+      i * ThreadsPerBlock, thread_src_ptr, reinterpret_cast<T(&)[vec_size]>(dst_items[i * vec_size]));


Major · Functional Correctness · Quick win

Supported by static analysis

Script executed:

git diff cb60424c559952f127b9bdb12afeb4b40e2b3271 f41c362208b10611684300eb522e5eadad37318f -- cub/cub/block/block_load.cuh sed -n '440,495p' cub/cub/block/block_load.cuh sed -n '1225,1260p' cub/cub/block/block_load.cuh

Repository: NVIDIA/cccl

Length of output: 9864

Script executed:

rg -n -F -e 'LoadDirectBlocked(' -e 'LoadDirectStripedVectorized(' -- cub/cub/block/block_load.cuh rg -n -F -e 'LOAD_VOLATILE' -e 'ThreadLoad(' -e 'CacheModifiedInputIterator' -- cub/cub/thread/thread_load.cuh cub/cub/iterator/cache_modified_input_iterator.cuh cub/cub/util_ptx.cuh cub/cub/util_type.cuh cat -n cub/cub/block/block_load.cuh | sed -n '55,145p;345,405p;405,490p;1190,1265p' cat -n cub/cub/iterator/cache_modified_input_iterator.cuh | sed -n '55,250p' cat -n cub/cub/thread/thread_load.cuh | sed -n '45,240p'

Repository: NVIDIA/cccl

Length of output: 38247

Script executed:

cat -n cub/cub/thread/thread_load.cuh | sed -n '240,400p'

Repository: NVIDIA/cccl

Length of output: 8139

important: Preserve Modifier in scalar loads.

When a non-volatile input falls back to scalar loads, LoadDirectBlocked uses ordinary pointer loads and ignores Modifier. This also affects BlockLoad with a cache-modified iterator. For LOAD_VOLATILE, the loads lack the volatile semantics of ThreadLoad<LOAD_VOLATILE>.

Preserve the modifier in the fallback and test the generated loads. Preserve the supported input types and their copy behavior.

github-actions · 2026-11-11T20:09:28Z

🥳 CI Workflow Results

🟩 Finished in 3h 40m: Pass: 100%/214 | Total: 10d 14h | Max: 3h 39m | Hits: 16%/1670711

See results here.

bernhardmgruber · 2026-11-11T20:48:35Z

+_CCCL_DEVICE _CCCL_FORCEINLINE void LoadDirectStripedVectorized(
+  int linear_tid, const InputT* block_src_ptr, OffsetT block_offset, T (&dst_items)[ItemsPerThread])


As discussed offline, I think this shouldn't be necessary since something like LoadDirectStripedVectorized(tid, assume_aligned<A>(ptr + offset), items) should give the desired effect.

I am especially unhappy because this is a public API and would then be the only one with an offset argument. Users will be confused.

If we really need this, can we make the one with an offset parameter internal and leave the public one as is?

miscco requested a review from a team as a code owner November 11, 2026 15:11

miscco requested a review from pauleonix November 11, 2026 15:11

github-project-automation Bot added this to CCCL Nov 11, 2026

github-project-automation Bot moved this to Todo in CCCL Nov 11, 2026

miscco mentioned this pull request Nov 11, 2026

[CUB] Take a block offset in LoadDirectStripedVectorized miscco/cccl#31

Closed

1 of 2 tasks

cccl-authenticator-app Bot moved this from Todo to In Review in CCCL Nov 11, 2026

miscco force-pushed the striped_vectorized_offset_main branch from c7da36f to f41c362 Compare November 11, 2026 16:23

miscco commented Nov 11, 2026

View reviewed changes

coderabbitai Bot reviewed Nov 11, 2026

View reviewed changes

bernhardmgruber reviewed Nov 11, 2026

View reviewed changes

Navigation Menu

Repository navigation

[CUB] Take a block offset in `LoadDirectStripedVectorized` - #12601

[CUB] Take a block offset in `LoadDirectStripedVectorized`#12601
miscco wants to merge 1 commit into
NVIDIA:mainfrom
miscco:striped_vectorized_offset_main

miscco commented Nov 11, 2026

Uh oh!

coderabbitai Bot commented Nov 11, 2026 •

edited

Loading

Summary by CodeRabbit

Uh oh!

miscco Nov 11, 2026

Uh oh!

coderabbitai Bot left a comment

Uh oh!

coderabbitai Bot Nov 11, 2026

Uh oh!

github-actions Bot commented Nov 11, 2026

Uh oh!

bernhardmgruber Nov 11, 2026

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

		_CUB::LoadDirectBlocked(
		i * ThreadsPerBlock, thread_src_ptr, reinterpret_cast<T(&)[vec_size]>(dst_items[i * vec_size]));

		_CCCL_DEVICE _CCCL_FORCEINLINE void LoadDirectStripedVectorized(
		int linear_tid, const InputT* block_src_ptr, OffsetT block_offset, T (&dst_items)[ItemsPerThread])

Conversation

miscco commented Nov 11, 2026

Summary

Test plan

Uh oh!

coderabbitai Bot commented Nov 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary by CodeRabbit

Uh oh!

miscco Nov 11, 2026

Choose a reason for hiding this comment

Uh oh!

coderabbitai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

coderabbitai Bot Nov 11, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented Nov 11, 2026

🥳 CI Workflow Results

🟩 Finished in 3h 40m: Pass: 100%/214 | Total: 10d 14h | Max: 3h 39m | Hits: 16%/1670711

Uh oh!

bernhardmgruber Nov 11, 2026

Choose a reason for hiding this comment

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

coderabbitai Bot commented Nov 11, 2026 •

edited

Loading