Repository navigation
Description
This is a rather low-priority issue given that this specialization of cub::WarpScan is only used for LOGICAL_WARP_THREADS is not a power of 2 which I assume is quite exotic. But while adding new member functions in #5379 I observed quite a few potential inefficiencies. Most of them come from the fact that when CUB was sprinkled with __syncwarp() for Volta/Independent Thread Scheduling in 83ab806, there was little regard for how else the code could be modified to fit the new paradigm.
- All loads and store from/to shared memory are done with
ThreadLoad<LOAD_VOLATILE>/ThreadStore<STORE_VOLATILE>since long before the Volta changes. Doing volatile loads/stores should not be necessary in the presence of the explicit warp synchronization. - Afaik there is at least one occurrence of redundant synchronization in line 417 here:
cccl/cub/cub/warp/specializations/warp_scan_smem.cuh
Lines 411 to 423 in 3d67fa8
ThreadStore<STORE_VOLATILE>(&temp_storage[HALF_WARP_THREADS + lane_id], (CellT) inclusive); __syncwarp(member_mask); warp_aggregate = (T) ThreadLoad<LOAD_VOLATILE>(&temp_storage[WARP_SMEM_ELEMENTS - 1]); __syncwarp(member_mask); // Update inclusive with initial value inclusive = scan_op(initial_value, inclusive); // Get exclusive from exclusive ThreadStore<STORE_VOLATILE>(&temp_storage[HALF_WARP_THREADS + lane_id - 1], (CellT) inclusive);
The second store is explicitly done with an offset such that the aggregate is not overwritten. - There is an explicit optimization for
cuda::std::pluson primitive types based on knowing the identity value. That optimization could easily be done for other operators as well (cuda::std::multiplies,cuda::minimum,cuda::maximum,cuda::std::bit_or,cuda::std::bit_and,cuda::std::bit_xor) incccl/cub/cub/warp/specializations/warp_scan_smem.cuh
Lines 134 to 139 in 3d67fa8
// Update partial if addend is in range if (HAS_IDENTITY || (lane_id >= OFFSET)) { T addend = (T) ThreadLoad<LOAD_VOLATILE>(&temp_storage[HALF_WARP_THREADS + lane_id - OFFSET]); partial = scan_op(addend, partial); } HAS_IDENTITYcould be true for all of these operators. See alsoidentity_vfrom CUB - Add internal thread and warp utils (SplitWarpReducePR) #5317 - Another optimization currently only done for
cuda::std::pluson integers is calculating the exclusive scan from the inclusive scan asexclusive = inclusive - inputinstead of communicating through shared memory. The same could be done forcuda::std::bit_xorand in theorycuda::std::multipliesalthough integer division might not actually be faster than going through shared memory. - Remove
CellTand all the C-style casts associated with it.cccl/cub/cub/warp/specializations/warp_scan_smem.cuh
Lines 86 to 87 in 3d67fa8
/// Storage cell type (workaround for SM1x compiler bugs with custom-ops like Max() on signed chars) using CellT = T;
Metadata
Metadata
Assignees
Labels
Type
Projects
- StatusShow more project fieldsTodo
Activity