Repository navigation
Add spilling support for HashJoin #12952
Description
Is your feature request related to a problem or challenge?
It would be helpful to have spilling support for the HashJoin. If there is not enough memory on the machine the join can leverage local disk to spill intermittent results
For reference the spilling support for SortMergeJoin
#9359
Some ideas was also covered in
#1599
More reading also https://facebookincubator.github.io/velox/develop/spilling.html
Describe the solution you'd like
No response
Describe alternatives you've considered
No response
Additional context
No response
Activity
comphead commentedon Oct 15, 2024
@andygrove @viirya cc
s5dsn-eqee commentedon Oct 27, 2024
take
feat(physical-plan/hash_join): introduce partitioned_build module
feat(hash-join): grace-hash spill to disk when memory is exhausted
alamb commentedon Aug 29, 2026
@jayzhan211 consolidated this project / plan in
Let's focus our discussion there