Repository navigation
Support sort merge join with a join condition #398
Description
What is the problem the feature request solves?
Currently SMJ with join condition is not supported by Comet and falls back to Spark. The feature was added into DataFusion but we've not incorporated it yet. We should do it to add the support.
Describe the potential solution
No response
Additional context
No response
Activity
comphead commentedon May 31, 2024
@viirya can I take it as I spent some time with SMJ?
andygrove commentedon Sep 20, 2024
Re-opening this since we discovered that there are some correctness and performance issues and currently have this feature disabled by default
andygrove commentedon Nov 5, 2024
There is an existing test
SortMergeJoin with join filterthat we need to enable as part of closing this issue. It was ignored in #1057comphead commentedon Nov 5, 2024
Thanks @andygrove I'll take if from now
comphead commentedon Nov 18, 2024
Waiting for DataFusion 44.0.0
andygrove commentedon Jan 10, 2025
Re-opening this issue because we only enable this feature in tests currently due to poor performance in benchmarks
comphead commentedon May 8, 2026
@mbutrovich do you think this can be closed?
mbutrovich commentedon May 8, 2026
When we pick up DF 54. The config gets removed in the
df54branch already.