You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
push_down_filter infers a null-rejecting predicate onto the subquery side of a null-aware NOT IN join, producing wrong results #23900
push_down_filter infers equi-key predicates across a join's ON keys and pushes them to the opposite input. For a null-aware join — the LeftAnti join produced by NOT IN with a nullable subquery — an outer predicate on the left join key is rewritten onto the right/subquery side. Because an inferred predicate must be null-rejecting to be pushed through a non-inner join, this drops the subquery's NULL rows, which breaks the three-valued NOT IN semantics: a NULL in the subquery key must reach the join so the result is empty (NOT IN is UNKNOWN for every outer row).
This is the same class of bug as #23847 / #23848 (FilterNullJoinKeys), in a different rule.
To Reproduce
CREATETABLEouter_t(id INT) ASVALUES (3), (7);
CREATETABLEsub_t(id INT) ASVALUES (NULL);
SELECT id FROM outer_t WHERE id >5AND id NOT IN (SELECT id FROM sub_t);
Expected behavior
0 rows. The subquery contains a NULL, so id NOT IN (SELECT id FROM sub_t) evaluates to UNKNOWN for every row, and WHERE keeps only TRUE rows. (Matches PostgreSQL.)
Actual behavior
Returns 7. push_down_filter infers sub_t.id > 5 from the outer predicate outer_t.id > 5 (via the outer_t.id = sub_t.id join key) and pushes it onto the subquery input, dropping its NULL row. The anti-join then sees an empty subquery and keeps 7.
Confirmed by disabling the fix and re-running: the query returns 7 without the guard and 0 rows with it.
Additional context
The offending transform is in infer_join_predicates (datafusion/optimizer/src/push_down_filter.rs), which does not check join.null_aware.
Activity
fix: don't infer join predicates for null-aware joins in push_down_fi…
fix: don't infer join predicates for null-aware joins in push_down_fi…