Repository navigation
Correlated IN subquery fails with "Projections require unique expression names" when its column is cast #26033
Description
Describe the bug
A correlated IN subquery fails to plan when its output column is cast, for example because the IN value has a different type. The query is valid, and DuckDB runs it.
To Reproduce
CREATE TABLE o(k INT, b BIGINT) AS VALUES (1, 1), (2, 2), (3, 30);
CREATE TABLE i(y INT) AS VALUES (1), (2), (3);
SELECT o.k FROM o WHERE o.b IN (SELECT i.y FROM i WHERE i.y = o.k) ORDER BY o.k;Optimizer rule 'decorrelate_predicate_subquery' failed
caused by
Error during planning: Projections require unique expression names but the expression "CAST(i.y AS Int64)" at position 0 and "i.y" at position 1 have the same name. Consider aliasing ("AS") one of them.
The same error happens for 1 IN (...), o.k + 0 IN (...), NOT IN, and an explicit SELECT CAST(i.y AS BIGINT) in the subquery. With SELECT i.y + 0 AS y, the error is Schema contains qualified field name i.y and unqualified field name y which would be ambiguous.
It works when no cast is needed, as in o.k IN (SELECT i.y FROM i WHERE i.y = o.b), when the subquery is not correlated, or when the cast has another name (CAST(i.y AS BIGINT) AS y2).
Expected behavior
1, 2, which DuckDB 1.1.2 returns.
Additional context
PullUpCorrelatedExpr adds the correlated column i.y to the projection of the subquery, so that the join can read it. It skips the column only when the projection already holds the exact expression i.y:
datafusion/datafusion/optimizer/src/decorrelate.rs
Lines 679 to 695 in c3ef346
| fn collect_missing_exprs( | |
| &self, | |
| exprs: &[Expr], | |
| correlated_subquery_cols: &BTreeSet<Column>, | |
| ) -> Result<Vec<Expr>> { | |
| let mut missing_exprs = vec![]; | |
| for expr in exprs { | |
| if !missing_exprs.contains(expr) { | |
| missing_exprs.push(expr.clone()) | |
| } | |
| } | |
| for col in correlated_subquery_cols.iter() { | |
| let col_expr = Expr::Column(col.clone()); | |
| if !missing_exprs.contains(&col_expr) { | |
| missing_exprs.push(col_expr) | |
| } | |
| } |
Here the projection holds CAST(i.y AS Int64), which type coercion added for the IN comparison and which has the same output name. LogicalPlanBuilder::project then rejects the new projection.
#4478 is the same name rule for a projection that a user writes, SELECT a, CAST(a AS INT). Here the user writes no duplicate name, and the pull up creates it.
Found on main at c3ef346.
Activity
vinay121314 commentedon Oct 5, 2026
Hi @comphead, are you actively working on this? If not, I'd be happy to pick it up.
fix: look up a node's outputs by column, not by flat name, in PushDow…
alamb commentedon Oct 28, 2026
I think we should implement support support for this feature as part of the more general subquery decorrelation project described here
in my opinion WE SHOULD NOT try and extend the existing subquery rewrites to support this type of query as that risks introducing many more wrong results, see #5492 (comment)