Skip to content

x > ALL (subquery) in the SELECT list fails with an internal schema error unless it is aliased #26622

Open

Description

@alpaylan

Describe the bug

A quantified comparison such as x > ALL (SELECT ...) or x = ANY (SELECT ...) fails to plan when it appears in the SELECT list without an alias. The same query works when the expression is aliased (AS r) and when it is used in WHERE. Using it inside an aggregate, for example bool_or(x > ALL (...)), fails with Schema error: No field named "bool_or(t.x > ALL)".

The error output itself says "This issue was likely caused by a bug in DataFusion's code."

To Reproduce

CREATE TABLE t(x INT) AS VALUES (1), (6);
CREATE TABLE s(y INT) AS VALUES (5);
SELECT x > ALL (SELECT y FROM s) FROM t;
-- duckdb 1.5.6: false, true
-- datafusion main: Internal error (below)
SELECT bool_or(x > ALL (SELECT y FROM s)) FROM t;
-- duckdb 1.5.6: true
-- datafusion main: Schema error: No field named "bool_or(t.x > ALL)"

Actual output on main (datafusion-cli; schema dump shortened):

Error: Optimizer rule 'decorrelate_predicate_subquery' failed
caused by
Check optimizer-specific invariants after optimizer rule: decorrelate_predicate_subquery
caused by
Internal error: Assertion failed: compatible: Failed due to a difference in schemas: original schema: DFSchema { inner: Schema { fields: [Field { name: "t.x > ALL", data_type: Boolean, nullable: true }], ... }, new schema: DFSchema { inner: Schema { fields: [Field { name: "CASE WHEN EXISTS THEN Boolean(false) WHEN EXISTS THEN Boolean(NULL) ELSE Boolean(true) END", data_type: Boolean, nullable: true }], ... }.
This issue was likely caused by a bug in DataFusion's code. Please help us to resolve this by filing a bug report in our issue tracker: https://github.com/apache/datafusion/issues

Error: Optimizer rule 'optimize_projections' failed
caused by
Schema error: No field named "bool_or(t.x > ALL)".
Valid fields are "bool_or(CASE WHEN EXISTS THEN Boolean(false) WHEN EXISTS THEN Boolean(NULL) ELSE Boolean(true) END)".

For comparison, these work on the same build:

SELECT x > ALL (SELECT y FROM s) AS r FROM t;   -- false, true
SELECT x FROM t WHERE x > ALL (SELECT y FROM s); -- 6
SELECT x IN (SELECT y FROM s) FROM t;            -- false, false

Expected behavior

The first query should return false, true (as DuckDB 1.5.6 does), the same result as the aliased version. An optimizer rewrite should not change a plan's output schema.

Additional context

Willingness to contribute

I would like to contribute a fix for this bug, but request guidance from the DataFusion community

Activity

  1. added theissue type on Oct 23, 2026
  2. yashaggarwal85d commented on Oct 23, 2026

    @yashaggarwal85d
    Contributor

    Hi @alpaylan, thanks for the detailed report and the pointer to NamePreserver. I saw you mentioned wanting to fix this yourself. If you haven't started yet, I have a small fix ready that wraps the rewrite in RewriteSetComparison::rewrite_plan with NamePreserver, plus sqllogictest cases for > ALL, = ANY, and < ANY in the SELECT list. Would you mind if I open the PR? I'd leave the bool_or(...) variant (the "Physical plan does not support logical expression Exists" limitation) out of scope so it can be tracked separately, and I'm happy to step back if you already have something in progress.

  3. added a commit that references this issue on Oct 23, 2026
    2a93bcb
  4. alpaylan commented on Oct 23, 2026

    @alpaylan
    Author

    Hi @yashaggarwal85d that's completely fine!

  5. alamb commented on Oct 28, 2026

    @alamb
    Contributor

    I think we should implement support support for this feature as part of the more general subquery decorrelation project described here

    in my opinion WE SHOULD NOT try and extend the existing subquery rewrites to support this type of query as that risks introducing many more wrong results, see #5492 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions