Skip to content

Regression: array_sort fails on list with inner non-nullable field #19947

Closed
Bug
#19948
Closed
Regression: array_sort fails on list with inner non-nullable field#19947
Bug
#19948
Assignees
Jefffrey
Labels
bugSomething isn't workingregressionSomething that used to work no longer does

Description

@timsaucer
timsaucer
opened on Jan 23, 2026

Describe the bug

When you have a ListArray and the inner field is non-nullable, array_sort generates an error in 51.0.0. See below for steps to reproduce.

To Reproduce

The following generates an error:

select array_sort(arrow_cast(make_array(1, 3, 5, -5), 'List(non-null Int32)'));

DataFusion error: Internal("Assertion failed: result_data_type == *expected_type: Function 'array_sort' returned value of type 'List(Field { data_type: Int32 })' while the following type was promised at planning time and expected: 'List(Field { data_type: Int32, nullable: true })'")

Expected behavior

You cannot do the above sql with 50.0.0 (In 50.0.0 it doesn't support the "non-null" in parser) but I did test by manually creating and registering a record batch that has a list array with inner non-nullable field. It passes in 50.0.0 and fails in 51.0.0.

Additional context

I believe this was introduced in #17657

Activity

  1. timsaucer
    added
    bugSomething isn't working
    on Jan 23, 2026
  2. timsaucer
    added the
    Bug
    issue type on Jan 23, 2026

  3. timsaucer commented on Jan 23, 2026

    @timsaucer
    timsaucer
    on Jan 23, 2026
    MemberAuthor

    FYI @Jefffrey

  4. Jefffrey commented on Jan 23, 2026

    @Jefffrey
    Jefffrey
    on Jan 23, 2026
    Contributor

    Whoops! Thanks for spotting this, should be a quick fix 👍

  5. Jefffrey
    self-assigned this
    on Jan 23, 2026
  6. github-merge-queue
    added a commit that references this issue on Jan 25, 2026

    fix: maintain inner list nullability for `array_sort` (#19948)

    8023947
  7. comphead
    closed this as completedin #19948on Jan 25, 2026
  8. alamb
    added
    regressionSomething that used to work no longer does
    on Mar 10, 2026

  9. alamb commented on Mar 10, 2026

    @alamb
    alamb
    on Mar 10, 2026
    Contributor

    I am scoping out bugs / regressions to backport to branch-52 if we make another patchset release

    Would this be a good candidate?

  10. alamb
    added 2 commits that reference this issue on Mar 11, 2026

    fix: maintain inner list nullability for `array_sort` (apache#19948)

    a1a392d

    [branch-52] fix: maintain inner list nullability for (#19948) (#20878)

    ff4c630
  11. lukekim
    added a commit that references this issue on Mar 12, 2026

    [branch-52] fix: maintain inner list nullability for (apache#19948) (a…

    fc51241
  12. LiaCastaneda
    added a commit that references this issue on Mar 31, 2026

    Merge datafusion upstream 52.4.0 into branch-52 (#98)

    1a1fb91
  13. rerun-sync
    added a commit that references this issue on Sep 25, 2026

    Remove workarounds for upstream fixes that have landed

    1e9f9f1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugSomething isn't workingregressionSomething that used to work no longer does

Type

Bug

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions