Skip to content

Support nested array with struct elements in sort_array #4584

Closed
Closed
Support nested array with struct elements in sort_array#4584
Assignees
comphead
Labels
enhancementNew feature or requestpriority:mediumFunctional bugs, performance regressions, broken features

Description

@hsiang-c
hsiang-c
opened on Jun 3, 2026

What is the problem the feature request solves?

When we sort an array<array<struct<...>>> or array<struct<array<...>>>, Comet falls back to Spark b/c DF doesn't support these nested types.

One of our use cases is sort_array(collect_set(named_struct())) and the result of collect_set is of type ArrayType(StructType(... ArrayType(StructType..)))

Describe the potential solution

No response

Additional context

No response

Activity

  1. hsiang-c
    added
    enhancementNew feature or request
    on Jun 3, 2026
  2. github-actions
    added
    requires-triage
    on Jun 3, 2026
  3. comphead
    self-assigned this
    on Jun 4, 2026
  4. comphead
    added
    priority:mediumFunctional bugs, performance regressions, broken features
    and removed
    requires-triage
    on Jun 4, 2026

  5. comphead commented on Jun 5, 2026

    @comphead
    comphead
    on Jun 5, 2026
    Contributor

    DF actually support sort of nested arrays

    > select array_sort(make_array(make_array(4, 1, 2, 3, null), make_array(null)));
+-----------------------------------------------------------------------------------------------+
| array_sort(make_array(make_array(Int64(4),Int64(1),Int64(2),Int64(3),NULL),make_array(NULL))) |
+-----------------------------------------------------------------------------------------------+
| [[NULL], [4, 1, 2, 3, NULL]]                                                                  |
+-----------------------------------------------------------------------------------------------+

  6. comphead commented on Jun 5, 2026

    @comphead
    comphead
    on Jun 5, 2026
    Contributor
    >   SELECT array_sort(make_array(
    make_array(named_struct('a', 2)),
    make_array(named_struct('a', 1))
  ));

+-------------------------------------------------------------------------------------------------------------------+
| array_sort(make_array(make_array(named_struct(Utf8("a"),Int64(2))),make_array(named_struct(Utf8("a"),Int64(1))))) |
+-------------------------------------------------------------------------------------------------------------------+
| [[{a: 1}], [{a: 2}]]                                                                                              |
+-------------------------------------------------------------------------------------------------------------------+
1 row(s) fetched. 
Elapsed 0.002 seconds.

  7. comphead commented on Jun 5, 2026

    @comphead
    comphead
    on Jun 5, 2026
    Contributor

    Looks like it was fixed by apache/datafusion#21041

  8. hsiang-c
    closed this as completedon Jul 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

enhancementNew feature or requestpriority:mediumFunctional bugs, performance regressions, broken features

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions