Repository navigation
array functions don't work with FixedSizeList #8084
Description
Describe the bug
none of the scalar array_ functions work on FixedSizeList dtype.
To Reproduce
given a table of
+-----------------+--------+
| fixed_size_list | list |
+-----------------+--------+
| [0, 1] | [0, 1] |
| [0, 2] | [0, 2] |
+-----------------+--------+
I should be able to use array_* functions on both list and fixed_size_list fields
SELECT array_length(list) FROM my_table
+-----------------------------+
| array_length(my_table.list) |
+-----------------------------+
| 2 |
| 2 |
+-----------------------------+Full reproducible example here: https://gist.github.com/universalmind303/e2e59cd6d84656f5f20e78bd6e9c6c74
Expected behavior
SELECT array_length(fixed_size_list) FROM my_table
+----------------------------------------+
| array_length(my_table.fixed_size_list) |
+----------------------------------------+
| 2 |
| 2 |
+----------------------------------------+Additional context
I don't think all of them should work. Such as ones that mutate the array (repeat, remove, ...). But I'd expect many of the immutable functions to work such as has, length, element, and so on.
Activity
alamb commentedon Nov 7, 2023
The same thing probably goes for
LargeListas well (it isn't supported)Weijun-H commentedon Nov 8, 2023
Yes, I believe the array function now only accepts
List. I am happy to take this ticket.alamb commentedon Nov 8, 2023
Thanks @Weijun-H
In order to make this project tractable, I suggest picking one array function (ideally a simple one) and figuring out a good pattern for:
In particular, I think this may require writing some generic code (over OffsetSize) trait
jayzhan211 commentedon Nov 8, 2023
For LargeList, you can take #7897 for reference
Jefffrey commentedon Sep 19, 2025
I plan to go through the array functions in datafusion-functions-nested and check this support as well as tests
FixedSizeListArrayfor array #8249