Support reverse for ListView

Is your feature request related to a problem or challenge?

ListView, as opposed to List, is not supported by the reverse function, yet.

Describe the solution you'd like

Unless I'm misunderstanding the ListView spec, I think this should be possible by performing offset + size - 1 to compute the new offsets and just size * -1.

Hopefully, downstream usages use iterators and not regular for loops that assume strictly increasing indexes, but I'd call that a bug if they didn't.

Describe alternatives you've considered

Cast to List instead, however, based on some profiling we've done in the other direction (converting List to ListView), this is a terrible idea.

Additional context

No response

take

I don't think negative sizes would be possible currently, as sizes are ScalarBuffer<OffsetSize> where offsets rely on being usize, e.g. supports as_usize().
https://github.com/apache/arrow-rs/blob/cd61ead4f8f4365e9406f7f3d1056d195159e748/arrow-array/src/array/list_view_array.rs#L111

https://github.com/apache/arrow-rs/blob/cd61ead4f8f4365e9406f7f3d1056d195159e748/arrow-array/src/array/list_view_array.rs#L293-L294

@alamb

@alamb could you chime in whether the implementation is in the spirit of the spec? As far as I understood the spec, size should be possible to be a signed integer, no?

In the meantime I might implement it using a less efficient approach, and then add a benchmark so that it will be satisfying to move to this better approach if we get the blessing to do so! But that requires some type changes and thus releases in arrow to be unblocked.

I've pushed a PR to implement reverse #18424 (using take over the values , not operations on offsets and sizes) and a PR with a benchmark #18425.

On second thought, maybe it's better that reverse is not supported for ListView until it can be done without materializing (which my PR does)?

@alamb could you chime in whether the implementation is in the spirit of the spec? As far as I understood the spec, size should be possible to be a signed integer, no?

I think the spec allows signed integer for compatibility with Java (which doesn't have an unsigned integer type). Indeed if I try a negative size on pyarrow it sees the resultant array as invalid:

>>> import pyarrow as pa
>>> pa.__version__
'22.0.0'
>>> values = pa.array([1, 2, 3, 4])
>>> offsets = pa.array([0, 1, 2])
>>> sizes = pa.array([2, 2, -2])
>>> pa.ListViewArray.from_arrays(offsets, sizes, values)
<pyarrow.lib.ListViewArray object at 0x1023caa40>
[
  [
    1,
    2
  ],
  [
    2,
    3
  ],
<Invalid array: Array length is negative>
]

So I don't think it'll be as simple as negating the size.

Hopefully, downstream usages use iterators and not regular for loops that assume strictly increasing indexes, but I'd call that a bug if they didn't.

I think the whole point of ListView is that it has offset + length precisely so that the indexes do not have to be strictly increasing

List / LargeList require strictly increasing indexes as there are no separately encoded lengths

I think reversing a list view can be done by reversing the offsets and lengths separately

I think reversing a list view can be done by reversing the offsets and lengths separately

But this will still require negative arguments for sizes (which is not currently supported because it uses Offset), or am I misunderstanding? If we have an array which is physically at [n, n + len], the reverse will be [n + len, n] which needs to have a negative size/stride.

Sizes and offsets have the identical bit width and both 32-bit and 64-bit signed integer options are supported.

If allowing for negative sizes is in the spirit of the spec, should we open a tracking issue for that on arrow-rs to discuss how to approach it? "Support negative sizes for ListViews"

(Posted this in a comment here, but probably worth adding to this discussion.)

Support reverse for ListView (#18424)

Support reverse for ListView (apache#18424)

Navigation Menu

Repository navigation

Support `reverse` for ListView #18350

Is your feature request related to a problem or challenge?

Describe the solution you'd like

Describe alternatives you've considered

Additional context

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Support reverse for ListView #18350

Description

Is your feature request related to a problem or challenge?

Describe the solution you'd like

Describe alternatives you've considered

Additional context

Activity

vegarsti commented on Oct 30, 2025

vegarsti commented on Oct 30, 2025

brancz commented on Oct 30, 2025

vegarsti commented on Oct 31, 2025

vegarsti commented on Nov 1, 2025

vegarsti commented on Nov 1, 2025

Jefffrey commented on Nov 2, 2025

alamb commented on Nov 3, 2025

alamb commented on Nov 3, 2025

vegarsti commented on Nov 3, 2025

vegarsti commented on Nov 3, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

Support `reverse` for ListView #18350