You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Support merge RowSelectors when creating RowSelection #2858
let mut selection = RowSelection::from(vec![
RowSelector::skip(10),
RowSelector::select(10),
RowSelector::skip(5),
RowSelector::skip(5),
]);
we have tree pages each have 10 rows.
Before introduced #2473, when skipping two RowSelector::skip(5):
the first one is not enough to skip the whole page, so we need read the page bytes to skip 5 values decoding.
But after #2473, parquet reader not fetch the third page's bytes according to the selection
Activity
Ted-Jiang commentedon Oct 11, 2022
@tustvold @thinkharderdev @alamb
alamb commentedon Oct 13, 2022
@Ted-Jiang I remember some code that combines RowSelectors somewhere -- perhaps https://docs.rs/parquet/24.0.0/parquet/arrow/arrow_reader/struct.RowSelection.html has some inspiration
alamb commentedon Nov 11, 2022
label_issue.pyautomatically added labels {'parquet'} from #2994