Repository navigation
[EPIC] first class support for struct field / Variant access in Parquet #21308
Description
This issue tracks the hard problems that block efficient querying of struct field columns in Parquet
To investigate, I used an MRE based on a variant column represented as a nested struct. Though, the findings generalize to any schema where queries access leaf fields inside struct columns
Activity
adriangb commentedon Aug 5, 2026
I've opened #24119 as an EPIC for the Parquet-side struct access work — projection pruning, filter pushdown, row-group pruning and the statistics that feed them — since a batch of concrete follow-ups just came out of #24090 (nested schema pruning for narrowed declared schemas) and they needed a home.
That leaves some overlap with this issue, so a suggestion: would it make sense to reframe this EPIC to focus specifically on Variant? The two linked issues here point in slightly different directions:
GetFieldFunc#21306 (struct-aware optimizations are hardcoded toGetFieldFunc, sovariant_getgets none of them) is squarely the Variant blocker, and it's the one that needs a general capability API rather than more struct-specific machinery. It's relevant to both EPICs; I've listed it under [EPIC] Improvements to struct access in Parquet #24119 as shared rather than claiming it.ProjectionExecproduces unknown statistics for allScalarFunctionExproutputs #21307 (ProjectionExecproduces unknown statistics for allScalarFunctionExproutputs) is really a physical-expr / statistics issue that happens to bite struct and Variant queries hardest — arguably it belongs to neither EPIC exclusively.Splitting along "Variant as a first-class type" vs "make Parquet read less for struct columns" seems like the cleaner line to me, but you filed this one and have the fuller picture of where Variant is heading — happy to fold #24119 back in here instead if you'd rather keep it as one umbrella.
cc @alamb
alamb commentedon Aug 7, 2026
i agree this split makes sene to me