You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Projection field metadata is lost during physical plan optimization #24721
Field metadata on a projection's output schema (for example Arrow extension type metadata, or custom key/value metadata added via a select with an alias) can silently disappear when the physical optimizer removes or rewrites ProjectionExec nodes.
The user-visible effects are:
The schema of query results loses field-level metadata that the plan declared (e.g. extension types like arrow.uuid are stripped)
Expressions that read field metadata return wrong results, because they see the underlying source field instead of the projected field that carried the metadata
To Reproduce
Run a query whose plan contains a projection that only adds field metadata, e.g.:
ProjectionExec: i@0 AS i
output field metadata = {"event_field": "true"}
DataSourceExec: i
field metadata = {}
The optimizer removes the projection (it looks like an identity projection), and the output schema no longer has the metadata.
Similarly, with a function that reads field metadata (such as a UDF like arrow_metadata):
ProjectionExec: arrow_metadata(i, 'event_field') AS metadata
ProjectionExec: i@0 AS i
output metadata = {"event_field": "true"}
DataSourceExec: i
metadata = {}
the projections are collapsed and the function evaluates against the scan field (no metadata) instead of the inner projection field.
Expected behavior
Physical plan optimization should not change the output schema of the plan: field metadata present on projection output fields before optimization should still be present afterwards, and metadata-reading expressions should return the same results with and without optimization (true rather than NULL in the example above).
Activity
gene-bordegaray commentedon Aug 27, 2026
take
fix: preserve projection metadata during optimization (apache#24670)