You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
partition_to_path panics for identity/truncate fields whose source column was dropped #3381
Verified on main at 49b91d1 (the version field only lists releases).
PartitionSpec::partition_to_path panics when a partition field's source column is no longer in the schema and the field is identity or truncate over a non-string type.
#2773 made partition_type fall back to a source-independent result type for dropped source columns (Unknown for identity/truncate), so the partition_type(..).unwrap() succeeds. The panic happens later: to_human_string falls back to Display for Datum, which has no arm for an Unknown-typed int/long and hits unreachable!() (datum.rs).
Java's PartitionSpec.partitionToPath handles the same case: resultType uses UnknownType for a dropped source and toHumanString falls back to value.toString().
Today this is reachable through the public partition_to_path. #3253 also starts summarizing deleted files under their historical spec, which would put it on the commit path (see the discussion there).
To Reproduce
use iceberg::spec::{Literal,NestedField,PartitionSpec,PrimitiveType,Schema,Struct,Transform,Type};let old_schema = Schema::builder().with_fields(vec![NestedField::required(1,"x",Type::Primitive(PrimitiveType::Int)).into(),NestedField::required(2,"y",Type::Primitive(PrimitiveType::Int)).into(),]).build().unwrap();let spec = PartitionSpec::builder(old_schema).add_partition_field("y","p",Transform::Identity).unwrap().build().unwrap();// `y` has been dropped from the current schema.let current = Schema::builder().with_fields(vec![NestedField::required(1,"x",Type::Primitive(PrimitiveType::Int)).into()]).build().unwrap();// Panics: internal error: entered unreachable code (datum.rs)
spec.partition_to_path(&Struct::from_iter([Some(Literal::int(1))]), current.into());
Expected behavior
No panic. Format the value as Java does for an unknown source type (p=1, p=10).
Activity