You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
map_sort falls back to Spark for non-scalar map key types #5590
supportedScalarSortElementType (serde/QueryPlanSerde.scala:1238) admits the scalar types only, and excludes collated strings, so map_sort on a map keyed by a struct, array or collated string fails the operator.
MapSort.doGenCode sorts with Spark's interpreted ordering, which handles all of those, and MapType is supported recursively by CometBatchKernelCodegen.isSupportedDataType, so the dispatcher covers the gap.
Fix: mix CodegenDispatchFallback into CometMapSort.
The serde's other arm — Incompatible under spark.comet.exec.strictFloatingPoint=true for a floating-point key (:42-46) — would be routed by the same mixin, and that is the right outcome: strict mode is asking for Spark-exact behavior, which is precisely what the dispatcher gives.
This is a Spark 4.x-only serde, so the test belongs with the other 4.x-gated suites.
Activity
sam-1112 commentedon Sep 9, 2026
take