You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
least orders a null inside a struct or array last, where Spark orders it first #6759
least over structs or arrays orders a null inside the value after every other value. Spark orders it before every other value, in least as in greatest, so Comet can return a different value from Spark.
Comet uses its own SparkGreatestLeast only for floats and nested types with a float leaf. Other nested types go to DataFusion's least, whose comparator uses nulls_first: false.
With spark.comet.exec.range.enabled=true, spark.comet.sparkToColumnar.enabled=true, and spark.comet.sparkToColumnar.supportedOperatorList=Range, Comet runs this natively. For id = 3 Spark returns [null, 3] and Comet returns [3, 1]. least(array(CAST(id AS INT), 1), array(IF(id = 3, NULL, 0), 3)) behaves the same way.
Expected behavior
A null inside a struct or array ranks before every other value, so least returns [null, 3] for id = 3.
Activity