You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
IntegralDivide returns wrong result for Long.MinValue div -1 under ANSI mode #5065
Under ANSI mode, IntegralDivide (the div operator) silently returns a wrong value for Long.MinValue div -1 instead of throwing ARITHMETIC_OVERFLOW.
Spark 4.1 (arithmetic.scala) sets checkDivideOverflow = true for LongType when failOnError is set, and throws QueryExecutionErrors.overflowInIntegralDivideError ("Overflow in integral divide. Use try_divide to tolerate divisor being 0 and return NULL instead.") when input1 == Long.MinValue && input2 == -1.
Comet's serde (spark/src/main/scala/org/apache/comet/serde/arithmetic.scala, CometIntegralDivide) casts both operands to DECIMAL(19,0) and performs a decimal divide. The result 9223372036854775808 fits in precision 19, so the CheckOverflow wrapper passes. The final cast back to LongType is hard-coded to legacy eval mode:
The legacy decimal-to-int64 path truncates (native/spark-expr/src/conversion_funcs/numeric.rs, cast_decimal_to_int32_up! non-ANSI arm), producing -9223372036854775808 with no error.
SparkError::IntegralDivideOverflow already exists in native/common/src/error.rs and is wired through ShimSparkErrorConverter, but no native kernel ever constructs it.
Steps to reproduce
SETspark.sql.ansi.enabled=true;
SELECT CAST(-9223372036854775808AS LONG) div CAST(-1AS LONG);
Spark: throws SparkArithmeticException with error class ARITHMETIC_OVERFLOW ("Overflow in integral divide").
Comet: returns -9223372036854775808.
Expected behavior
Comet should throw the same ARITHMETIC_OVERFLOW error as Spark under ANSI mode (and return the wrapped value under legacy mode, which it already does).
Additional context
Found by an ANSI-mode audit of all native expressions against Spark 4.1.1. Divide-by-zero handling in IntegralDivide is correct; only the Long.MinValue div -1 overflow case is affected. There is no existing test for this input.
Activity
docs: document known correctness issues in the compatibility guide (#…
docs: drop compatibility notes for bugs that are now fixed
docs: drop compatibility notes for bugs that are now fixed (#5154)
docs: document known correctness issues in the compatibility guide (a…
docs: drop compatibility notes for bugs that are now fixed (apache#5154)