Repository navigation
[Bug] make_timestamp does not throw under spark.sql.ansi.enabled=true #4679
Description
Describe the bug
MakeTimestamp honours spark.sql.ansi.enabled in Spark 3.4 / 3.5 / 4.0: under ANSI mode it throws on an invalid (year, month, day, hour, min, sec[, timezone]) argument (ansiDateTimeArgumentOutOfRange / invalidFractionOfSecondError), and returns NULL only when ANSI is disabled.
The Comet wiring (CometMakeTimestamp, spark/src/main/scala/org/apache/comet/serde/datetime.scala) does not check failOnError. CometMakeTimestamp is a bare CometCodegenDispatch[MakeTimestamp] with no ANSI gating, so under ANSI mode Comet returns NULL where Spark would throw.
This is the same shape as the make_date ANSI divergence captured in #4451 (make_date_ansi.sql).
Steps to reproduce
With spark.sql.ansi.enabled=true, evaluate make_timestamp with an out-of-range argument (for example an invalid month or a seconds value outside the valid fraction range). Spark throws; Comet returns NULL.
Expected behavior
Under spark.sql.ansi.enabled=true, Comet should match Spark and throw on invalid arguments. The support level should be gated appropriately until a fix lands, and a make_timestamp_ansi.sql regression test should be added once the fix is in place (analogous to make_date_ansi.sql).
Additional context
Split out from #4502 (item 4), surfaced by the audit-comet-expression skill run in #4448. Cross-references #4451.
Activity
marvelshan commentedon Jun 27, 2026
Sorry! @comphead I didn't notice that you'd self-assigned it
andygrove commentedon Jun 29, 2026
Verified empirically on Spark 4.1 (ANSI on by default):
make_timestampalready throws under ANSI for invalid arguments, matching Spark, so this is not a divergence on the current codebase.Root cause of the stale report:
CometMakeTimestampis a codegen-dispatch serde, andspark.comet.exec.scalaUDF.codegen.enableddefaults totrue. The dispatcher compiles Spark's ownMakeTimestamp.doGenCodeinto the per-batch kernel, so under ANSI it produces the same throw site as Spark, and the JVM-UDF path propagates the exception (native/spark-expr/src/jvm_udf/mod.rscheck_exception) rather than returning NULL. NofailOnErrorgating is needed.This was resolved by #4417 (merged 2026-05-26), which added the codegen-dispatch date/time path and the
make_timestamp_ansi.sqlregression test, predating this issue (filed 2026-06-18); the audit that split this out from #4502 did not account for #4417.Confirmed passing:
make_timestamp_ansi.sql(month/day/hour out-of-range throw, plus a sentinelcheckSparkAnswerAndOperatorquery proving native execution, so the throws are not vacuous Spark fallbacks).make_timestamp(..., 61)throwsSecondOfMinuteandmake_timestamp(..., 60.5)throws the fraction-of-second error, both matching Spark, with the sentinel confirming native execution.Closing as already resolved.