You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
String-to-timestamp cast rejects explicit positive years accepted by Spark #5716
Spark accepts a leading + on a timestamp year, but Comet's native string-to-timestamp parser returns NULL. Widening the timestamp fuzz alphabet in PR #5682 exposed this on Spark 4.1.3. Observed inputs include +7528, +00463, +79821, and +2976.
To reproduce
Use Parquet-backed column input to avoid constant folding:
SETspark.sql.session.timeZone=UTC;
SETspark.sql.ansi.enabled=false;
CREATETABLEpositive_year_probe USING parquet ASSELECT'+7528'AS s;
SELECT s, CAST(s ASTIMESTAMP) FROM positive_year_probe;
With native execution disabled, Spark returns the start of year 7528. With Comet enabled, the cast returns NULL. The failing differential run on PR head 0a1fc76a0 plus the review follow-up used 10,000 seeded random strings and exposed nine such bare positive-year inputs. No other mismatch appeared in that run.
Expected behavior / proposed fix
Accept explicit positive years according to Spark's timestamp segment rules. Current patterns use -?[0-9]{4,6}, excluding +. Audit both timestamp and timestamp_ntz paths and numeric decoding before broadening the sign grammar; add column-input Legacy/TRY/ANSI coverage, including signed dates and malformed signs. Remove the narrow positive-bare-year fuzz exclusion in CometNativeCastSuite once fixed.
Activity
peterxcli commentedon Sep 5, 2026
take