Skip to content

ANSI mode test coverage: re-enable stale ignored tests and add missing cases #5076

Closed

Description

@andygrove

What is the problem the feature request solves?

An ANSI-mode audit against Spark 4.1.1 found several test-coverage gaps around ANSI behavior. Closing them locks in the behavior that is already correct and captures the known divergences as regression tests.

Stale disabled tests to re-enable:

  • spark/src/test/resources/sql-tests/expressions/array/element_at_ansi.sql and array/get_array_item_ansi.sql are entirely query ignore(#3375) with comments claiming "Comet throws: different error message". ANSI mode array access error messages don't match Spark format #3375 is closed: the error-message work merged (native GetArrayItem re-enabled shortly after), and CometArrayExpressionSuite now asserts exact Spark message parity for both expressions. These files should be un-ignored.

Missing ANSI SQL file tests (compare the existing math/abs_ansi.sql, math/arithmetic_ansi.sql, url/parse_url_ansi.sql pattern):

  • round: no ANSI case at all; also no integral negative-scale overflow case in any mode.
  • conv: no ANSI overflow case (e.g. conv('FFFFFFFFFFFFFFFFF', 16, 10) throws under ANSI).
  • elt: no out-of-range index case, no ANSI config.

Specific untested inputs surfaced by the audit (each is the repro for a filed bug; add as query ignore(<issue-url>) until fixed, then un-ignore):

  • Long.MinValue div -1 under ANSI (IntegralDivide overflow).
  • CAST(1.0 AS DOUBLE) % CAST(0.0 AS DOUBLE) under ANSI (float remainder by zero).
  • CAST(true AS DECIMAL(1,1)) in legacy/try mode (boolean-to-decimal overflow).
  • round(5000000000000000000L, -19) in both modes.

Assertion strength: several suites assert only on message substrings, which is why wrong-exception-type divergences pass today (decimal div-by-zero, next_day, make_date, decimal-to-decimal cast). Where the error class is expected to match, assert on the exception type / getCondition() rather than substrings, and remove the decimal-to-decimal special case at CometCastSuite.scala:2402-2405 once the underlying issue is fixed.

Additional context

Found by an ANSI-mode audit of all native expressions against Spark 4.1.1.

Activity

  1. rich7420 commented on Sep 9, 2026

    @rich7420
    Contributor

    take

  2. rich7420 commented on Sep 9, 2026

    @rich7420
    Contributor

    I plan to split this into three focused PRs:

    1. Restore the remaining get_array_item and element_at ANSI tests.
    2. Fill the remaining round, conv and elt coverage, including the listed overflow, division and cast regressions.
    3. Strengthen exception-type and error-condition assertions where Spark parity is established.

    I'll start with array access. Cases blocked by open bugs will retain their tracking links until fixed.

  3. andygrove commented on Sep 28, 2026

    @andygrove
    MemberAuthor

    This is done: #5798 re-enabled the element_at and get_array_item ANSI files, #5799 added the round, conv and elt ANSI cases along with the listed repros, and #5800 moved the arithmetic assertions to error class and SQLSTATE parity. The decimal-to-decimal special case in CometNativeCastSuite is the only leftover, and it goes away with the #5072 fix in #5169.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions