Skip to content

[Bug] next_day trims whitespace from dayOfWeek; Spark does not #4450

Closed

Description

@andygrove

Describe the bug

next_day(date, dayOfWeek) in Comet trims leading/trailing whitespace from the dayOfWeek argument before matching. Spark's DateTimeUtils.getDayOfWeekFromString does not trim, so values like ' MO ' succeed in Comet but Spark either returns NULL (non-ANSI) or throws (ANSI).

The trim happens in datafusion-spark::SparkNextDay.

Steps to reproduce

SELECT next_day(date('2024-01-01'), ' MO ');
-- Spark: NULL (or throws under ANSI)
-- Comet: 2024-01-08

Expected behavior

Do not trim; match Spark's character-for-character matching.

Additional context

Surfaced by the date/time audit (#4448). Reproducer captured as an ignored test in spark/src/test/resources/sql-tests/expressions/datetime/next_day.sql; remove the ignore(...) when this is fixed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions