[BUG] [Spark 4] Exceptions from DateTimeException
s do not match Spark exceptions with ANSI enabled
#11641
Labels
Spark 4.0+
Spark 4.0+ issues
Description:
With ANSI enabled, when reading invalid date in EXCEPTION mode, the exception string from Spark does not match the exception from Spark.
Noticed in
csv_test.py::test_read_valid_and_invalid_dates
On Spark pre 4.0:
DateTimeException
On Spark 4.0+:
On the Spark RAPIDS:
DateTimeException: One or more values is not a valid date
Repro:
Workaround for the test failure is in flight, once that fix is in -- run the
test_read_valid_and_invalid_dates
test on Spark 4.0 with EXCEPTION mode enabled.Expected behavior
The overflow exception should match what is produced from Spark 4.
Misc:
Similar to #11556 + #11552 + #11550 -- exception names and types not aligning in ansi mode Spark 4.0+
The text was updated successfully, but these errors were encountered: