Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG] row-based_udf_test.py::test_hive_empty_* fail for Databricks 14.3 #11990

Closed
razajafri opened this issue Jan 21, 2025 · 3 comments · Fixed by #12082
Closed

[BUG] row-based_udf_test.py::test_hive_empty_* fail for Databricks 14.3 #11990

razajafri opened this issue Jan 21, 2025 · 3 comments · Fixed by #12082
Assignees
@gerashegalov
Labels
audit_4.0.0 Audit related tasks for 4.0.0 bug Something isn't working test Only impacts tests

Comments

@razajafri
Copy link
Collaborator

integration test are failing

FAILED ../../src/main/python/row-based_udf_test.py::test_hive_empty_generic_udf

with

pyspark.sql.utils.AnalysisException: Can not load class 'com.nvidia.spark.rapids.tests.udf.hive.EmptyHiveGenericUDF' when registering the function 'emptygeneric', please make sure it is on the classpath

This could be related to #4883
@razajafri razajafri added ? - Needs Triage Need team to review and classify bug Something isn't working labels Jan 21, 2025
@mattahrens mattahrens added the audit_4.0.0 Audit related tasks for 4.0.0 label Jan 21, 2025
@mattahrens mattahrens removed the ? - Needs Triage Need team to review and classify label Jan 21, 2025
@nartal1
Copy link
Collaborator

nartal1 commented Jan 22, 2025

These tests passed on Spark-4.0, so it could be DB-14.3 specific.

[gw1] [ 50%] PASSED ../../../../integration_tests/src/main/python/row-based_udf_test.py::test_hive_empty_generic_udf[DATAGEN_SEED=1737570019, TZ=UTC, INJECT_OOM] 
[gw0] [100%] XPASS ../../../../integration_tests/src/main/python/row-based_udf_test.py::test_hive_empty_simple_udf[DATAGEN_SEED=1737570019, TZ=UTC, INJECT_OOM] -

@razajafri
Copy link
Collaborator Author

Might be related to #11988

@mythrocks mythrocks changed the title [BUG] row-based_udf_test.py::test_hive_empty_* fail for Databricks 14.3 [TEST BUG] row-based_udf_test.py::test_hive_empty_* fail for Databricks 14.3 Feb 7, 2025
@razajafri razajafri removed their assignment Feb 7, 2025
@mythrocks mythrocks changed the title [TEST BUG] row-based_udf_test.py::test_hive_empty_* fail for Databricks 14.3 [BUG] row-based_udf_test.py::test_hive_empty_* fail for Databricks 14.3 Feb 7, 2025
@mythrocks mythrocks added the test Only impacts tests label Feb 7, 2025
@gerashegalov gerashegalov self-assigned this Feb 7, 2025
@razajafri razajafri mentioned this issue Feb 7, 2025
Open
16 tasks
@gerashegalov gerashegalov mentioned this issue Feb 7, 2025
Merged
@gerashegalov gerashegalov linked a pull request Feb 7, 2025 that will close this issue
Use 350db143 jars in integration tests on [databricks] 14.3 #12082
Merged
gerashegalov added a commit that referenced this issue Feb 10, 2025
@gerashegalov
Use 350db143 jars in integration tests on [databricks] 14.3 (#12082)
1fff480 
Resolves #11988
Resolves #11990
Resolves #12020 

 and related issues

- Consolidate DBR-specific logic in jenkins/databricks/common_vars.sh
- Add DBR versions suffix when necessary

---------

Signed-off-by: Gera Shegalov <gshegalov@nvidia.com>
@gerashegalov
Copy link
Collaborator

Fixed by #12082

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@gerashegalov gerashegalov

Labels
audit_4.0.0 Audit related tasks for 4.0.0 bug Something isn't working test Only impacts tests
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Use 350db143 jars in integration tests on [databricks] 14.3 gerashegalov/spark-rapids
5 participants
@gerashegalov @mattahrens @mythrocks @razajafri @nartal1