Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[AUDIT][SPARK-36612][SQL] Support left outer join build left or right outer join build right in shuffled hash join #9161

Open
cindyyuanjiang opened this issue Aug 31, 2023 · 2 comments
Assignees
Labels
audit General label for audit related tasks audit_3.5.0 feature request New feature or request

Comments

@cindyyuanjiang
Copy link
Collaborator

cindyyuanjiang commented Aug 31, 2023

This change adds support for shuffled hash join in these scenarios:

  • left outer join with left-side build
  • right outer join with right-side build

We support GpuShuffledHashJoinExec. I am filing this to raise visibility.

Apache commit: apache/spark@0effbec16e

@cindyyuanjiang cindyyuanjiang added feature request New feature or request ? - Needs Triage Need team to review and classify audit General label for audit related tasks audit_3.5.0 labels Aug 31, 2023
@revans2
Copy link
Collaborator

revans2 commented Sep 1, 2023

I'm not sure how critical this is. We will fall back to the CPU if we see this happen, and ShuffledHashJoins are off by default in Spark, so it is more likely to be a sort merge join that we see, and because we don't have any info about the side of the join, we just pick a build side that works for us.

@mattahrens mattahrens removed the ? - Needs Triage Need team to review and classify label Sep 6, 2023
@res-life res-life self-assigned this Feb 26, 2025
@res-life
Copy link
Collaborator

A customer needs this, I'm working on this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
audit General label for audit related tasks audit_3.5.0 feature request New feature or request
Projects
None yet
Development

No branches or pull requests

4 participants