Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-21826][SQL] outer broadcast hash join should not throw NPE #19036

Closed
wants to merge 1 commit into from

Conversation

cloud-fan
Copy link
Contributor

What changes were proposed in this pull request?

This is a bug introduced by https://github.com/apache/spark/pull/11274/files#diff-7adb688cbfa583b5711801f196a074bbL274 .

Non-equal join condition should only be applied when the equal-join condition matches.

How was this patch tested?

regression test

@cloud-fan
Copy link
Contributor Author

@SparkQA
Copy link

SparkQA commented Aug 24, 2017

Test build #81073 has finished for PR 19036 at commit 5d0548d.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@@ -283,8 +283,8 @@ case class BroadcastHashJoinExec(
s"""
|boolean $conditionPassed = true;
|${eval.trim}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems to evaluate the variables used by condition. Should we also move it into the if branch?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is also used in L309

@viirya
Copy link
Member

viirya commented Aug 24, 2017

LGTM

1 similar comment
@hvanhovell
Copy link
Contributor

LGTM

@hvanhovell
Copy link
Contributor

Merging to master/2.2

@asfgit asfgit closed this in 2dd37d8 Aug 24, 2017
asfgit pushed a commit that referenced this pull request Aug 24, 2017
This is a bug introduced by https://github.com/apache/spark/pull/11274/files#diff-7adb688cbfa583b5711801f196a074bbL274 .

Non-equal join condition should only be applied when the equal-join condition matches.

regression test

Author: Wenchen Fan <wenchen@databricks.com>

Closes #19036 from cloud-fan/bug.

(cherry picked from commit 2dd37d8)
Signed-off-by: Herman van Hovell <hvanhovell@databricks.com>
cloud-fan added a commit to cloud-fan/spark that referenced this pull request Aug 24, 2017
This is a bug introduced by https://github.com/apache/spark/pull/11274/files#diff-7adb688cbfa583b5711801f196a074bbL274 .

Non-equal join condition should only be applied when the equal-join condition matches.

regression test

Author: Wenchen Fan <wenchen@databricks.com>

Closes apache#19036 from cloud-fan/bug.
asfgit pushed a commit that referenced this pull request Aug 24, 2017
…ow NPE

backport #19036 to branch 2.1 and 2.0

Author: Wenchen Fan <wenchen@databricks.com>

Closes #19040 from cloud-fan/bug.
asfgit pushed a commit that referenced this pull request Aug 24, 2017
…ow NPE

backport #19036 to branch 2.1 and 2.0

Author: Wenchen Fan <wenchen@databricks.com>

Closes #19040 from cloud-fan/bug.

(cherry picked from commit 5769753)
Signed-off-by: gatorsmile <gatorsmile@gmail.com>
MatthewRBruce pushed a commit to Shopify/spark that referenced this pull request Jul 31, 2018
This is a bug introduced by https://github.com/apache/spark/pull/11274/files#diff-7adb688cbfa583b5711801f196a074bbL274 .

Non-equal join condition should only be applied when the equal-join condition matches.

regression test

Author: Wenchen Fan <wenchen@databricks.com>

Closes apache#19036 from cloud-fan/bug.

(cherry picked from commit 2dd37d8)
Signed-off-by: Herman van Hovell <hvanhovell@databricks.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants