Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-21583][HOTFIX] Removed intercept in test causing failures #19098

Conversation

BryanCutler
Copy link
Member

Removing a check in the ColumnarBatchSuite that depended on a Java assertion. This assertion is being compiled out in the Maven builds causing the test to fail. This part of the test is not specifically from to the functionality that is being tested here.

@@ -1308,10 +1308,6 @@ class ColumnarBatchSuite extends SparkFunSuite {
}
}

intercept[java.lang.AssertionError] {
batch.getRow(100)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add another test in a follow-up PR.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @gatorsmile , I'll put this in another test once I figure out why it wasn't being hit

@gatorsmile
Copy link
Member

LGTM. Merging to master.

@asfgit asfgit closed this in 501370d Aug 31, 2017
@srowen
Copy link
Member

srowen commented Aug 31, 2017

Hm, I wonder how that results in an assertion? that's a normal error case and shouldn't cause an assert. Is it from a third-party library in this case, like Arrow? really we should fix that somehow so that the user-visible contract for this behavior never involves AssertionError.

Still, assertions ought to be enabled during tests anyway, so I don't see how this doesn't actually fire. If it only affects the Maven build I'd suspect that maybe the scalatest-maven-plugin somehow doesn't turn on assertions? but it has -ea in its command line.

@BryanCutler
Copy link
Member Author

@srowen , the assertion is from the Spark ColumnarBatch here.

public ColumnarBatch.Row getRow(int rowId) {
    assert(rowId >= 0);
    assert(rowId < numRows);
    row.rowId = rowId;
    return row;
  }

I'm also not quite sure why this wasn't working, I checked and -ea is an added argument in the pom. Still, I wonder if we should change the asserts in this class to something better. Maybe these are used instead of exceptions for performance?

@srowen
Copy link
Member

srowen commented Aug 31, 2017

The asserts are wrong if this can be called from user code. It should be require. The reason is, basically, exactly this. If you don't have assertions on this argument is accepted.

@SparkQA
Copy link

SparkQA commented Aug 31, 2017

Test build #81293 has finished for PR 19098 at commit 567487d.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@BryanCutler BryanCutler deleted the hotfix-ColumnarBatchSuite-assertion branch March 6, 2018 23:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants