Skip to content

Commit

Permalink
[SPARK-32804][LAUNCHER][FOLLOWUP] Fix SparkSubmitCommandBuilderSuite …
Browse files Browse the repository at this point in the history
…test failure without jars

### What changes were proposed in this pull request?

It's a followup of #29653.
Tests in `SparkSubmitCommandBuilderSuite` may fail if you didn't build first and have jars before test,
so if `isTesting` we should set a dummy `SparkLauncher.NO_RESOURCE`.

### Why are the changes needed?

Fix tests failure.

### Does this PR introduce _any_ user-facing change?

No.

### How was this patch tested?

mvn clean test (test without jars built first).

Closes #29769 from KevinSmile/bug-fix-master.

Authored-by: KevinSmile <kevinwang013@hotmail.com>
Signed-off-by: HyukjinKwon <gurwls223@apache.org>
  • Loading branch information
KevinSmile authored and HyukjinKwon committed Sep 16, 2020
1 parent 3bc13e6 commit 355ab6a
Show file tree
Hide file tree
Showing 2 changed files with 11 additions and 6 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -404,12 +404,17 @@ private boolean isThriftServer(String mainClass) {
}

private String findExamplesAppJar() {
for (String exampleJar : findExamplesJars()) {
if (new File(exampleJar).getName().startsWith("spark-examples")) {
return exampleJar;
boolean isTesting = "1".equals(getenv("SPARK_TESTING"));
if (isTesting) {
return SparkLauncher.NO_RESOURCE;
} else {
for (String exampleJar : findExamplesJars()) {
if (new File(exampleJar).getName().startsWith("spark-examples")) {
return exampleJar;
}
}
throw new IllegalStateException("Failed to find examples' main app jar.");
}
throw new IllegalStateException("Failed to find examples' main app jar.");
}

private List<String> findExamplesJars() {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -259,8 +259,8 @@ public void testExamplesRunnerPrimaryResource() throws Exception {
findArgValue(cmd, parser.CLASS));
assertEquals("cluster", findArgValue(cmd, parser.DEPLOY_MODE));
String primaryResource = cmd.get(cmd.size() - 2);
assertTrue(new File(primaryResource).getName().startsWith("spark-examples"));
assertFalse(cmd.contains(SparkLauncher.NO_RESOURCE));
assertTrue(primaryResource.equals(SparkLauncher.NO_RESOURCE)
|| new File(primaryResource).getName().startsWith("spark-examples"));
}

@Test(expected = IllegalArgumentException.class)
Expand Down

0 comments on commit 355ab6a

Please sign in to comment.