Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[hotfix] [build] Make sure JAVA_HOME is set for tests. #5441

Closed
wants to merge 3 commits into from
Closed

[hotfix] [build] Make sure JAVA_HOME is set for tests. #5441

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
6bda399
WIP: Testing to see whether this fixes the yarn test issue.
Apr 9, 2015
d71f1bb
And sbt too.
Apr 9, 2015
3eeec30
Use JAVA_HOME when available, java.home otherwise.
Apr 9, 2015
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1194,6 +1194,7 @@
launched by the tests have access to the correct test-time classpath.
-->
<SPARK_DIST_CLASSPATH>${test_classpath}</SPARK_DIST_CLASSPATH>
<JAVA_HOME>${java.home}</JAVA_HOME>
</environmentVariables>
<systemProperties>
<java.awt.headless>true</java.awt.headless>
Expand Down Expand Up @@ -1224,6 +1225,7 @@
launched by the tests have access to the correct test-time classpath.
-->
<SPARK_DIST_CLASSPATH>${test_classpath}</SPARK_DIST_CLASSPATH>
<JAVA_HOME>${java.home}</JAVA_HOME>
</environmentVariables>
<systemProperties>
<java.awt.headless>true</java.awt.headless>
Expand Down
8 changes: 5 additions & 3 deletions project/SparkBuild.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -119,7 +119,7 @@ object SparkBuild extends PomBuild {
lazy val publishLocalBoth = TaskKey[Unit]("publish-local", "publish local for m2 and ivy")

lazy val sharedSettings = graphSettings ++ genjavadocSettings ++ Seq (
javaHome := Properties.envOrNone("JAVA_HOME").map(file),
javaHome := sys.props.get("java.home").map(file),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm testing out this patch on one of the Jenkins boxes and it looks like this ended up breaking compilation because java.home points to a JRE, which will cause us to fail with an error when SBT tries to shell out to javac:

java.io.IOException: Cannot run program "/usr/java/jdk1.7.0_71/jre/bin/javac": error=2, No such file or directory
    at java.lang.ProcessBuilder.start(ProcessBuilder.java:1047)
    at sbt.SimpleProcessBuilder.run(ProcessImpl.scala:349)
    at sbt.AbstractProcessBuilder.run(ProcessImpl.scala:128)
    at sbt.AbstractProcessBuilder$$anonfun$runBuffered$1.apply(ProcessImpl.scala:159)
    at sbt.AbstractProcessBuilder$$anonfun$runBuffered$1.apply(ProcessImpl.scala:159)
    at sbt.compiler.JavacLogger.buffer(AggressiveCompile.scala:235)
    at sbt.AbstractProcessBuilder.runBuffered(ProcessImpl.scala:159)
    at sbt.AbstractProcessBuilder.$bang(ProcessImpl.scala:156)
[...]

incOptions := incOptions.value.withNameHashing(true),
retrieveManaged := true,
retrievePattern := "[type]s/[artifact](-[revision])(-[classifier]).[ext]",
Expand Down Expand Up @@ -426,8 +426,10 @@ object TestSettings {
fork := true,
// Setting SPARK_DIST_CLASSPATH is a simple way to make sure any child processes
// launched by the tests have access to the correct test-time classpath.
envVars in Test += ("SPARK_DIST_CLASSPATH" ->
(fullClasspath in Test).value.files.map(_.getAbsolutePath).mkString(":").stripSuffix(":")),
envVars in Test ++= Map(
"SPARK_DIST_CLASSPATH" ->
(fullClasspath in Test).value.files.map(_.getAbsolutePath).mkString(":").stripSuffix(":"),
"JAVA_HOME" -> sys.props("java.home")),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If JAVA_HOME was actually set by the user who invoked SBT (or Maven), then I think this change (and the Maven change) will cause us to ignore their preferred JRE in favor of the JRE used to launch the build tool, which might be confusing.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let me fix that. In maven, though, that's probably harder...

javaOptions in Test += "-Dspark.test.home=" + sparkHome,
javaOptions in Test += "-Dspark.testing=1",
javaOptions in Test += "-Dspark.port.maxRetries=100",
Expand Down