Skip to content

Commit

Permalink
Merge pull request apache#330 from tgravescs/fix_addjars_null_handling
Browse files Browse the repository at this point in the history
Fix handling of empty SPARK_EXAMPLES_JAR

Currently if SPARK_EXAMPLES_JAR is left unset you get a null pointer exception when running the examples (atleast on spark on yarn).  The null now gets turned into a string of "null" when its put into the SparkConf so addJar no longer properly ignores it. This fixes that so that it can be left unset.
  • Loading branch information
pwendell committed Jan 6, 2014
2 parents a2e7e04 + 25446dd commit 357083c
Showing 1 changed file with 3 additions and 2 deletions.
5 changes: 3 additions & 2 deletions core/src/main/scala/org/apache/spark/SparkConf.scala
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ import com.typesafe.config.ConfigFactory
*
* @param loadDefaults whether to load values from the system properties and classpath
*/
class SparkConf(loadDefaults: Boolean) extends Serializable with Cloneable {
class SparkConf(loadDefaults: Boolean) extends Serializable with Cloneable with Logging {

/** Create a SparkConf that loads defaults from system properties and the classpath */
def this() = this(true)
Expand Down Expand Up @@ -67,7 +67,8 @@ class SparkConf(loadDefaults: Boolean) extends Serializable with Cloneable {

/** Set JAR files to distribute to the cluster. */
def setJars(jars: Seq[String]): SparkConf = {
set("spark.jars", jars.mkString(","))
for (jar <- jars if (jar == null)) logWarning("null jar passed to SparkContext constructor")
set("spark.jars", jars.filter(_ != null).mkString(","))
}

/** Set JAR files to distribute to the cluster. (Java-friendly version.) */
Expand Down

0 comments on commit 357083c

Please sign in to comment.