Skip to content

Commit

Permalink
Merge pull request alteryx#113 from pwendell/master
Browse files Browse the repository at this point in the history
Improve error message when multiple assembly jars are present.

This can happen easily if building different hadoop versions. Right now it gives a class not found exception.
(cherry picked from commit e018f2d)

Signed-off-by: Patrick Wendell <pwendell@gmail.com>
  • Loading branch information
pwendell committed Oct 26, 2013
1 parent 3ec2f51 commit cb24278
Showing 1 changed file with 11 additions and 4 deletions.
15 changes: 11 additions & 4 deletions spark-class
Original file line number Diff line number Diff line change
Expand Up @@ -95,10 +95,17 @@ export JAVA_OPTS

if [ ! -f "$FWDIR/RELEASE" ]; then
# Exit if the user hasn't compiled Spark
ls "$FWDIR"/assembly/target/scala-$SCALA_VERSION/spark-assembly*hadoop*.jar >& /dev/null
if [[ $? != 0 ]]; then
echo "Failed to find Spark assembly in $FWDIR/assembly/target" >&2
echo "You need to build Spark with sbt/sbt assembly before running this program" >&2
num_jars=$(ls "$FWDIR"/assembly/target/scala-$SCALA_VERSION/ | grep "spark-assembly.*hadoop.*.jar" | wc -l)
jars_list=$(ls "$FWDIR"/assembly/target/scala-$SCALA_VERSION/ | grep "spark-assembly.*hadoop.*.jar")
if [ "$num_jars" -eq "0" ]; then
echo "Failed to find Spark assembly in $FWDIR/assembly/target/scala-$SCALA_VERSION/" >&2
echo "You need to build Spark with 'sbt/sbt assembly' before running this program." >&2
exit 1
fi
if [ "$num_jars" -gt "1" ]; then
echo "Found multiple Spark assembly jars in $FWDIR/assembly/target/scala-$SCALA_VERSION:" >&2
echo "$jars_list"
echo "Please remove all but one jar."
exit 1
fi
fi
Expand Down

0 comments on commit cb24278

Please sign in to comment.