Skip to content

Commit

Permalink
[SPARK-3710] Fix Yarn integration tests on Hadoop 2.2.
Browse files Browse the repository at this point in the history
It seems some dependencies are not declared when pulling the 2.2
test dependencies, so we need to add them manually for the Yarn
cluster to come up.

These don't seem to be necessary for 2.3 and beyond, so restrict
them to the hadoop-2.2 profile.

Author: Marcelo Vanzin <vanzin@cloudera.com>

Closes #2682 from vanzin/SPARK-3710 and squashes the following commits:

701d4fb [Marcelo Vanzin] Add comment.
0540bdf [Marcelo Vanzin] [SPARK-3710] Fix Yarn integration tests on Hadoop 2.2.
  • Loading branch information
Marcelo Vanzin authored and pwendell committed Oct 8, 2014
1 parent c781843 commit 35afdfd
Showing 1 changed file with 51 additions and 0 deletions.
51 changes: 51 additions & 0 deletions yarn/stable/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -41,4 +41,55 @@
</dependency>
</dependencies>

<!--
See SPARK-3710. hadoop-yarn-server-tests in Hadoop 2.2 fails to pull some needed
dependencies, so they need to be added manually for the tests to work.
-->
<profiles>
<profile>
<id>hadoop-2.2</id>
<properties>
<jersey.version>1.9</jersey.version>
</properties>
<dependencies>
<dependency>
<groupId>org.mortbay.jetty</groupId>
<artifactId>jetty</artifactId>
<version>6.1.26</version>
<exclusions>
<exclusion>
<groupId>org.mortbay.jetty</groupId>
<artifactId>servlet-api</artifactId>
</exclusion>
</exclusions>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.sun.jersey</groupId>
<artifactId>jersey-core</artifactId>
<version>${jersey.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.sun.jersey</groupId>
<artifactId>jersey-json</artifactId>
<version>${jersey.version}</version>
<scope>test</scope>
<exclusions>
<exclusion>
<groupId>stax</groupId>
<artifactId>stax-api</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>com.sun.jersey</groupId>
<artifactId>jersey-server</artifactId>
<version>${jersey.version}</version>
<scope>test</scope>
</dependency>
</dependencies>
</profile>
</profiles>

</project>

0 comments on commit 35afdfd

Please sign in to comment.