[SPARK-3710] Fix Yarn integration tests on Hadoop 2.2.

[vanzin]

It seems some dependencies are not declared when pulling the 2.2
test dependencies, so we need to add them manually for the Yarn
cluster to come up.

These don't seem to be necessary for 2.3 and beyond, so restrict
them to the hadoop-2.2 profile.

[AmplabJenkins]

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21342/Test FAILed.

[vanzin]

Jenkins, retest this please.

[srowen]

Is it the Hadoop 2.2 test code that needs these things, or Spark test code? I am guessing it is the former, and this is because test deps aren't transitive but happen to get covered by Hadoop 2.3 non-test transitive dependencies otherwise?

[SparkQA]

QA tests have started for PR 2682 at commit 0540bdf.

• This patch merges cleanly.

[vanzin]

Yes, it's the hadoop side that's missing those. They were added in the 2.3 release, so this is just needed when running against 2.2. I haven't looked at details of the hadoop poms, though.

[SparkQA]

QA tests have finished for PR 2682 at commit 0540bdf.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21343/Test PASSed.

[pwendell]

Would you mind adding a comment here with the JIRA number in there and explaining these are for YARN integration tests?

[SparkQA]

QA tests have started for PR 2682 at commit 701d4fb.

• This patch merges cleanly.

[SparkQA]

Tests timed out for PR 2682 at commit 701d4fb after a configured wait of 120m.

[AmplabJenkins]

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21384/Test FAILed.

[vanzin]

Failed some Spark SQL test, flaky?

[vanzin]

Jenkins, retest this please.

[AmplabJenkins]

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21393/Test FAILed.

[vanzin]

And now, infrastructure issues?

[vanzin]

Jenkins, retest this please.

[SparkQA]

QA tests have started for PR 2682 at commit 701d4fb.

• This patch merges cleanly.

[SparkQA]

QA tests have finished for PR 2682 at commit 701d4fb.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21395/Test PASSed.

[pwendell]

Great, let's take it for a spin.