SPARK-2879 [BUILD] Use HTTPS to access Maven Central and other repos

[srowen]

Maven Central has just now enabled HTTPS access for everyone to Maven Central (http://central.sonatype.org/articles/2014/Aug/03/https-support-launching-now/) This is timely, as a reminder of how easily an attacker can slip malicious code into a build that's downloading artifacts over HTTP (http://blog.ontoillogical.com/blog/2014/07/28/how-to-take-over-any-java-developer/).

In the meantime, it looks like the Spring repo also now supports HTTPS, so can be used this way too.

I propose to use HTTPS to access these repos.

[SparkQA]

QA tests have started for PR 1805. This patch merges cleanly.
View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18016/consoleFull

[srowen]

Jenkins, retest this please.

[SparkQA]

QA tests have started for PR 1805. This patch merges cleanly.
View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18025/consoleFull

[SparkQA]

QA results for PR 1805:
- This patch PASSES unit tests.
- This patch merges cleanly
- This patch adds no public classes

For more information see test ouptut:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18025/consoleFull

[rxin]

any reason we call apache maven "central"? (the old name is confusing too)

[srowen]

The default repo that everyone inherits in any Maven build is Sonatype's repo, which has just been called "Maven Central" for as long as I can remember: http://search.maven.org/ It's not an Apache repo.

The reason I changed the name is that its ID in the default Maven parent pom is "central". Right now, it's not actually overriding the default. Maven Central repo is included twice in the list of repos, which does very little harm except to cost a duplicate check to Maven Central when an artifact isn't found.

Still, it seemed more reasonable to actually override it as intended. I suppose that otherwise, you'd be leaking your (failed) requests for artifacts even after this change to secure these requests, although that's very minor.

Anyway that's why I changed it to "central", since that's its ID in the default Maven parent.

[pwendell]

Thanks @srowen - for a long time we had users periodically submitting PR's to change this to https and then someone would submit a PR to change it back because it didn't work for them. I think previously maven central had limited support for this in parts of the mirror network, so hopefully now it works everywhere.

[pwendell]

Okay I'm merging this.

[pwendell]

@srowen I just got this error in Maven when trying to package a release. I'm going to retry this, but wondering if it's related.

Failed to execute goal org.apache.avro:avro-maven-plugin:1.7.3:idl-protocol (default) on project 
spark-streaming-flume-sink_2.10: Execution default of goal 
org.apache.avro:avro-maven-plugin:1.7.3:idl-protocol failed: Plugin 
org.apache.avro:avro-maven-plugin:1.7.3 or one of its dependencies could not be resolved: 
Could not transfer artifact com.thoughtworks.paranamer:paranamer:jar:2.3 
from/to central (https://repo1.maven.org/maven2): hostname in certificate 
didn't match: <repo1.maven.org> != <repo.maven.apache.org> OR <repo.maven.apache.org> -> [Help 1]

[pwendell]

@srowen so I just re-ran the build and it worked... maybe this is a transient problem

[aarondav]

Just wait for the posts on the user list...

[srowen]

@pwendell You're right that actually repo1.maven.org is canonical (http://central.stage.sonatype.org/pages/consumers.html) I'll send another small PR to touch that up, and one other small thing.