diff --git a/docs/building-with-maven.md b/docs/building-with-maven.md index a7d7bd3ccb1f2..ec3831896f0d6 100644 --- a/docs/building-with-maven.md +++ b/docs/building-with-maven.md @@ -156,4 +156,23 @@ then ship it over to the cluster. We are investigating the exact cause for this. The assembly jar produced by `mvn package` will, by default, include all of Spark's dependencies, including Hadoop and some of its ecosystem projects. On YARN deployments, this causes multiple versions of these to appear on executor classpaths: the version packaged in the Spark assembly and the version on each node, included with yarn.application.classpath. The `hadoop-provided` profile builds the assembly without including Hadoop-ecosystem projects, like ZooKeeper and Hadoop itself. +# Building under http-proxy environment +Sometimes,spark is built in http-proxy environment. We recommend the following settings: + +
export MAVEN_OPTS="-Dmaven.wagon.http.ssl.insecure=true -Dmaven.wagon.http.ssl.allowall=true"If you don’t run this, you may see warnings like the following:
+ +Downloading: https://repo.maven.apache.org/maven2/org/apache/hadoop/hadoop-client/1.0.4/hadoop-client-1.0.4.pom
+Aug 27, 2014 5:08:21 PM org.apache.maven.wagon.providers.http.httpclient.client.protocol.RequestAuthenticationBase process
+WARNING: NTLM authentication error: Credentials cannot be used for NTLM authentication: org.apache.maven.wagon.providers.http.httpclient.auth.UsernamePasswordCredentials
+Downloading: https://repository.apache.org/content/repositories/releases/org/apache/hadoop/hadoop-client/1.0.4/hadoop-client-1.0.4.pom
+Aug 27, 2014 5:08:21 PM org.apache.maven.wagon.providers.http.httpclient.client.protocol.RequestAuthenticationBase process
+WARNING: NTLM authentication error: Credentials cannot be used for NTLM authentication: org.apache.maven.wagon.providers.http.httpclient.auth.UsernamePasswordCredentials
+
+
+You can fix this by setting the MAVEN_OPTS variable as discussed before.
Note: the setting could lead to security risks of your Maven server.