title: How to resolve external packages with spark-shell when behind a corporate proxy?
body: <p>I would like to run spark-shell with a external package behind a corporate proxy. Unfortunately external packages passed via <code>--packages</code> option are not resolved.</p>
  
  <p>E.g., when running</p>
  
  <pre><code>bin/spark-shell --packages datastax:spark-cassandra-connector:1.5.0-s_2.10
  </code></pre>
  
  <p>the cassandra connector package is not resolved (stuck at last line):</p>
  
  <pre><code>Ivy Default Cache set to: /root/.ivy2/cache
  The jars for the packages stored in: /root/.ivy2/jars
  :: loading settings :: url = jar:file:/opt/spark/lib/spark-assembly-1.6.1-hadoop2.6.0.jar!/org/apache/ivy/core/settings/ivysettings.xml
  datastax#spark-cassandra-connector added as a dependency
  :: resolving dependencies :: org.apache.spark#spark-submit-parent;1.0
          confs: [default]
  </code></pre>
  
  <p>After some time the connection times out containing error messages like this:</p>
  
  <pre><code>:::: ERRORS
          Server access error at url https://repo1.maven.org/maven2/datastax/spark-cassandra-connector/1.5.0-s_2.10/spark-cassandra-connector-1.5.0-s_2.10.pom (java.net.ConnectException: Connection timed out)
  </code></pre>
  
  <p>When i deactivate the VPN with the corporate proxy the package gets resolved and downloaded immediately.</p>
  
  <p>What i tried so far:</p>
  
  <p>Exposing proxies as environment variables:</p>
  
  <pre><code>export http_proxy=&lt;proxyHost&gt;:&lt;proxyPort&gt;
  export https_proxy=&lt;proxyHost&gt;:&lt;proxyPort&gt;
  export JAVA_OPTS="-Dhttp.proxyHost=&lt;proxyHost&gt; -Dhttp.proxyPort=&lt;proxyPort&gt;"
  export ANT_OPTS="-Dhttp.proxyHost=&lt;proxyHost&gt; -Dhttp.proxyPort=&lt;proxyPort&gt;"
  </code></pre>
  
  <p>Running spark-shell with extra java options:</p>
  
  <pre><code>bin/spark-shell --conf "spark.driver.extraJavaOptions=-Dhttp.proxyHost=&lt;proxyHost&gt; -Dhttp.proxyPort=&lt;proxyPort&gt;" --conf "spark.executor.extraJavaOptions=-Dhttp.proxyHost=&lt;proxyHost&gt; -Dhttp.proxyPort=&lt;proxyPort&gt;" --packages datastax:spark-cassandra-connector:1.6.0-M1-s_2.10
  </code></pre>
  
  <p>Is there some other configuration possibility i am missing?</p>