title: Register custom URLStreamHandler in Spring web application (Tomcat)
body: <p>I'm trying to register a custom <code>URLStreamHandler</code> to handle requests to Amazon S3 URLs in a generic fashion. The implementation of the handler looks pretty much like <a href="https://github.com/jenshadlich/S3-URLStreamHandler" rel="nofollow noreferrer">S3-URLStreamHandler (github)</a>. I did not put my <code>Handler</code> class into the <code>sun.net.www.protocol.s3</code> package, but used the custom package <code>com.github.dpr.protocol.s3</code>. To make Java pickup this package, I provided the system property <code>-Djava.protocol.handler.pkgs=&quot;com.github.dpr.protocol&quot;</code> following the documentation of the <a href="https://docs.oracle.com/javase/7/docs/api/java/net/URL.html" rel="nofollow noreferrer">URL class</a>. However, if I try to process a s3-URL like <code>s3://my.bucket/some-awesome-file.txt</code>, I get a <code>MalformedURLException</code>:</p>
  <pre><code>Caused by: java.net.MalformedURLException: unknown protocol: s3
      at java.net.URL.&lt;init&gt;(URL.java:600)
      at java.net.URL.&lt;init&gt;(URL.java:490)
      at java.net.URL.&lt;init&gt;(URL.java:439)
      at java.net.URI.toURL(URI.java:1089)
      ...
  </code></pre>
  <p>My application is a Spring based web application, that currently runs in tomcat, but should not be cluttered with any knowledge about the underlying application container.</p>
  <p>I already debugged the respective code and found that my <code>URLStreamHandler</code> can't be initialized as the classloader used to load the class doesn't know it. This is the respective code from <code>java.net.URL</code> (jdk 1.8.0_92):</p>
  <pre><code>1174: try {
  1175:   cls = Class.forName(clsName);
  1176: } catch (ClassNotFoundException e) {
  1177:   ClassLoader cl = ClassLoader.getSystemClassLoader();
  1178:   if (cl != null) {
  1179:     cls = cl.loadClass(clsName);
  1180:   }
  1181: }
  </code></pre>
  <p>The class loader of the <code>java.net.URL</code> class (used by <code>Class.forName</code>) is <code>null</code> indicating the bootstrap class loader and the system class loader doesn't know my class. If I put a break point in there and try to load my handler class using the current thread's class loader, it works fine. That is my class is apparently there and is in the application's class path, but Java uses the &quot;wrong&quot; class loaders to lookup the handler.</p>
  <p>I'm aware of <a href="https://stackoverflow.com/questions/26363573/registering-and-using-a-custom-java-net-url-protocol">this question on SOF</a>, but I must not register a custom <code>URLStreamHandlerFactory</code> as tomcat registers it's own factory (<code>org.apache.naming.resources.DirContextURLStreamHandlerFactory</code>) on application start and there must only be one factory registered for a single JVM. Tomcat's <code>DirContextURLStreamHandlerFactory</code> allows to add user factories, that could be used to handle custom protocols, but I don't want to add a dependency to Tomcat in my application code, as the application should run in different containers.</p>
  <p>Is there any way to register a handler for custom URLs in a container independent fashion?</p>
  <hr />
  <p><strong>UPDATE 2017-01-25:</strong><br />
  I gave the different options @Nicolas Filotto proposed a try:</p>
  <p><em>Option 1 - Set custom URLStreamHandlerFactory using reflection</em><br />
  This option works as expected. But by using reflection it introduces a tight dependency to the inner workings of the <code>java.net.URL</code> class. Luckily Oracle is not too eager with fixing convenience issues in the basic java classes - actually <a href="http://bugs.java.com/bugdatabase/view_bug.do?bug_id=4648098" rel="nofollow noreferrer">this related bug report</a> is open for almost <strong>14 years</strong> (great work Sun/Oracle) -  and this can easily be unit tested.</p>
  <p><em>Option 2 - Put handler JAR into {JAVA_HOME}/jre/lib/ext</em><br />
  This option works as well. But only adding the handler jar as system extension won't do the trick - of course. One will need to add all dependencies of the handler as well. As these classes are visible to all applications using this Java installation, this may lead to undesired effects due to different versions of the same library in the classpath.</p>
  <p><em>Option 3 - Put handler JAR into {CATALINA_HOME}/lib</em><br />
  This doesn't work. According to <a href="http://tomcat.apache.org/tomcat-7.0-doc/class-loader-howto.html" rel="nofollow noreferrer">Tomcat's classloader documentation</a> resources put into this directory will be loaded using Tomcat's <em>Common</em> classloader. This classloader will not be used by <code>java.net.URL</code> to lookup the protocol handler.</p>
  <p>Given these options I'll stay with the reflection variant. All of the options are not very nice, but at least the first one can easily be tested and does not require any deployment logic. However I adapted the code slightly to use the same lock object for synchronization as <code>java.net.URL</code> does:</p>
  <pre><code>public static void registerFactory() throws Exception  {
    final Field factoryField = URL.class.getDeclaredField(&quot;factory&quot;);
    factoryField.setAccessible(true);
    final Field lockField = URL.class.getDeclaredField(&quot;streamHandlerLock&quot;);
    lockField.setAccessible(true);
    
    // use same lock as in java.net.URL.setURLStreamHandlerFactory
    synchronized (lockField.get(null)) {
      final URLStreamHandlerFactory urlStreamHandlerFactory = (URLStreamHandlerFactory) factoryField.get(null);
      // Reset the value to prevent Error due to a factory already defined
      factoryField.set(null, null);
      URL.setURLStreamHandlerFactory(new AmazonS3UrlStreamHandlerFactory(urlStreamHandlerFactory));
    }
  }
  </code></pre>