title: Scalding job failing with VerifyError on EMR version 4.2.0
body: <p>We have a Scalding job which I want to run on the AWS Elastic MapReduce using release label 4.2.0.</p>
  
  <p>This job ran successfully on AMI 2.4.2. When we upgraded it to AMI 3.7.0, we ran into a <code>java.lang.VerifyError</code> caused by incompatible jars. Our project uses version 1.5 of the commons-codec library but earlier, incompatible versions come with the AMI. Similarly, our project uses Scala 2.10 but version 2.11 comes with the AMI. We solved this problem by adding a bootstrap script to delete all files matching <code>commons-codec-1.[234].jar</code> or <code>scala-library-2.11.*.jar</code> from the cluster.</p>
  
  <p>Now we again want to upgrade to 4.2.0, and again get a VerifyError:</p>
  
  <pre>
  ```
  Exception in thread "main" java.lang.reflect.InvocationTargetException
      at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
      at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
      at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
      at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
      at com.twitter.scalding.Job$.apply(Job.scala:47)
      at com.twitter.scalding.Tool.getJob(Tool.scala:48)
      at com.twitter.scalding.Tool.run(Tool.scala:68)
      at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
      at com.snowplowanalytics.snowplow.enrich.hadoop.JobRunner$.main(JobRunner.scala:33)
      at com.snowplowanalytics.snowplow.enrich.hadoop.JobRunner.main(JobRunner.scala)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      at java.lang.reflect.Method.invoke(Method.java:606)
      at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
      at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
  Caused by: java.lang.VerifyError: Bad type on operand stack
  Exception Details:
    Location:
      com/snowplowanalytics/snowplow/enrich/common/utils/ConversionUtils$.decodeBase64Url(Ljava/lang/String;Ljava/lang/String;)Lscalaz/Validation; @5: invokevirtual
    Reason:
      Type 'org/apache/commons/codec/binary/Base64' (current frame, stack[0]) is not assignable to 'org/apache/commons/codec/binary/BaseNCodec'
    Current Frame:
      bci: @5
      flags: { }
      locals: { 'com/snowplowanalytics/snowplow/enrich/common/utils/ConversionUtils$', 'java/lang/String', 'java/lang/String' }
      stack: { 'org/apache/commons/codec/binary/Base64', 'java/lang/String' }
    Bytecode:
      0000000: 2ab7 008a 2cb6 0090 3a04 bb00 5459 1904
      0000010: b200 96b7 0099 3a05 b200 9e19 05b9 00a4
      0000020: 0200 b900 aa01 00a7 003e 4eb2 009e bb00
      0000030: ac59 b200 4112 aeb6 00b1 b700 b4b2 0041
      0000040: 06bd 0004 5903 2b53 5904 2c53 5905 2db6
      0000050: 00b9 53b6 00bf b900 c502 00b9 00a4 0200
      0000060: b900 c801 00b0                         
    Exception Handler Table:
      bci [0, 42] => handler: 42
    Stackmap Table:
      same_locals_1_stack_item_frame(@42,Object[#182])
      same_locals_1_stack_item_frame(@101,Object[#206])
  
      at com.snowplowanalytics.snowplow.enrich.hadoop.EtlJobConfig$.com$snowplowanalytics$snowplow$enrich$hadoop$EtlJobConfig$$base64ToJsonNode(EtlJobConfig.scala:224)
      at com.snowplowanalytics.snowplow.enrich.hadoop.EtlJobConfig$.loadConfigAndFilesToCache(EtlJobConfig.scala:126)
      at com.snowplowanalytics.snowplow.enrich.hadoop.EtlJob.(EtlJob.scala:139)
      ... 16 more
  ```
  </pre>
  
  <p>Exploring which jars remain on the cluster following the purge:</p>
  
  <pre><code>$ sudo find / -name "*scala-*"
  /usr/share/aws/emr/emrfs/cli/lib/scala-library-2.10.5.jar
  /usr/share/aws/emr/emrfs/cli/lib/scala-reflect-2.10.4.jar
  /usr/share/aws/emr/emrfs/cli/lib/scala-logging-api_2.10-2.1.2.jar
  /usr/share/aws/emr/emrfs/cli/lib/nscala-time_2.10-1.2.0.jar
  /usr/share/aws/emr/emrfs/cli/lib/scala-logging-slf4j_2.10-2.1.2.jar
  $ sudo find / -name "*commons-codec*"
  /usr/share/aws/emr/node-provisioner/lib/commons-codec-1.9.jar
  /usr/share/aws/emr/emr-metrics/lib/commons-codec-1.6.jar
  /usr/share/aws/emr/emr-metrics-client/lib/commons-codec-1.6.jar
  /usr/share/aws/emr/emrfs/lib/commons-codec-1.9.jar
  /usr/share/aws/emr/hadoop-state-pusher/lib/commons-codec-1.8.jar
  /usr/lib/hbase/lib/commons-codec-1.7.jar
  /usr/lib/mahout/lib/commons-codec-1.7.jar
  </code></pre>
  
  <p>The same error occurs with AMI 4.1.0. What has changed between 3.7.0 and 4.x.x which is causing this issue, and what can I do to fix it?</p>