{"action":"created","issue":{"url":"https://api.github.com/repos/druid-io/druid/issues/4776","repository_url":"https://api.github.com/repos/druid-io/druid","labels_url":"https://api.github.com/repos/druid-io/druid/issues/4776/labels{/name}","comments_url":"https://api.github.com/repos/druid-io/druid/issues/4776/comments","events_url":"https://api.github.com/repos/druid-io/druid/issues/4776/events","html_url":"https://github.com/druid-io/druid/pull/4776","id":256678765,"number":4776,"title":"Fix ClassNotFoundException in druid-kerberos extension","user":{"login":"shiroari","id":11080799,"avatar_url":"https://avatars0.githubusercontent.com/u/11080799?v=4","gravatar_id":"","url":"https://api.github.com/users/shiroari","html_url":"https://github.com/shiroari","followers_url":"https://api.github.com/users/shiroari/followers","following_url":"https://api.github.com/users/shiroari/following{/other_user}","gists_url":"https://api.github.com/users/shiroari/gists{/gist_id}","starred_url":"https://api.github.com/users/shiroari/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/shiroari/subscriptions","organizations_url":"https://api.github.com/users/shiroari/orgs","repos_url":"https://api.github.com/users/shiroari/repos","events_url":"https://api.github.com/users/shiroari/events{/privacy}","received_events_url":"https://api.github.com/users/shiroari/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2017-09-11T12:05:43Z","updated_at":"2017-09-20T19:26:49Z","closed_at":null,"author_association":"NONE","pull_request":{"url":"https://api.github.com/repos/druid-io/druid/pulls/4776","html_url":"https://github.com/druid-io/druid/pull/4776","diff_url":"https://github.com/druid-io/druid/pull/4776.diff","patch_url":"https://github.com/druid-io/druid/pull/4776.patch"},"body":"Hi,\r\n\r\nHere is a one-line fix for `druid-kerberos` extension. There is a problem with class loader for hadoop `Configuration` class. By default, configuration uses the context class loader of the current thread set to the class loader used to load the application. But application class loader doesn't contain any dependencies due to isolation. As a result we can not load any class located in \r\n`hadoop-common` (it contains the most important implementations for configuring hadoop).\r\n\r\nThere are 2 solutions: use `Configuration` class loader or get class loader of any class in extension. I decided to use module class to make more explicit the fact that we use extension class loader.\r\n\r\nSome stacktrace for reference:\r\n\r\n```\r\n2017-09-07T13:16:58,928 INFO [main] io.druid.indexing.common.actions.RemoteTaskActionClient - Submitting action for task[index_hadoop_task_2017-09-07T13:16:52.833Z] to overlord[http://host:8090/druid/indexer/v1/action]: LockTryAcquireAction{interval=2017-08-09T00:00:00.000Z/2017-08-11T00:00:00.000Z}\r\n2017-09-07T13:16:59,011 ERROR [main] io.druid.cli.CliPeon - Error when starting up.  Failing.\r\njava.lang.reflect.InvocationTargetException\r\n                at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_121]\r\n                at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_121]\r\n                at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_121]\r\n                at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_121]\r\n                at com.metamx.common.lifecycle.Lifecycle$AnnotationBasedHandler.start(Lifecycle.java:350) ~[java-util-0.27.10.jar:?]\r\n                at com.metamx.common.lifecycle.Lifecycle.start(Lifecycle.java:259) ~[java-util-0.27.10.jar:?]\r\n                at io.druid.guice.LifecycleModule$2.start(LifecycleModule.java:155) ~[druid-api-0.9.2.2.6.1.0-129.jar:0.9.2.2.6.1.0-129]\r\n                at io.druid.cli.GuiceRunnable.initLifecycle(GuiceRunnable.java:101) [druid-services-0.9.2.2.6.1.0-129.jar:0.9.2.2.6.1.0-129]\r\n                at io.druid.cli.CliPeon.run(CliPeon.java:274) [druid-services-0.9.2.2.6.1.0-129.jar:0.9.2.2.6.1.0-129]\r\n                at io.druid.cli.Main.main(Main.java:106) [druid-services-0.9.2.2.6.1.0-129.jar:0.9.2.2.6.1.0-129]\r\nCaused by: com.metamx.common.ISE: Failed to run isReady\r\n                at io.druid.indexing.worker.executor.ExecutorLifecycle.start(ExecutorLifecycle.java:173) ~[druid-indexing-service-0.9.2.2.6.1.0-129.jar:0.9.2.2.6.1.0-129]\r\n                ... 10 more\r\nCaused by: java.lang.RuntimeException: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.hadoop.security.CompositeGroupsMapping not found\r\n                at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2273) ~[?:?]\r\n                at org.apache.hadoop.security.Groups.<init>(Groups.java:99) ~[?:?]\r\n                at org.apache.hadoop.security.Groups.<init>(Groups.java:95) ~[?:?]\r\n                at org.apache.hadoop.security.Groups.getUserToGroupsMappingService(Groups.java:420) ~[?:?]\r\n                at org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:324) ~[?:?]\r\n                at org.apache.hadoop.security.UserGroupInformation.setConfiguration(UserGroupInformation.java:352) ~[?:?]\r\n                at io.druid.security.kerberos.DruidKerberosUtil.authenticateIfRequired(DruidKerberosUtil.java:101) ~[?:?]\r\n                at io.druid.security.kerberos.KerberosHttpClient.inner_go(KerberosHttpClient.java:97) ~[?:?]\r\n                at io.druid.security.kerberos.KerberosHttpClient.go(KerberosHttpClient.java:67) ~[?:?]\r\n                at com.metamx.http.client.AbstractHttpClient.go(AbstractHttpClient.java:14) ~[http-client-1.0.4.jar:?]\r\n                at io.druid.indexing.common.actions.RemoteTaskActionClient.submit(RemoteTaskActionClient.java:101) ~[druid-indexing-service-0.9.2.2.6.1.0-129.jar:0.9.2.2.6.1.0-129]\r\n                at io.druid.indexing.common.task.HadoopIndexTask.isReady(HadoopIndexTask.java:137) ~[druid-indexing-service-0.9.2.2.6.1.0-129.jar:0.9.2.2.6.1.0-129]\r\n                at io.druid.indexing.worker.executor.ExecutorLifecycle.start(ExecutorLifecycle.java:168) ~[druid-indexing-service-0.9.2.2.6.1.0-129.jar:0.9.2.2.6.1.0-129]\r\n                ... 10 more\r\nCaused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.hadoop.security.CompositeGroupsMapping not found\r\n                at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2241) ~[?:?]\r\n                at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2265) ~[?:?]\r\n                at org.apache.hadoop.security.Groups.<init>(Groups.java:99) ~[?:?]\r\n                at org.apache.hadoop.security.Groups.<init>(Groups.java:95) ~[?:?]\r\n                at org.apache.hadoop.security.Groups.getUserToGroupsMappingService(Groups.java:420) ~[?:?]\r\n                at org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:324) ~[?:?]\r\n                at org.apache.hadoop.security.UserGroupInformation.setConfiguration(UserGroupInformation.java:352) ~[?:?]\r\n                at io.druid.security.kerberos.DruidKerberosUtil.authenticateIfRequired(DruidKerberosUtil.java:101) ~[?:?]\r\n                at io.druid.security.kerberos.KerberosHttpClient.inner_go(KerberosHttpClient.java:97) ~[?:?]\r\n                at io.druid.security.kerberos.KerberosHttpClient.go(KerberosHttpClient.java:67) ~[?:?]\r\n                at com.metamx.http.client.AbstractHttpClient.go(AbstractHttpClient.java:14) ~[http-client-1.0.4.jar:?]\r\n                at io.druid.indexing.common.actions.RemoteTaskActionClient.submit(RemoteTaskActionClient.java:101) ~[druid-indexing-service-0.9.2.2.6.1.0-129.jar:0.9.2.2.6.1.0-129]\r\n                at io.druid.indexing.common.task.HadoopIndexTask.isReady(HadoopIndexTask.java:137) ~[druid-indexing-service-0.9.2.2.6.1.0-129.jar:0.9.2.2.6.1.0-129]\r\n                at io.druid.indexing.worker.executor.ExecutorLifecycle.start(ExecutorLifecycle.java:168) ~[druid-indexing-service-0.9.2.2.6.1.0-129.jar:0.9.2.2.6.1.0-129]\r\n                ... 10 more\r\nCaused by: java.lang.ClassNotFoundException: Class org.apache.hadoop.security.CompositeGroupsMapping not found\r\n                at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2147) ~[?:?]\r\n                at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2239) ~[?:?]\r\n                at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2265) ~[?:?]\r\n                at org.apache.hadoop.security.Groups.<init>(Groups.java:99) ~[?:?]\r\n                at org.apache.hadoop.security.Groups.<init>(Groups.java:95) ~[?:?]\r\n                at org.apache.hadoop.security.Groups.getUserToGroupsMappingService(Groups.java:420) ~[?:?]\r\n                at org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:324) ~[?:?]\r\n                at org.apache.hadoop.security.UserGroupInformation.setConfiguration(UserGroupInformation.java:352) ~[?:?]\r\n                at io.druid.security.kerberos.DruidKerberosUtil.authenticateIfRequired(DruidKerberosUtil.java:101) ~[?:?]\r\n                at io.druid.security.kerberos.KerberosHttpClient.inner_go(KerberosHttpClient.java:97) ~[?:?]\r\n                at io.druid.security.kerberos.KerberosHttpClient.go(KerberosHttpClient.java:67) ~[?:?]\r\n                at com.metamx.http.client.AbstractHttpClient.go(AbstractHttpClient.java:14) ~[http-client-1.0.4.jar:?]\r\n                at io.druid.indexing.common.actions.RemoteTaskActionClient.submit(RemoteTaskActionClient.java:101) ~[druid-indexing-service-0.9.2.2.6.1.0-129.jar:0.9.2.2.6.1.0-129]\r\n                at io.druid.indexing.common.task.HadoopIndexTask.isReady(HadoopIndexTask.java:137) ~[druid-indexing-service-0.9.2.2.6.1.0-129.jar:0.9.2.2.6.1.0-129]\r\n                at io.druid.indexing.worker.executor.ExecutorLifecycle.start(ExecutorLifecycle.java:168) ~[druid-indexing-service-0.9.2.2.6.1.0-129.jar:0.9.2.2.6.1.0-129]\r\n                ... 10 more\r\n```\r\n"},"comment":{"url":"https://api.github.com/repos/druid-io/druid/issues/comments/330955636","html_url":"https://github.com/druid-io/druid/pull/4776#issuecomment-330955636","issue_url":"https://api.github.com/repos/druid-io/druid/issues/4776","id":330955636,"user":{"login":"drcrallen","id":8213081,"avatar_url":"https://avatars2.githubusercontent.com/u/8213081?v=4","gravatar_id":"","url":"https://api.github.com/users/drcrallen","html_url":"https://github.com/drcrallen","followers_url":"https://api.github.com/users/drcrallen/followers","following_url":"https://api.github.com/users/drcrallen/following{/other_user}","gists_url":"https://api.github.com/users/drcrallen/gists{/gist_id}","starred_url":"https://api.github.com/users/drcrallen/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/drcrallen/subscriptions","organizations_url":"https://api.github.com/users/drcrallen/orgs","repos_url":"https://api.github.com/users/drcrallen/repos","events_url":"https://api.github.com/users/drcrallen/events{/privacy}","received_events_url":"https://api.github.com/users/drcrallen/received_events","type":"User","site_admin":false},"created_at":"2017-09-20T19:26:49Z","updated_at":"2017-09-20T19:26:49Z","author_association":"OWNER","body":"@himanshug or @nishantmonu51 can one of you guys look at this one?"}}
{"actor":{"display_login":"drcrallen"}}