{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/17582","repository_url":"https://api.github.com/repos/apache/spark","labels_url":"https://api.github.com/repos/apache/spark/issues/17582/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/17582/comments","events_url":"https://api.github.com/repos/apache/spark/issues/17582/events","html_url":"https://github.com/apache/spark/pull/17582","id":220527512,"number":17582,"title":"[SPARK-20239][Core] Improve HistoryServer's ACL mechanism","user":{"login":"jerryshao","id":850797,"avatar_url":"https://avatars1.githubusercontent.com/u/850797?v=3","gravatar_id":"","url":"https://api.github.com/users/jerryshao","html_url":"https://github.com/jerryshao","followers_url":"https://api.github.com/users/jerryshao/followers","following_url":"https://api.github.com/users/jerryshao/following{/other_user}","gists_url":"https://api.github.com/users/jerryshao/gists{/gist_id}","starred_url":"https://api.github.com/users/jerryshao/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jerryshao/subscriptions","organizations_url":"https://api.github.com/users/jerryshao/orgs","repos_url":"https://api.github.com/users/jerryshao/repos","events_url":"https://api.github.com/users/jerryshao/events{/privacy}","received_events_url":"https://api.github.com/users/jerryshao/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":19,"created_at":"2017-04-10T03:16:50Z","updated_at":"2017-04-20T20:27:37Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/17582","html_url":"https://github.com/apache/spark/pull/17582","diff_url":"https://github.com/apache/spark/pull/17582.diff","patch_url":"https://github.com/apache/spark/pull/17582.patch"},"body":"## What changes were proposed in this pull request?\r\n\r\nCurrent SHS (Spark History Server) two different ACLs:\r\n\r\n* ACL of base URL, it is controlled by \"spark.acls.enabled\" or \"spark.ui.acls.enabled\", and with this enabled, only user configured with \"spark.admin.acls\" (or group) or \"spark.ui.view.acls\" (or group), or the user who started SHS could list all the applications, otherwise none of them can be listed. This will also affect REST APIs which listing the summary of all apps and one app.\r\n* Per application ACL. This is controlled by \"spark.history.ui.acls.enabled\". With this enabled only history admin user and user/group who ran this app can access the details of this app.\r\n\r\nWith this two ACLs, we may encounter several unexpected behaviors:\r\n\r\n1. if base URL's ACL (`spark.acls.enable`) is enabled but user A has no view permission. User \"A\" cannot see the app list but could still access details of it's own app.\r\n2. if ACLs of base URL (`spark.acls.enable`) is disabled. Then user \"A\" could see the summary of all the apps, even some apps didn't run by user \"A\", but can only access its own app's details.\r\n3. if ACLs of base URL (`spark.acls.enable`) is disabled, then user \"A\" could download any application's event log, even it is not run by user \"A\".\r\n4. history admin ACL (if it is not also configured in `spark.admin.acls`) has no permission to list all apps.\r\n4. The changes of Live UI's ACL will affect History UI's ACL which share the same conf file.\r\n\r\nThe unexpected behaviors is mainly because we have two different ACLs, ideally we should have only one to manage all.\r\n\r\nSo to improve SHS's ACL mechanism, here in this PR proposed to:\r\n\r\n1. Unify two different ACLs into one (\"spark.history.ui.acl.enable\").\r\n2. Do not enable `SecurityFilter` in the base URL to do \"all or nothing\" filter, leverage app base ACL to partially list apps which can be accessed by user.\r\n\r\nWith this PR:\r\n\r\n1. Admin user could see/download the list of all applications, as well as application details.\r\n2. Normal user could see/download the list of applications accessible by him, as well as the details.\r\n\r\n## How was this patch tested?\r\n\r\nNew UTs are added, also verified in real cluster.\r\n\r\nCC @tgravescs @vanzin please help to review, this PR changes the semantics you did previously. Thanks a lot.\r\n"},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/295895556","html_url":"https://github.com/apache/spark/pull/17582#issuecomment-295895556","issue_url":"https://api.github.com/repos/apache/spark/issues/17582","id":295895556,"user":{"login":"AmplabJenkins","id":4031291,"avatar_url":"https://avatars0.githubusercontent.com/u/4031291?v=3","gravatar_id":"","url":"https://api.github.com/users/AmplabJenkins","html_url":"https://github.com/AmplabJenkins","followers_url":"https://api.github.com/users/AmplabJenkins/followers","following_url":"https://api.github.com/users/AmplabJenkins/following{/other_user}","gists_url":"https://api.github.com/users/AmplabJenkins/gists{/gist_id}","starred_url":"https://api.github.com/users/AmplabJenkins/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/AmplabJenkins/subscriptions","organizations_url":"https://api.github.com/users/AmplabJenkins/orgs","repos_url":"https://api.github.com/users/AmplabJenkins/repos","events_url":"https://api.github.com/users/AmplabJenkins/events{/privacy}","received_events_url":"https://api.github.com/users/AmplabJenkins/received_events","type":"User","site_admin":false},"created_at":"2017-04-20T20:27:37Z","updated_at":"2017-04-20T20:27:37Z","body":"Merged build finished. Test PASSed."}}
{"actor":{"display_login":"AmplabJenkins"}}