{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/16883","repository_url":"https://api.github.com/repos/apache/spark","labels_url":"https://api.github.com/repos/apache/spark/issues/16883/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/16883/comments","events_url":"https://api.github.com/repos/apache/spark/issues/16883/events","html_url":"https://github.com/apache/spark/pull/16883","id":206718412,"number":16883,"title":"[SPARK-17498][ML] StringIndexer enhancement for handling unseen labels","user":{"login":"VinceShieh","id":2673819,"avatar_url":"https://avatars3.githubusercontent.com/u/2673819?v=3","gravatar_id":"","url":"https://api.github.com/users/VinceShieh","html_url":"https://github.com/VinceShieh","followers_url":"https://api.github.com/users/VinceShieh/followers","following_url":"https://api.github.com/users/VinceShieh/following{/other_user}","gists_url":"https://api.github.com/users/VinceShieh/gists{/gist_id}","starred_url":"https://api.github.com/users/VinceShieh/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/VinceShieh/subscriptions","organizations_url":"https://api.github.com/users/VinceShieh/orgs","repos_url":"https://api.github.com/users/VinceShieh/repos","events_url":"https://api.github.com/users/VinceShieh/events{/privacy}","received_events_url":"https://api.github.com/users/VinceShieh/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":14,"created_at":"2017-02-10T06:24:47Z","updated_at":"2017-03-01T02:32:35Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/16883","html_url":"https://github.com/apache/spark/pull/16883","diff_url":"https://github.com/apache/spark/pull/16883.diff","patch_url":"https://github.com/apache/spark/pull/16883.patch"},"body":"## What changes were proposed in this pull request?\r\nThis PR is an enhancement to ML StringIndexer. \r\nBefore this PR, String Indexer only supports \"skip\"/\"error\" options to deal with unseen records.\r\nBut those unseen records might still be useful and user would like to keep the unseen labels in\r\ncertain use cases, This PR enables StringIndexer to support keeping unseen labels as\r\nindices [numLabels].\r\n\r\n'''Before\r\nStringIndexer().setHandleInvalid(\"skip\")\r\nStringIndexer().setHandleInvalid(\"error\")\r\n'''After\r\nsupport the third option \"keep\"\r\nStringIndexer().setHandleInvalid(\"keep\")\r\n\r\n## How was this patch tested?\r\nTest added in StringIndexerSuite\r\n\r\nSigned-off-by: VinceShieh <vincent.xie@intel.com>\r\n(Please fill in changes proposed in this fix)"},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/283226842","html_url":"https://github.com/apache/spark/pull/16883#issuecomment-283226842","issue_url":"https://api.github.com/repos/apache/spark/issues/16883","id":283226842,"user":{"login":"SparkQA","id":8108735,"avatar_url":"https://avatars2.githubusercontent.com/u/8108735?v=3","gravatar_id":"","url":"https://api.github.com/users/SparkQA","html_url":"https://github.com/SparkQA","followers_url":"https://api.github.com/users/SparkQA/followers","following_url":"https://api.github.com/users/SparkQA/following{/other_user}","gists_url":"https://api.github.com/users/SparkQA/gists{/gist_id}","starred_url":"https://api.github.com/users/SparkQA/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/SparkQA/subscriptions","organizations_url":"https://api.github.com/users/SparkQA/orgs","repos_url":"https://api.github.com/users/SparkQA/repos","events_url":"https://api.github.com/users/SparkQA/events{/privacy}","received_events_url":"https://api.github.com/users/SparkQA/received_events","type":"User","site_admin":false},"created_at":"2017-03-01T02:32:35Z","updated_at":"2017-03-01T02:32:35Z","body":"**[Test build #73643 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73643/testReport)** for PR 16883 at commit [`fa24e43`](https://github.com/apache/spark/commit/fa24e433c3f9fe6f76fe0a55df4551881f194d7b)."}}
{"actor":{"display_login":"SparkQA"}}