{"action":"closed","number":660,"pull_request":{"url":"https://api.github.com/repos/HazyResearch/snorkel/pulls/660","id":123908289,"html_url":"https://github.com/HazyResearch/snorkel/pull/660","diff_url":"https://github.com/HazyResearch/snorkel/pull/660.diff","patch_url":"https://github.com/HazyResearch/snorkel/pull/660.patch","issue_url":"https://api.github.com/repos/HazyResearch/snorkel/issues/660","number":660,"state":"closed","locked":false,"title":"Scoped categorical","user":{"login":"ajratner","id":1585405,"avatar_url":"https://avatars3.githubusercontent.com/u/1585405?v=3","gravatar_id":"","url":"https://api.github.com/users/ajratner","html_url":"https://github.com/ajratner","followers_url":"https://api.github.com/users/ajratner/followers","following_url":"https://api.github.com/users/ajratner/following{/other_user}","gists_url":"https://api.github.com/users/ajratner/gists{/gist_id}","starred_url":"https://api.github.com/users/ajratner/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ajratner/subscriptions","organizations_url":"https://api.github.com/users/ajratner/orgs","repos_url":"https://api.github.com/users/ajratner/repos","events_url":"https://api.github.com/users/ajratner/events{/privacy}","received_events_url":"https://api.github.com/users/ajratner/received_events","type":"User","site_admin":false},"body":"Adds support for \"scoped\" categorical mode in `GenerativeModel` by setting `GenerativeModel.train(..., scoped_categorical=True)`.  In this setting, we assume there is a large cardinality but that each candidate has a much smaller effective cardinality; entries are re-mapped to row-specific values so that sampling is not massively inefficient.\r\n\r\nNotes:\r\n* No support currently for pre-specified per-candidate support sets; these are inferred as the union of all LF labels applied to a given candidate\r\n* +1 smoothing is used w.r.t. cardinality: when computing the LF accuracies, we add +1 to the inferred cardinality to avoid weirdness when e.g. inferred cardinality would be 1 (could be quite common)\r\n* A sparse matrix is returned (re-mapped to the correct original value range)","created_at":"2017-06-05T03:31:15Z","updated_at":"2017-06-06T05:03:24Z","closed_at":"2017-06-06T05:03:24Z","merged_at":"2017-06-06T05:03:24Z","merge_commit_sha":"36a94bb69bb40142c3c6b848a3997935ad58e545","assignee":null,"assignees":[],"requested_reviewers":[],"milestone":null,"commits_url":"https://api.github.com/repos/HazyResearch/snorkel/pulls/660/commits","review_comments_url":"https://api.github.com/repos/HazyResearch/snorkel/pulls/660/comments","review_comment_url":"https://api.github.com/repos/HazyResearch/snorkel/pulls/comments{/number}","comments_url":"https://api.github.com/repos/HazyResearch/snorkel/issues/660/comments","statuses_url":"https://api.github.com/repos/HazyResearch/snorkel/statuses/10f9b1ee22c791dceb9a93947eb366355982e174","head":{"label":"HazyResearch:scoped-categorical","ref":"scoped-categorical","sha":"10f9b1ee22c791dceb9a93947eb366355982e174","user":{"login":"HazyResearch","id":2165246,"avatar_url":"https://avatars2.githubusercontent.com/u/2165246?v=3","gravatar_id":"","url":"https://api.github.com/users/HazyResearch","html_url":"https://github.com/HazyResearch","followers_url":"https://api.github.com/users/HazyResearch/followers","following_url":"https://api.github.com/users/HazyResearch/following{/other_user}","gists_url":"https://api.github.com/users/HazyResearch/gists{/gist_id}","starred_url":"https://api.github.com/users/HazyResearch/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/HazyResearch/subscriptions","organizations_url":"https://api.github.com/users/HazyResearch/orgs","repos_url":"https://api.github.com/users/HazyResearch/repos","events_url":"https://api.github.com/users/HazyResearch/events{/privacy}","received_events_url":"https://api.github.com/users/HazyResearch/received_events","type":"Organization","site_admin":false},"repo":{"id":52581991,"name":"snorkel","full_name":"HazyResearch/snorkel","owner":{"login":"HazyResearch","id":2165246,"avatar_url":"https://avatars2.githubusercontent.com/u/2165246?v=3","gravatar_id":"","url":"https://api.github.com/users/HazyResearch","html_url":"https://github.com/HazyResearch","followers_url":"https://api.github.com/users/HazyResearch/followers","following_url":"https://api.github.com/users/HazyResearch/following{/other_user}","gists_url":"https://api.github.com/users/HazyResearch/gists{/gist_id}","starred_url":"https://api.github.com/users/HazyResearch/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/HazyResearch/subscriptions","organizations_url":"https://api.github.com/users/HazyResearch/orgs","repos_url":"https://api.github.com/users/HazyResearch/repos","events_url":"https://api.github.com/users/HazyResearch/events{/privacy}","received_events_url":"https://api.github.com/users/HazyResearch/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/HazyResearch/snorkel","description":"A training data creation and management system focused on information extraction","fork":false,"url":"https://api.github.com/repos/HazyResearch/snorkel","forks_url":"https://api.github.com/repos/HazyResearch/snorkel/forks","keys_url":"https://api.github.com/repos/HazyResearch/snorkel/keys{/key_id}","collaborators_url":"https://api.github.com/repos/HazyResearch/snorkel/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/HazyResearch/snorkel/teams","hooks_url":"https://api.github.com/repos/HazyResearch/snorkel/hooks","issue_events_url":"https://api.github.com/repos/HazyResearch/snorkel/issues/events{/number}","events_url":"https://api.github.com/repos/HazyResearch/snorkel/events","assignees_url":"https://api.github.com/repos/HazyResearch/snorkel/assignees{/user}","branches_url":"https://api.github.com/repos/HazyResearch/snorkel/branches{/branch}","tags_url":"https://api.github.com/repos/HazyResearch/snorkel/tags","blobs_url":"https://api.github.com/repos/HazyResearch/snorkel/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/HazyResearch/snorkel/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/HazyResearch/snorkel/git/refs{/sha}","trees_url":"https://api.github.com/repos/HazyResearch/snorkel/git/trees{/sha}","statuses_url":"https://api.github.com/repos/HazyResearch/snorkel/statuses/{sha}","languages_url":"https://api.github.com/repos/HazyResearch/snorkel/languages","stargazers_url":"https://api.github.com/repos/HazyResearch/snorkel/stargazers","contributors_url":"https://api.github.com/repos/HazyResearch/snorkel/contributors","subscribers_url":"https://api.github.com/repos/HazyResearch/snorkel/subscribers","subscription_url":"https://api.github.com/repos/HazyResearch/snorkel/subscription","commits_url":"https://api.github.com/repos/HazyResearch/snorkel/commits{/sha}","git_commits_url":"https://api.github.com/repos/HazyResearch/snorkel/git/commits{/sha}","comments_url":"https://api.github.com/repos/HazyResearch/snorkel/comments{/number}","issue_comment_url":"https://api.github.com/repos/HazyResearch/snorkel/issues/comments{/number}","contents_url":"https://api.github.com/repos/HazyResearch/snorkel/contents/{+path}","compare_url":"https://api.github.com/repos/HazyResearch/snorkel/compare/{base}...{head}","merges_url":"https://api.github.com/repos/HazyResearch/snorkel/merges","archive_url":"https://api.github.com/repos/HazyResearch/snorkel/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/HazyResearch/snorkel/downloads","issues_url":"https://api.github.com/repos/HazyResearch/snorkel/issues{/number}","pulls_url":"https://api.github.com/repos/HazyResearch/snorkel/pulls{/number}","milestones_url":"https://api.github.com/repos/HazyResearch/snorkel/milestones{/number}","notifications_url":"https://api.github.com/repos/HazyResearch/snorkel/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/HazyResearch/snorkel/labels{/name}","releases_url":"https://api.github.com/repos/HazyResearch/snorkel/releases{/id}","deployments_url":"https://api.github.com/repos/HazyResearch/snorkel/deployments","created_at":"2016-02-26T05:52:45Z","updated_at":"2017-06-03T12:56:19Z","pushed_at":"2017-06-06T05:03:24Z","git_url":"git://github.com/HazyResearch/snorkel.git","ssh_url":"git@github.com:HazyResearch/snorkel.git","clone_url":"https://github.com/HazyResearch/snorkel.git","svn_url":"https://github.com/HazyResearch/snorkel","homepage":"http://snorkel.stanford.edu","size":315898,"stargazers_count":313,"watchers_count":313,"language":"Python","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":true,"forks_count":84,"mirror_url":null,"open_issues_count":76,"forks":84,"open_issues":76,"watchers":313,"default_branch":"master"}},"base":{"label":"HazyResearch:dev","ref":"dev","sha":"01487b3775309238098760a974f4cdb44c3f4596","user":{"login":"HazyResearch","id":2165246,"avatar_url":"https://avatars2.githubusercontent.com/u/2165246?v=3","gravatar_id":"","url":"https://api.github.com/users/HazyResearch","html_url":"https://github.com/HazyResearch","followers_url":"https://api.github.com/users/HazyResearch/followers","following_url":"https://api.github.com/users/HazyResearch/following{/other_user}","gists_url":"https://api.github.com/users/HazyResearch/gists{/gist_id}","starred_url":"https://api.github.com/users/HazyResearch/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/HazyResearch/subscriptions","organizations_url":"https://api.github.com/users/HazyResearch/orgs","repos_url":"https://api.github.com/users/HazyResearch/repos","events_url":"https://api.github.com/users/HazyResearch/events{/privacy}","received_events_url":"https://api.github.com/users/HazyResearch/received_events","type":"Organization","site_admin":false},"repo":{"id":52581991,"name":"snorkel","full_name":"HazyResearch/snorkel","owner":{"login":"HazyResearch","id":2165246,"avatar_url":"https://avatars2.githubusercontent.com/u/2165246?v=3","gravatar_id":"","url":"https://api.github.com/users/HazyResearch","html_url":"https://github.com/HazyResearch","followers_url":"https://api.github.com/users/HazyResearch/followers","following_url":"https://api.github.com/users/HazyResearch/following{/other_user}","gists_url":"https://api.github.com/users/HazyResearch/gists{/gist_id}","starred_url":"https://api.github.com/users/HazyResearch/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/HazyResearch/subscriptions","organizations_url":"https://api.github.com/users/HazyResearch/orgs","repos_url":"https://api.github.com/users/HazyResearch/repos","events_url":"https://api.github.com/users/HazyResearch/events{/privacy}","received_events_url":"https://api.github.com/users/HazyResearch/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/HazyResearch/snorkel","description":"A training data creation and management system focused on information extraction","fork":false,"url":"https://api.github.com/repos/HazyResearch/snorkel","forks_url":"https://api.github.com/repos/HazyResearch/snorkel/forks","keys_url":"https://api.github.com/repos/HazyResearch/snorkel/keys{/key_id}","collaborators_url":"https://api.github.com/repos/HazyResearch/snorkel/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/HazyResearch/snorkel/teams","hooks_url":"https://api.github.com/repos/HazyResearch/snorkel/hooks","issue_events_url":"https://api.github.com/repos/HazyResearch/snorkel/issues/events{/number}","events_url":"https://api.github.com/repos/HazyResearch/snorkel/events","assignees_url":"https://api.github.com/repos/HazyResearch/snorkel/assignees{/user}","branches_url":"https://api.github.com/repos/HazyResearch/snorkel/branches{/branch}","tags_url":"https://api.github.com/repos/HazyResearch/snorkel/tags","blobs_url":"https://api.github.com/repos/HazyResearch/snorkel/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/HazyResearch/snorkel/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/HazyResearch/snorkel/git/refs{/sha}","trees_url":"https://api.github.com/repos/HazyResearch/snorkel/git/trees{/sha}","statuses_url":"https://api.github.com/repos/HazyResearch/snorkel/statuses/{sha}","languages_url":"https://api.github.com/repos/HazyResearch/snorkel/languages","stargazers_url":"https://api.github.com/repos/HazyResearch/snorkel/stargazers","contributors_url":"https://api.github.com/repos/HazyResearch/snorkel/contributors","subscribers_url":"https://api.github.com/repos/HazyResearch/snorkel/subscribers","subscription_url":"https://api.github.com/repos/HazyResearch/snorkel/subscription","commits_url":"https://api.github.com/repos/HazyResearch/snorkel/commits{/sha}","git_commits_url":"https://api.github.com/repos/HazyResearch/snorkel/git/commits{/sha}","comments_url":"https://api.github.com/repos/HazyResearch/snorkel/comments{/number}","issue_comment_url":"https://api.github.com/repos/HazyResearch/snorkel/issues/comments{/number}","contents_url":"https://api.github.com/repos/HazyResearch/snorkel/contents/{+path}","compare_url":"https://api.github.com/repos/HazyResearch/snorkel/compare/{base}...{head}","merges_url":"https://api.github.com/repos/HazyResearch/snorkel/merges","archive_url":"https://api.github.com/repos/HazyResearch/snorkel/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/HazyResearch/snorkel/downloads","issues_url":"https://api.github.com/repos/HazyResearch/snorkel/issues{/number}","pulls_url":"https://api.github.com/repos/HazyResearch/snorkel/pulls{/number}","milestones_url":"https://api.github.com/repos/HazyResearch/snorkel/milestones{/number}","notifications_url":"https://api.github.com/repos/HazyResearch/snorkel/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/HazyResearch/snorkel/labels{/name}","releases_url":"https://api.github.com/repos/HazyResearch/snorkel/releases{/id}","deployments_url":"https://api.github.com/repos/HazyResearch/snorkel/deployments","created_at":"2016-02-26T05:52:45Z","updated_at":"2017-06-03T12:56:19Z","pushed_at":"2017-06-06T05:03:24Z","git_url":"git://github.com/HazyResearch/snorkel.git","ssh_url":"git@github.com:HazyResearch/snorkel.git","clone_url":"https://github.com/HazyResearch/snorkel.git","svn_url":"https://github.com/HazyResearch/snorkel","homepage":"http://snorkel.stanford.edu","size":315898,"stargazers_count":313,"watchers_count":313,"language":"Python","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":true,"forks_count":84,"mirror_url":null,"open_issues_count":76,"forks":84,"open_issues":76,"watchers":313,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/HazyResearch/snorkel/pulls/660"},"html":{"href":"https://github.com/HazyResearch/snorkel/pull/660"},"issue":{"href":"https://api.github.com/repos/HazyResearch/snorkel/issues/660"},"comments":{"href":"https://api.github.com/repos/HazyResearch/snorkel/issues/660/comments"},"review_comments":{"href":"https://api.github.com/repos/HazyResearch/snorkel/pulls/660/comments"},"review_comment":{"href":"https://api.github.com/repos/HazyResearch/snorkel/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/HazyResearch/snorkel/pulls/660/commits"},"statuses":{"href":"https://api.github.com/repos/HazyResearch/snorkel/statuses/10f9b1ee22c791dceb9a93947eb366355982e174"}},"merged":true,"mergeable":null,"rebaseable":null,"mergeable_state":"unknown","merged_by":{"login":"ajratner","id":1585405,"avatar_url":"https://avatars3.githubusercontent.com/u/1585405?v=3","gravatar_id":"","url":"https://api.github.com/users/ajratner","html_url":"https://github.com/ajratner","followers_url":"https://api.github.com/users/ajratner/followers","following_url":"https://api.github.com/users/ajratner/following{/other_user}","gists_url":"https://api.github.com/users/ajratner/gists{/gist_id}","starred_url":"https://api.github.com/users/ajratner/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ajratner/subscriptions","organizations_url":"https://api.github.com/users/ajratner/orgs","repos_url":"https://api.github.com/users/ajratner/repos","events_url":"https://api.github.com/users/ajratner/events{/privacy}","received_events_url":"https://api.github.com/users/ajratner/received_events","type":"User","site_admin":false},"comments":1,"review_comments":7,"maintainer_can_modify":false,"commits":22,"additions":283,"deletions":159,"changed_files":4}}
{"actor":{"display_login":"ajratner"}}