{"action":"created","comment":{"url":"https://api.github.com/repos/typelevel/frameless/pulls/comments/122730426","pull_request_review_id":44871723,"id":122730426,"diff_hunk":"@@ -0,0 +1,19 @@\n+package frameless\n+package ops\n+\n+import shapeless.HList\n+import shapeless.ops.hlist.Filter\n+\n+import scala.annotation.implicitNotFound\n+\n+\n+@implicitNotFound(\"HList of type '${L}' is NOT comprised of only elements of type '${U}'\")\n+trait MatchesAll[L <: HList, U]\n+\n+object MatchesAll {\n+  implicit def matchesAll[L <: HList, U, Rest <: HList]\n+  (implicit\n+   filtered: Filter.Aux[L, U, Rest],\n+   remainedSame: Rest =:= L","path":"dataset/src/main/scala/frameless/ops/MatchesAll.scala","position":17,"original_position":17,"commit_id":"1fb594972d1b78b7298b04a04b8996a4cc527afa","original_commit_id":"1fb594972d1b78b7298b04a04b8996a4cc527afa","user":{"login":"OlivierBlanvillain","id":1085297,"avatar_url":"https://avatars1.githubusercontent.com/u/1085297?v=3","gravatar_id":"","url":"https://api.github.com/users/OlivierBlanvillain","html_url":"https://github.com/OlivierBlanvillain","followers_url":"https://api.github.com/users/OlivierBlanvillain/followers","following_url":"https://api.github.com/users/OlivierBlanvillain/following{/other_user}","gists_url":"https://api.github.com/users/OlivierBlanvillain/gists{/gist_id}","starred_url":"https://api.github.com/users/OlivierBlanvillain/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/OlivierBlanvillain/subscriptions","organizations_url":"https://api.github.com/users/OlivierBlanvillain/orgs","repos_url":"https://api.github.com/users/OlivierBlanvillain/repos","events_url":"https://api.github.com/users/OlivierBlanvillain/events{/privacy}","received_events_url":"https://api.github.com/users/OlivierBlanvillain/received_events","type":"User","site_admin":false},"body":"Yes, `FilterNot.Aux[L, U, HNil]` (← equivalent to `FilterNot[Values, P]#Out =:= HNil`), that was my original suggestion :smile: ","created_at":"2017-06-19T14:54:47Z","updated_at":"2017-06-19T14:54:47Z","html_url":"https://github.com/typelevel/frameless/pull/150#discussion_r122730426","pull_request_url":"https://api.github.com/repos/typelevel/frameless/pulls/150","_links":{"self":{"href":"https://api.github.com/repos/typelevel/frameless/pulls/comments/122730426"},"html":{"href":"https://github.com/typelevel/frameless/pull/150#discussion_r122730426"},"pull_request":{"href":"https://api.github.com/repos/typelevel/frameless/pulls/150"}}},"pull_request":{"url":"https://api.github.com/repos/typelevel/frameless/pulls/150","id":125001046,"html_url":"https://github.com/typelevel/frameless/pull/150","diff_url":"https://github.com/typelevel/frameless/pull/150.diff","patch_url":"https://github.com/typelevel/frameless/pull/150.patch","issue_url":"https://api.github.com/repos/typelevel/frameless/issues/150","number":150,"state":"open","locked":false,"title":"Fixing #135: Adding support for pivot","user":{"login":"imarios","id":6893839,"avatar_url":"https://avatars3.githubusercontent.com/u/6893839?v=3","gravatar_id":"","url":"https://api.github.com/users/imarios","html_url":"https://github.com/imarios","followers_url":"https://api.github.com/users/imarios/followers","following_url":"https://api.github.com/users/imarios/following{/other_user}","gists_url":"https://api.github.com/users/imarios/gists{/gist_id}","starred_url":"https://api.github.com/users/imarios/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/imarios/subscriptions","organizations_url":"https://api.github.com/users/imarios/orgs","repos_url":"https://api.github.com/users/imarios/repos","events_url":"https://api.github.com/users/imarios/events{/privacy}","received_events_url":"https://api.github.com/users/imarios/received_events","type":"User","site_admin":false},"body":"More on pivot: https://databricks.com/blog/2016/02/09/reshaping-data-with-pivot-in-apache-spark.html\r\n\r\n```scala\r\ncase class G(a: String, b: String, x: Int)\r\nval t = TypedDataset.create(Seq(\r\n   G(\"a\",\"x\", 1), G(\"a\",\"y\", 2), G(\"a\",\"z\", 2), \r\n   G(\"a\",\"z\", 1), G(\"a\",\"z\", 22), G(\"b\",\"y\", 20), G(\"b\",\"x\", 10))\r\n)\r\n\r\n t.groupBy(t('a)).pivot(t('b), /*the values we want to pivot*/ \"x\"::\"z\"::\"y\"::HNil).agg(sum(t('x)))\r\nres1: frameless.TypedDataset[(String, Option[Long], Option[Long], Option[Long])] \r\n   = [_1: string, _2: bigint ... 2 more fields]\r\n// Since a missing value on a pivot is encoded as null in Spark, we encode the aggregated values on pivot to be Option[_]\r\n// The resulting type comprises of the groupBy column(s) followed by the aggregated values.\r\n\r\nres1.collect().run().foreach(println)\r\n> (b,Some(10),None,Some(20)) // \"z\" doesn't exists for group \"b\" so the result of sum is None\r\n> (a,Some(1),Some(25),Some(2))\r\n\r\n// More aggregations generate more columns. Each pivoted value (here x,z,y) will have one triplet of aggregated values starting from left to right.  The return type is adjusted accordingly. \r\nscala> t.groupBy(t('a)).pivot(t('b), \"x\"::\"z\"::\"y\"::HNil).agg(sum(t('x)), count[G], first(t('x)))\r\nres10: frameless.TypedDataset[(String, Option[Long], Option[Long], Option[Int], Option[Long], Option[Long], Option[Int], Option[Long], Option[Long], Option[Int])] = [_1: string, _2: bigint ... 8 more fields]\r\n```","created_at":"2017-06-10T18:58:52Z","updated_at":"2017-06-19T14:54:47Z","closed_at":null,"merged_at":null,"merge_commit_sha":"3ce65fc1f49d99e6399aa1b80e31f093c269888c","assignee":null,"assignees":[],"requested_reviewers":[],"milestone":{"url":"https://api.github.com/repos/typelevel/frameless/milestones/1","html_url":"https://github.com/typelevel/frameless/milestone/1","labels_url":"https://api.github.com/repos/typelevel/frameless/milestones/1/labels","id":2528645,"number":1,"title":"0.4-release","description":"","creator":{"login":"imarios","id":6893839,"avatar_url":"https://avatars3.githubusercontent.com/u/6893839?v=3","gravatar_id":"","url":"https://api.github.com/users/imarios","html_url":"https://github.com/imarios","followers_url":"https://api.github.com/users/imarios/followers","following_url":"https://api.github.com/users/imarios/following{/other_user}","gists_url":"https://api.github.com/users/imarios/gists{/gist_id}","starred_url":"https://api.github.com/users/imarios/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/imarios/subscriptions","organizations_url":"https://api.github.com/users/imarios/orgs","repos_url":"https://api.github.com/users/imarios/repos","events_url":"https://api.github.com/users/imarios/events{/privacy}","received_events_url":"https://api.github.com/users/imarios/received_events","type":"User","site_admin":false},"open_issues":3,"closed_issues":0,"state":"open","created_at":"2017-05-18T08:06:13Z","updated_at":"2017-06-16T07:11:54Z","due_on":null,"closed_at":null},"commits_url":"https://api.github.com/repos/typelevel/frameless/pulls/150/commits","review_comments_url":"https://api.github.com/repos/typelevel/frameless/pulls/150/comments","review_comment_url":"https://api.github.com/repos/typelevel/frameless/pulls/comments{/number}","comments_url":"https://api.github.com/repos/typelevel/frameless/issues/150/comments","statuses_url":"https://api.github.com/repos/typelevel/frameless/statuses/1fb594972d1b78b7298b04a04b8996a4cc527afa","head":{"label":"imarios:pivot","ref":"pivot","sha":"1fb594972d1b78b7298b04a04b8996a4cc527afa","user":{"login":"imarios","id":6893839,"avatar_url":"https://avatars3.githubusercontent.com/u/6893839?v=3","gravatar_id":"","url":"https://api.github.com/users/imarios","html_url":"https://github.com/imarios","followers_url":"https://api.github.com/users/imarios/followers","following_url":"https://api.github.com/users/imarios/following{/other_user}","gists_url":"https://api.github.com/users/imarios/gists{/gist_id}","starred_url":"https://api.github.com/users/imarios/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/imarios/subscriptions","organizations_url":"https://api.github.com/users/imarios/orgs","repos_url":"https://api.github.com/users/imarios/repos","events_url":"https://api.github.com/users/imarios/events{/privacy}","received_events_url":"https://api.github.com/users/imarios/received_events","type":"User","site_admin":false},"repo":{"id":70028426,"name":"frameless","full_name":"imarios/frameless","owner":{"login":"imarios","id":6893839,"avatar_url":"https://avatars3.githubusercontent.com/u/6893839?v=3","gravatar_id":"","url":"https://api.github.com/users/imarios","html_url":"https://github.com/imarios","followers_url":"https://api.github.com/users/imarios/followers","following_url":"https://api.github.com/users/imarios/following{/other_user}","gists_url":"https://api.github.com/users/imarios/gists{/gist_id}","starred_url":"https://api.github.com/users/imarios/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/imarios/subscriptions","organizations_url":"https://api.github.com/users/imarios/orgs","repos_url":"https://api.github.com/users/imarios/repos","events_url":"https://api.github.com/users/imarios/events{/privacy}","received_events_url":"https://api.github.com/users/imarios/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/imarios/frameless","description":"Expressive types for Spark.","fork":true,"url":"https://api.github.com/repos/imarios/frameless","forks_url":"https://api.github.com/repos/imarios/frameless/forks","keys_url":"https://api.github.com/repos/imarios/frameless/keys{/key_id}","collaborators_url":"https://api.github.com/repos/imarios/frameless/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/imarios/frameless/teams","hooks_url":"https://api.github.com/repos/imarios/frameless/hooks","issue_events_url":"https://api.github.com/repos/imarios/frameless/issues/events{/number}","events_url":"https://api.github.com/repos/imarios/frameless/events","assignees_url":"https://api.github.com/repos/imarios/frameless/assignees{/user}","branches_url":"https://api.github.com/repos/imarios/frameless/branches{/branch}","tags_url":"https://api.github.com/repos/imarios/frameless/tags","blobs_url":"https://api.github.com/repos/imarios/frameless/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/imarios/frameless/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/imarios/frameless/git/refs{/sha}","trees_url":"https://api.github.com/repos/imarios/frameless/git/trees{/sha}","statuses_url":"https://api.github.com/repos/imarios/frameless/statuses/{sha}","languages_url":"https://api.github.com/repos/imarios/frameless/languages","stargazers_url":"https://api.github.com/repos/imarios/frameless/stargazers","contributors_url":"https://api.github.com/repos/imarios/frameless/contributors","subscribers_url":"https://api.github.com/repos/imarios/frameless/subscribers","subscription_url":"https://api.github.com/repos/imarios/frameless/subscription","commits_url":"https://api.github.com/repos/imarios/frameless/commits{/sha}","git_commits_url":"https://api.github.com/repos/imarios/frameless/git/commits{/sha}","comments_url":"https://api.github.com/repos/imarios/frameless/comments{/number}","issue_comment_url":"https://api.github.com/repos/imarios/frameless/issues/comments{/number}","contents_url":"https://api.github.com/repos/imarios/frameless/contents/{+path}","compare_url":"https://api.github.com/repos/imarios/frameless/compare/{base}...{head}","merges_url":"https://api.github.com/repos/imarios/frameless/merges","archive_url":"https://api.github.com/repos/imarios/frameless/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/imarios/frameless/downloads","issues_url":"https://api.github.com/repos/imarios/frameless/issues{/number}","pulls_url":"https://api.github.com/repos/imarios/frameless/pulls{/number}","milestones_url":"https://api.github.com/repos/imarios/frameless/milestones{/number}","notifications_url":"https://api.github.com/repos/imarios/frameless/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/imarios/frameless/labels{/name}","releases_url":"https://api.github.com/repos/imarios/frameless/releases{/id}","deployments_url":"https://api.github.com/repos/imarios/frameless/deployments","created_at":"2016-10-05T03:54:16Z","updated_at":"2016-10-24T16:35:42Z","pushed_at":"2017-06-18T14:27:20Z","git_url":"git://github.com/imarios/frameless.git","ssh_url":"git@github.com:imarios/frameless.git","clone_url":"https://github.com/imarios/frameless.git","svn_url":"https://github.com/imarios/frameless","homepage":"","size":459,"stargazers_count":1,"watchers_count":1,"language":"Scala","has_issues":false,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":0,"forks":0,"open_issues":0,"watchers":1,"default_branch":"master"}},"base":{"label":"typelevel:master","ref":"master","sha":"5b0f36b49ef72884054478e794853de83fa91ca6","user":{"login":"typelevel","id":3731824,"avatar_url":"https://avatars3.githubusercontent.com/u/3731824?v=3","gravatar_id":"","url":"https://api.github.com/users/typelevel","html_url":"https://github.com/typelevel","followers_url":"https://api.github.com/users/typelevel/followers","following_url":"https://api.github.com/users/typelevel/following{/other_user}","gists_url":"https://api.github.com/users/typelevel/gists{/gist_id}","starred_url":"https://api.github.com/users/typelevel/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/typelevel/subscriptions","organizations_url":"https://api.github.com/users/typelevel/orgs","repos_url":"https://api.github.com/users/typelevel/repos","events_url":"https://api.github.com/users/typelevel/events{/privacy}","received_events_url":"https://api.github.com/users/typelevel/received_events","type":"Organization","site_admin":false},"repo":{"id":33022955,"name":"frameless","full_name":"typelevel/frameless","owner":{"login":"typelevel","id":3731824,"avatar_url":"https://avatars3.githubusercontent.com/u/3731824?v=3","gravatar_id":"","url":"https://api.github.com/users/typelevel","html_url":"https://github.com/typelevel","followers_url":"https://api.github.com/users/typelevel/followers","following_url":"https://api.github.com/users/typelevel/following{/other_user}","gists_url":"https://api.github.com/users/typelevel/gists{/gist_id}","starred_url":"https://api.github.com/users/typelevel/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/typelevel/subscriptions","organizations_url":"https://api.github.com/users/typelevel/orgs","repos_url":"https://api.github.com/users/typelevel/repos","events_url":"https://api.github.com/users/typelevel/events{/privacy}","received_events_url":"https://api.github.com/users/typelevel/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/typelevel/frameless","description":"Expressive types for Spark.","fork":false,"url":"https://api.github.com/repos/typelevel/frameless","forks_url":"https://api.github.com/repos/typelevel/frameless/forks","keys_url":"https://api.github.com/repos/typelevel/frameless/keys{/key_id}","collaborators_url":"https://api.github.com/repos/typelevel/frameless/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/typelevel/frameless/teams","hooks_url":"https://api.github.com/repos/typelevel/frameless/hooks","issue_events_url":"https://api.github.com/repos/typelevel/frameless/issues/events{/number}","events_url":"https://api.github.com/repos/typelevel/frameless/events","assignees_url":"https://api.github.com/repos/typelevel/frameless/assignees{/user}","branches_url":"https://api.github.com/repos/typelevel/frameless/branches{/branch}","tags_url":"https://api.github.com/repos/typelevel/frameless/tags","blobs_url":"https://api.github.com/repos/typelevel/frameless/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/typelevel/frameless/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/typelevel/frameless/git/refs{/sha}","trees_url":"https://api.github.com/repos/typelevel/frameless/git/trees{/sha}","statuses_url":"https://api.github.com/repos/typelevel/frameless/statuses/{sha}","languages_url":"https://api.github.com/repos/typelevel/frameless/languages","stargazers_url":"https://api.github.com/repos/typelevel/frameless/stargazers","contributors_url":"https://api.github.com/repos/typelevel/frameless/contributors","subscribers_url":"https://api.github.com/repos/typelevel/frameless/subscribers","subscription_url":"https://api.github.com/repos/typelevel/frameless/subscription","commits_url":"https://api.github.com/repos/typelevel/frameless/commits{/sha}","git_commits_url":"https://api.github.com/repos/typelevel/frameless/git/commits{/sha}","comments_url":"https://api.github.com/repos/typelevel/frameless/comments{/number}","issue_comment_url":"https://api.github.com/repos/typelevel/frameless/issues/comments{/number}","contents_url":"https://api.github.com/repos/typelevel/frameless/contents/{+path}","compare_url":"https://api.github.com/repos/typelevel/frameless/compare/{base}...{head}","merges_url":"https://api.github.com/repos/typelevel/frameless/merges","archive_url":"https://api.github.com/repos/typelevel/frameless/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/typelevel/frameless/downloads","issues_url":"https://api.github.com/repos/typelevel/frameless/issues{/number}","pulls_url":"https://api.github.com/repos/typelevel/frameless/pulls{/number}","milestones_url":"https://api.github.com/repos/typelevel/frameless/milestones{/number}","notifications_url":"https://api.github.com/repos/typelevel/frameless/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/typelevel/frameless/labels{/name}","releases_url":"https://api.github.com/repos/typelevel/frameless/releases{/id}","deployments_url":"https://api.github.com/repos/typelevel/frameless/deployments","created_at":"2015-03-28T06:03:10Z","updated_at":"2017-06-19T07:27:23Z","pushed_at":"2017-06-18T14:27:21Z","git_url":"git://github.com/typelevel/frameless.git","ssh_url":"git@github.com:typelevel/frameless.git","clone_url":"https://github.com/typelevel/frameless.git","svn_url":"https://github.com/typelevel/frameless","homepage":"","size":1173,"stargazers_count":216,"watchers_count":216,"language":"Scala","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":true,"forks_count":24,"mirror_url":null,"open_issues_count":34,"forks":24,"open_issues":34,"watchers":216,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/typelevel/frameless/pulls/150"},"html":{"href":"https://github.com/typelevel/frameless/pull/150"},"issue":{"href":"https://api.github.com/repos/typelevel/frameless/issues/150"},"comments":{"href":"https://api.github.com/repos/typelevel/frameless/issues/150/comments"},"review_comments":{"href":"https://api.github.com/repos/typelevel/frameless/pulls/150/comments"},"review_comment":{"href":"https://api.github.com/repos/typelevel/frameless/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/typelevel/frameless/pulls/150/commits"},"statuses":{"href":"https://api.github.com/repos/typelevel/frameless/statuses/1fb594972d1b78b7298b04a04b8996a4cc527afa"}}}}
{"actor":{"display_login":"OlivierBlanvillain"}}