{"action":"created","comment":{"url":"https://api.github.com/repos/citusdata/citus/pulls/comments/111125113","pull_request_review_id":32346616,"id":111125113,"diff_hunk":"@@ -3048,6 +3044,63 @@ ErrorIfCannotPushdownSubquery(Query *subqueryTree, bool outerQueryHasLimit)\n \n \n /*\n+ * ErrorIfUnsupportedUnionQuery is a helper function for ErrorIfCannotPushdownSubquery().\n+ * It basically iterates over the subqueries that reside under the given set operations.\n+ *\n+ * The function also errors out for set operations INTERSECT and EXCEPT.\n+ */\n+static void\n+ErrorIfUnsupportedUnionQuery(List *rangeTableList,\n+\t\t\t\t\t\t\t SetOperationStmt *setOperationStatement,\n+\t\t\t\t\t\t\t bool outerQueryHasLimit)\n+{\n+\tNode *leftArgument = setOperationStatement->larg;\n+\tNode *rightArgument = setOperationStatement->rarg;\n+\n+\tif (setOperationStatement->op != SETOP_UNION)\n+\t{\n+\t\tereport(ERROR, (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),\n+\t\t\t\t\t\terrmsg(\"cannot push down this subquery\"),\n+\t\t\t\t\t\terrdetail(\"Intersect and Except are currently unsupported\")));\n+\t}\n+\n+\tif (IsA(leftArgument, RangeTblRef))\n+\t{\n+\t\tRangeTblRef *rangeTableReference = (RangeTblRef *) leftArgument;\n+\t\tRangeTblEntry *rangeTableEntry = rt_fetch(rangeTableReference->rtindex,\n+\t\t\t\t\t\t\t\t\t\t\t\t  rangeTableList);\n+\n+\t\tAssert(rangeTableEntry->rtekind == RTE_SUBQUERY);\n+\n+\t\tErrorIfCannotPushdownSubquery(rangeTableEntry->subquery, outerQueryHasLimit);\n+\t}\n+\telse if (IsA(leftArgument, SetOperationStmt))\n+\t{\n+\t\tSetOperationStmt *leftUnion = (SetOperationStmt *) leftArgument;\n+\n+\t\tErrorIfUnsupportedUnionQuery(rangeTableList, leftUnion, outerQueryHasLimit);","path":"src/backend/distributed/planner/multi_logical_optimizer.c","position":null,"original_position":76,"commit_id":"b1c4570abcedbdac0e48a9e0d16321cbe1254bb0","original_commit_id":"cdc53fd9797671bff2abce951b88d3d1e4a02577","user":{"login":"onderkalaci","id":7829470,"avatar_url":"https://avatars1.githubusercontent.com/u/7829470?v=3","gravatar_id":"","url":"https://api.github.com/users/onderkalaci","html_url":"https://github.com/onderkalaci","followers_url":"https://api.github.com/users/onderkalaci/followers","following_url":"https://api.github.com/users/onderkalaci/following{/other_user}","gists_url":"https://api.github.com/users/onderkalaci/gists{/gist_id}","starred_url":"https://api.github.com/users/onderkalaci/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/onderkalaci/subscriptions","organizations_url":"https://api.github.com/users/onderkalaci/orgs","repos_url":"https://api.github.com/users/onderkalaci/repos","events_url":"https://api.github.com/users/onderkalaci/events{/privacy}","received_events_url":"https://api.github.com/users/onderkalaci/received_events","type":"User","site_admin":false},"body":"Why is so? I'd be in favor of processing in the format that postgres keeps these structs.\r\n\r\n> We should also think about removing indirect recursion\r\n\r\nAgain, Postgres keeps exactly in that format. Why should we not follow it?\r\n","created_at":"2017-04-12T11:29:13Z","updated_at":"2017-04-12T11:57:02Z","html_url":"https://github.com/citusdata/citus/pull/1289#discussion_r111125113","pull_request_url":"https://api.github.com/repos/citusdata/citus/pulls/1289","_links":{"self":{"href":"https://api.github.com/repos/citusdata/citus/pulls/comments/111125113"},"html":{"href":"https://github.com/citusdata/citus/pull/1289#discussion_r111125113"},"pull_request":{"href":"https://api.github.com/repos/citusdata/citus/pulls/1289"}}},"pull_request":{"url":"https://api.github.com/repos/citusdata/citus/pulls/1289","id":112031775,"html_url":"https://github.com/citusdata/citus/pull/1289","diff_url":"https://github.com/citusdata/citus/pull/1289.diff","patch_url":"https://github.com/citusdata/citus/pull/1289.patch","issue_url":"https://api.github.com/repos/citusdata/citus/issues/1289","number":1289,"state":"open","locked":false,"title":"Subquery pushdown - Unions Phase - 1 ","user":{"login":"onderkalaci","id":7829470,"avatar_url":"https://avatars1.githubusercontent.com/u/7829470?v=3","gravatar_id":"","url":"https://api.github.com/users/onderkalaci","html_url":"https://github.com/onderkalaci","followers_url":"https://api.github.com/users/onderkalaci/followers","following_url":"https://api.github.com/users/onderkalaci/following{/other_user}","gists_url":"https://api.github.com/users/onderkalaci/gists{/gist_id}","starred_url":"https://api.github.com/users/onderkalaci/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/onderkalaci/subscriptions","organizations_url":"https://api.github.com/users/onderkalaci/orgs","repos_url":"https://api.github.com/users/onderkalaci/repos","events_url":"https://api.github.com/users/onderkalaci/events{/privacy}","received_events_url":"https://api.github.com/users/onderkalaci/received_events","type":"User","site_admin":false},"body":"(This PR allows `UNION` and `UNION ALL` that are not in the top level subquery. We'll open a separate PR for that.)\r\n\r\nIn general, the PR implements the following logic: Once a `UNION` or `UNION ALL` is found, each target list element at the same target list position  is added  for each subquery that is in the `UNION`/`UNION ALL` s.\r\n\r\nFor instance, for the following query, we add `events_subquery_1.user_id`, `events_subquery_2.user_id`, `events_subquery_3.user_id` and `events_subquery_4.user_id` to the equivelence class. \r\n\r\n```SQL\r\nSELECT (\"final_query\".\"event_types\") as types, count(*) AS sumOfEventType\r\nFROM\r\n  ( SELECT *, random()\r\n   FROM\r\n     ( SELECT \"t\".\"user_id\",\r\n              \"t\".\"time\",\r\n              unnest(\"t\".\"collected_events\") AS \"event_types\"\r\n      FROM\r\n        ( SELECT \"t1\".\"user_id\",\r\n                 min(\"t1\".\"time\") AS \"time\",\r\n                 array_agg((\"t1\".\"event\")\r\n                           ORDER BY TIME ASC, event DESC) AS collected_events\r\n         FROM (\r\n                 (SELECT *\r\n                  FROM\r\n                   (SELECT \"events\".\"user_id\",\r\n                          \"events\".\"time\",\r\n                         0 AS event\r\n                 FROM events_table as  \"events\"\r\n                WHERE event_type IN (10, 11, 12, 13, 14, 15) ) events_subquery_1) \r\n                UNION \r\n                 (SELECT *\r\n                  FROM\r\n                    (SELECT \"events\".\"user_id\", \"events\".\"time\", 1 AS event\r\n                     FROM events_table as \"events\"\r\n                     WHERE event_type IN (15, 16, 17, 18, 19) ) events_subquery_2)\r\n               UNION \r\n                 (SELECT *\r\n                  FROM\r\n                    (SELECT \"events\".\"user_id\", \"events\".\"time\", 2 AS event\r\n                     FROM events_table as  \"events\"\r\n                      WHERE event_type IN (20, 21, 22, 23, 24, 25) ) events_subquery_3)\r\n               UNION \r\n                 (SELECT *\r\n                  FROM\r\n                    (SELECT \"events\".\"user_id\", \"events\".\"time\", 3 AS event\r\n                     FROM events_table as \"events\"\r\n                     WHERE event_type IN (26, 27, 28, 29, 30, 13)) events_subquery_4)) t1\r\n         GROUP BY \"t1\".\"user_id\") AS t) \"q\" \r\nINNER JOIN\r\n     (SELECT \"users\".\"user_id\"\r\n      FROM users_table as \"users\"\r\n      WHERE value_1 > 50 and value_1 < 70) AS t ON (t.user_id = q.user_id)) as final_query\r\nGROUP BY types\r\nORDER BY types;\r\n```\r\n\r\n\r\n\r\n","created_at":"2017-03-22T15:04:39Z","updated_at":"2017-04-12T11:57:02Z","closed_at":null,"merged_at":null,"merge_commit_sha":"e40566ebdf362321f596e82409689167626d0936","assignee":{"login":"mtuncer","id":9413470,"avatar_url":"https://avatars0.githubusercontent.com/u/9413470?v=3","gravatar_id":"","url":"https://api.github.com/users/mtuncer","html_url":"https://github.com/mtuncer","followers_url":"https://api.github.com/users/mtuncer/followers","following_url":"https://api.github.com/users/mtuncer/following{/other_user}","gists_url":"https://api.github.com/users/mtuncer/gists{/gist_id}","starred_url":"https://api.github.com/users/mtuncer/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mtuncer/subscriptions","organizations_url":"https://api.github.com/users/mtuncer/orgs","repos_url":"https://api.github.com/users/mtuncer/repos","events_url":"https://api.github.com/users/mtuncer/events{/privacy}","received_events_url":"https://api.github.com/users/mtuncer/received_events","type":"User","site_admin":false},"assignees":[{"login":"mtuncer","id":9413470,"avatar_url":"https://avatars0.githubusercontent.com/u/9413470?v=3","gravatar_id":"","url":"https://api.github.com/users/mtuncer","html_url":"https://github.com/mtuncer","followers_url":"https://api.github.com/users/mtuncer/followers","following_url":"https://api.github.com/users/mtuncer/following{/other_user}","gists_url":"https://api.github.com/users/mtuncer/gists{/gist_id}","starred_url":"https://api.github.com/users/mtuncer/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mtuncer/subscriptions","organizations_url":"https://api.github.com/users/mtuncer/orgs","repos_url":"https://api.github.com/users/mtuncer/repos","events_url":"https://api.github.com/users/mtuncer/events{/privacy}","received_events_url":"https://api.github.com/users/mtuncer/received_events","type":"User","site_admin":false}],"milestone":null,"commits_url":"https://api.github.com/repos/citusdata/citus/pulls/1289/commits","review_comments_url":"https://api.github.com/repos/citusdata/citus/pulls/1289/comments","review_comment_url":"https://api.github.com/repos/citusdata/citus/pulls/comments{/number}","comments_url":"https://api.github.com/repos/citusdata/citus/issues/1289/comments","statuses_url":"https://api.github.com/repos/citusdata/citus/statuses/b1c4570abcedbdac0e48a9e0d16321cbe1254bb0","head":{"label":"citusdata:subquery_union","ref":"subquery_union","sha":"b1c4570abcedbdac0e48a9e0d16321cbe1254bb0","user":{"login":"citusdata","id":2545424,"avatar_url":"https://avatars1.githubusercontent.com/u/2545424?v=3","gravatar_id":"","url":"https://api.github.com/users/citusdata","html_url":"https://github.com/citusdata","followers_url":"https://api.github.com/users/citusdata/followers","following_url":"https://api.github.com/users/citusdata/following{/other_user}","gists_url":"https://api.github.com/users/citusdata/gists{/gist_id}","starred_url":"https://api.github.com/users/citusdata/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/citusdata/subscriptions","organizations_url":"https://api.github.com/users/citusdata/orgs","repos_url":"https://api.github.com/users/citusdata/repos","events_url":"https://api.github.com/users/citusdata/events{/privacy}","received_events_url":"https://api.github.com/users/citusdata/received_events","type":"Organization","site_admin":false},"repo":{"id":50874442,"name":"citus","full_name":"citusdata/citus","owner":{"login":"citusdata","id":2545424,"avatar_url":"https://avatars1.githubusercontent.com/u/2545424?v=3","gravatar_id":"","url":"https://api.github.com/users/citusdata","html_url":"https://github.com/citusdata","followers_url":"https://api.github.com/users/citusdata/followers","following_url":"https://api.github.com/users/citusdata/following{/other_user}","gists_url":"https://api.github.com/users/citusdata/gists{/gist_id}","starred_url":"https://api.github.com/users/citusdata/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/citusdata/subscriptions","organizations_url":"https://api.github.com/users/citusdata/orgs","repos_url":"https://api.github.com/users/citusdata/repos","events_url":"https://api.github.com/users/citusdata/events{/privacy}","received_events_url":"https://api.github.com/users/citusdata/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/citusdata/citus","description":"Scalable PostgreSQL for real-time workloads","fork":false,"url":"https://api.github.com/repos/citusdata/citus","forks_url":"https://api.github.com/repos/citusdata/citus/forks","keys_url":"https://api.github.com/repos/citusdata/citus/keys{/key_id}","collaborators_url":"https://api.github.com/repos/citusdata/citus/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/citusdata/citus/teams","hooks_url":"https://api.github.com/repos/citusdata/citus/hooks","issue_events_url":"https://api.github.com/repos/citusdata/citus/issues/events{/number}","events_url":"https://api.github.com/repos/citusdata/citus/events","assignees_url":"https://api.github.com/repos/citusdata/citus/assignees{/user}","branches_url":"https://api.github.com/repos/citusdata/citus/branches{/branch}","tags_url":"https://api.github.com/repos/citusdata/citus/tags","blobs_url":"https://api.github.com/repos/citusdata/citus/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/citusdata/citus/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/citusdata/citus/git/refs{/sha}","trees_url":"https://api.github.com/repos/citusdata/citus/git/trees{/sha}","statuses_url":"https://api.github.com/repos/citusdata/citus/statuses/{sha}","languages_url":"https://api.github.com/repos/citusdata/citus/languages","stargazers_url":"https://api.github.com/repos/citusdata/citus/stargazers","contributors_url":"https://api.github.com/repos/citusdata/citus/contributors","subscribers_url":"https://api.github.com/repos/citusdata/citus/subscribers","subscription_url":"https://api.github.com/repos/citusdata/citus/subscription","commits_url":"https://api.github.com/repos/citusdata/citus/commits{/sha}","git_commits_url":"https://api.github.com/repos/citusdata/citus/git/commits{/sha}","comments_url":"https://api.github.com/repos/citusdata/citus/comments{/number}","issue_comment_url":"https://api.github.com/repos/citusdata/citus/issues/comments{/number}","contents_url":"https://api.github.com/repos/citusdata/citus/contents/{+path}","compare_url":"https://api.github.com/repos/citusdata/citus/compare/{base}...{head}","merges_url":"https://api.github.com/repos/citusdata/citus/merges","archive_url":"https://api.github.com/repos/citusdata/citus/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/citusdata/citus/downloads","issues_url":"https://api.github.com/repos/citusdata/citus/issues{/number}","pulls_url":"https://api.github.com/repos/citusdata/citus/pulls{/number}","milestones_url":"https://api.github.com/repos/citusdata/citus/milestones{/number}","notifications_url":"https://api.github.com/repos/citusdata/citus/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/citusdata/citus/labels{/name}","releases_url":"https://api.github.com/repos/citusdata/citus/releases{/id}","deployments_url":"https://api.github.com/repos/citusdata/citus/deployments","created_at":"2016-02-01T21:58:39Z","updated_at":"2017-04-12T10:03:24Z","pushed_at":"2017-04-12T11:56:59Z","git_url":"git://github.com/citusdata/citus.git","ssh_url":"git@github.com:citusdata/citus.git","clone_url":"https://github.com/citusdata/citus.git","svn_url":"https://github.com/citusdata/citus","homepage":"https://www.citusdata.com","size":6300,"stargazers_count":1964,"watchers_count":1964,"language":"C","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":138,"mirror_url":null,"open_issues_count":451,"forks":138,"open_issues":451,"watchers":1964,"default_branch":"master"}},"base":{"label":"citusdata:subquery_main","ref":"subquery_main","sha":"e1beeb99a56851ae4c10d37f3a9b62c99a89c6ab","user":{"login":"citusdata","id":2545424,"avatar_url":"https://avatars1.githubusercontent.com/u/2545424?v=3","gravatar_id":"","url":"https://api.github.com/users/citusdata","html_url":"https://github.com/citusdata","followers_url":"https://api.github.com/users/citusdata/followers","following_url":"https://api.github.com/users/citusdata/following{/other_user}","gists_url":"https://api.github.com/users/citusdata/gists{/gist_id}","starred_url":"https://api.github.com/users/citusdata/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/citusdata/subscriptions","organizations_url":"https://api.github.com/users/citusdata/orgs","repos_url":"https://api.github.com/users/citusdata/repos","events_url":"https://api.github.com/users/citusdata/events{/privacy}","received_events_url":"https://api.github.com/users/citusdata/received_events","type":"Organization","site_admin":false},"repo":{"id":50874442,"name":"citus","full_name":"citusdata/citus","owner":{"login":"citusdata","id":2545424,"avatar_url":"https://avatars1.githubusercontent.com/u/2545424?v=3","gravatar_id":"","url":"https://api.github.com/users/citusdata","html_url":"https://github.com/citusdata","followers_url":"https://api.github.com/users/citusdata/followers","following_url":"https://api.github.com/users/citusdata/following{/other_user}","gists_url":"https://api.github.com/users/citusdata/gists{/gist_id}","starred_url":"https://api.github.com/users/citusdata/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/citusdata/subscriptions","organizations_url":"https://api.github.com/users/citusdata/orgs","repos_url":"https://api.github.com/users/citusdata/repos","events_url":"https://api.github.com/users/citusdata/events{/privacy}","received_events_url":"https://api.github.com/users/citusdata/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/citusdata/citus","description":"Scalable PostgreSQL for real-time workloads","fork":false,"url":"https://api.github.com/repos/citusdata/citus","forks_url":"https://api.github.com/repos/citusdata/citus/forks","keys_url":"https://api.github.com/repos/citusdata/citus/keys{/key_id}","collaborators_url":"https://api.github.com/repos/citusdata/citus/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/citusdata/citus/teams","hooks_url":"https://api.github.com/repos/citusdata/citus/hooks","issue_events_url":"https://api.github.com/repos/citusdata/citus/issues/events{/number}","events_url":"https://api.github.com/repos/citusdata/citus/events","assignees_url":"https://api.github.com/repos/citusdata/citus/assignees{/user}","branches_url":"https://api.github.com/repos/citusdata/citus/branches{/branch}","tags_url":"https://api.github.com/repos/citusdata/citus/tags","blobs_url":"https://api.github.com/repos/citusdata/citus/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/citusdata/citus/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/citusdata/citus/git/refs{/sha}","trees_url":"https://api.github.com/repos/citusdata/citus/git/trees{/sha}","statuses_url":"https://api.github.com/repos/citusdata/citus/statuses/{sha}","languages_url":"https://api.github.com/repos/citusdata/citus/languages","stargazers_url":"https://api.github.com/repos/citusdata/citus/stargazers","contributors_url":"https://api.github.com/repos/citusdata/citus/contributors","subscribers_url":"https://api.github.com/repos/citusdata/citus/subscribers","subscription_url":"https://api.github.com/repos/citusdata/citus/subscription","commits_url":"https://api.github.com/repos/citusdata/citus/commits{/sha}","git_commits_url":"https://api.github.com/repos/citusdata/citus/git/commits{/sha}","comments_url":"https://api.github.com/repos/citusdata/citus/comments{/number}","issue_comment_url":"https://api.github.com/repos/citusdata/citus/issues/comments{/number}","contents_url":"https://api.github.com/repos/citusdata/citus/contents/{+path}","compare_url":"https://api.github.com/repos/citusdata/citus/compare/{base}...{head}","merges_url":"https://api.github.com/repos/citusdata/citus/merges","archive_url":"https://api.github.com/repos/citusdata/citus/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/citusdata/citus/downloads","issues_url":"https://api.github.com/repos/citusdata/citus/issues{/number}","pulls_url":"https://api.github.com/repos/citusdata/citus/pulls{/number}","milestones_url":"https://api.github.com/repos/citusdata/citus/milestones{/number}","notifications_url":"https://api.github.com/repos/citusdata/citus/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/citusdata/citus/labels{/name}","releases_url":"https://api.github.com/repos/citusdata/citus/releases{/id}","deployments_url":"https://api.github.com/repos/citusdata/citus/deployments","created_at":"2016-02-01T21:58:39Z","updated_at":"2017-04-12T10:03:24Z","pushed_at":"2017-04-12T11:56:59Z","git_url":"git://github.com/citusdata/citus.git","ssh_url":"git@github.com:citusdata/citus.git","clone_url":"https://github.com/citusdata/citus.git","svn_url":"https://github.com/citusdata/citus","homepage":"https://www.citusdata.com","size":6300,"stargazers_count":1964,"watchers_count":1964,"language":"C","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":138,"mirror_url":null,"open_issues_count":451,"forks":138,"open_issues":451,"watchers":1964,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/citusdata/citus/pulls/1289"},"html":{"href":"https://github.com/citusdata/citus/pull/1289"},"issue":{"href":"https://api.github.com/repos/citusdata/citus/issues/1289"},"comments":{"href":"https://api.github.com/repos/citusdata/citus/issues/1289/comments"},"review_comments":{"href":"https://api.github.com/repos/citusdata/citus/pulls/1289/comments"},"review_comment":{"href":"https://api.github.com/repos/citusdata/citus/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/citusdata/citus/pulls/1289/commits"},"statuses":{"href":"https://api.github.com/repos/citusdata/citus/statuses/b1c4570abcedbdac0e48a9e0d16321cbe1254bb0"}}}}
{"actor":{"display_login":"onderkalaci"}}