{"action":"created","comment":{"url":"https://api.github.com/repos/basho/riak_kv/pulls/comments/95151556","pull_request_review_id":15696597,"id":95151556,"diff_hunk":"@@ -506,6 +487,26 @@ prepare_final_results(#state{\n             cancel_error_query(divide_by_zero, State)\n     end.\n \n+maybe_apply_offset_limit(?SQL_SELECT{'OFFSET' = [], 'LIMIT' = []}, Rows) ->\n+    Rows;\n+maybe_apply_offset_limit(?SQL_SELECT{'OFFSET' = [Offset], 'LIMIT' = []}, Rows) when is_integer(Offset), Offset >= 0 ->\n+    safe_nthtail(Offset, Rows);\n+maybe_apply_offset_limit(?SQL_SELECT{'OFFSET' = [], 'LIMIT' = [Limit]}, Rows) when is_integer(Limit), Limit >= 0 ->\n+    lists:sublist(Rows, Limit);\n+maybe_apply_offset_limit(?SQL_SELECT{'OFFSET' = [Offset], 'LIMIT' = [Limit]}, Rows) when is_integer(Offset), is_integer(Limit), Offset >= 0, Limit >= 0 ->\n+    lists:sublist(safe_nthtail(Offset, Rows), Limit);","path":"src/riak_kv_qry_worker.erl","position":59,"original_position":59,"commit_id":"9e0b0c45b076c9fc603f34166e9f0bc55a6fad33","original_commit_id":"9e0b0c45b076c9fc603f34166e9f0bc55a6fad33","user":{"login":"hmmr","id":1107079,"avatar_url":"https://avatars.githubusercontent.com/u/1107079?v=3","gravatar_id":"","url":"https://api.github.com/users/hmmr","html_url":"https://github.com/hmmr","followers_url":"https://api.github.com/users/hmmr/followers","following_url":"https://api.github.com/users/hmmr/following{/other_user}","gists_url":"https://api.github.com/users/hmmr/gists{/gist_id}","starred_url":"https://api.github.com/users/hmmr/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hmmr/subscriptions","organizations_url":"https://api.github.com/users/hmmr/orgs","repos_url":"https://api.github.com/users/hmmr/repos","events_url":"https://api.github.com/users/hmmr/events{/privacy}","received_events_url":"https://api.github.com/users/hmmr/received_events","type":"User","site_admin":false},"body":"The first paragraph in the comment above is me being inattentive. Sorting is not done on records within chunks, which I managed managed to miss.","created_at":"2017-01-09T13:07:18Z","updated_at":"2017-01-09T13:07:18Z","html_url":"https://github.com/basho/riak_kv/pull/1589#discussion_r95151556","pull_request_url":"https://api.github.com/repos/basho/riak_kv/pulls/1589","_links":{"self":{"href":"https://api.github.com/repos/basho/riak_kv/pulls/comments/95151556"},"html":{"href":"https://github.com/basho/riak_kv/pull/1589#discussion_r95151556"},"pull_request":{"href":"https://api.github.com/repos/basho/riak_kv/pulls/1589"}}},"pull_request":{"url":"https://api.github.com/repos/basho/riak_kv/pulls/1589","id":99969136,"html_url":"https://github.com/basho/riak_kv/pull/1589","diff_url":"https://github.com/basho/riak_kv/pull/1589.diff","patch_url":"https://github.com/basho/riak_kv/pull/1589.patch","issue_url":"https://api.github.com/repos/basho/riak_kv/issues/1589","number":1589,"state":"open","locked":false,"title":"In-memory SQL `LIMIT`","user":{"login":"andytill","id":213455,"avatar_url":"https://avatars.githubusercontent.com/u/213455?v=3","gravatar_id":"","url":"https://api.github.com/users/andytill","html_url":"https://github.com/andytill","followers_url":"https://api.github.com/users/andytill/followers","following_url":"https://api.github.com/users/andytill/following{/other_user}","gists_url":"https://api.github.com/users/andytill/gists{/gist_id}","starred_url":"https://api.github.com/users/andytill/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/andytill/subscriptions","organizations_url":"https://api.github.com/users/andytill/orgs","repos_url":"https://api.github.com/users/andytill/repos","events_url":"https://api.github.com/users/andytill/events{/privacy}","received_events_url":"https://api.github.com/users/andytill/received_events","type":"User","site_admin":false},"body":"When a query contains a `LIMIT` clause and optionally an `OFFSET` but no `ORDER BY` then Riak TS no longer creates a temporary table.\r\n\r\nIt now collects the results as it would for a normal query and then uses the `OFFSET` and `LIMIT` if provided to take a slice of the query results. For queries containing `LIMIT` but not `ORDER BY`, latency is reduced from 30x to 50x to 10%. The increase is measured from a query not containing a `LIMIT` clause.\r\n\r\nPerformance tests with eleveldb `LIMIT`:\r\n\r\n![image](https://cloud.githubusercontent.com/assets/213455/21612094/a0d8fd34-d1c7-11e6-93bf-266bb371c58e.png)\r\n\r\nPerformance tests with in-memory `LIMIT`:\r\n\r\n![image](https://cloud.githubusercontent.com/assets/213455/21612120/c3b72ace-d1c7-11e6-93cb-15ced9d55641.png)\r\n\r\nThis does not affect queries that have `ORDER BY` and `LIMIT` clauses.\r\n\r\nThe function head `riak_kv_qry_worker:estimate_query_size/1` which checks for a `LIMIT` has been removed. This has an existing bug where it is multiplying `LIMIT` by the chunk size, meaning that the estimated result size would be much larger than what it would otherwise calculate and making it more likely to cancel the query.\r\n","created_at":"2017-01-03T15:16:07Z","updated_at":"2017-01-09T13:07:18Z","closed_at":null,"merged_at":null,"merge_commit_sha":"d57f066aaf0ab3b314b724e75d2689f4237d6bb8","assignee":null,"assignees":[],"milestone":null,"commits_url":"https://api.github.com/repos/basho/riak_kv/pulls/1589/commits","review_comments_url":"https://api.github.com/repos/basho/riak_kv/pulls/1589/comments","review_comment_url":"https://api.github.com/repos/basho/riak_kv/pulls/comments{/number}","comments_url":"https://api.github.com/repos/basho/riak_kv/issues/1589/comments","statuses_url":"https://api.github.com/repos/basho/riak_kv/statuses/9e0b0c45b076c9fc603f34166e9f0bc55a6fad33","head":{"label":"basho:at-memory-limit","ref":"at-memory-limit","sha":"9e0b0c45b076c9fc603f34166e9f0bc55a6fad33","user":{"login":"basho","id":176293,"avatar_url":"https://avatars.githubusercontent.com/u/176293?v=3","gravatar_id":"","url":"https://api.github.com/users/basho","html_url":"https://github.com/basho","followers_url":"https://api.github.com/users/basho/followers","following_url":"https://api.github.com/users/basho/following{/other_user}","gists_url":"https://api.github.com/users/basho/gists{/gist_id}","starred_url":"https://api.github.com/users/basho/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/basho/subscriptions","organizations_url":"https://api.github.com/users/basho/orgs","repos_url":"https://api.github.com/users/basho/repos","events_url":"https://api.github.com/users/basho/events{/privacy}","received_events_url":"https://api.github.com/users/basho/received_events","type":"Organization","site_admin":false},"repo":{"id":791522,"name":"riak_kv","full_name":"basho/riak_kv","owner":{"login":"basho","id":176293,"avatar_url":"https://avatars.githubusercontent.com/u/176293?v=3","gravatar_id":"","url":"https://api.github.com/users/basho","html_url":"https://github.com/basho","followers_url":"https://api.github.com/users/basho/followers","following_url":"https://api.github.com/users/basho/following{/other_user}","gists_url":"https://api.github.com/users/basho/gists{/gist_id}","starred_url":"https://api.github.com/users/basho/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/basho/subscriptions","organizations_url":"https://api.github.com/users/basho/orgs","repos_url":"https://api.github.com/users/basho/repos","events_url":"https://api.github.com/users/basho/events{/privacy}","received_events_url":"https://api.github.com/users/basho/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/basho/riak_kv","description":"Riak Key/Value Store","fork":false,"url":"https://api.github.com/repos/basho/riak_kv","forks_url":"https://api.github.com/repos/basho/riak_kv/forks","keys_url":"https://api.github.com/repos/basho/riak_kv/keys{/key_id}","collaborators_url":"https://api.github.com/repos/basho/riak_kv/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/basho/riak_kv/teams","hooks_url":"https://api.github.com/repos/basho/riak_kv/hooks","issue_events_url":"https://api.github.com/repos/basho/riak_kv/issues/events{/number}","events_url":"https://api.github.com/repos/basho/riak_kv/events","assignees_url":"https://api.github.com/repos/basho/riak_kv/assignees{/user}","branches_url":"https://api.github.com/repos/basho/riak_kv/branches{/branch}","tags_url":"https://api.github.com/repos/basho/riak_kv/tags","blobs_url":"https://api.github.com/repos/basho/riak_kv/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/basho/riak_kv/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/basho/riak_kv/git/refs{/sha}","trees_url":"https://api.github.com/repos/basho/riak_kv/git/trees{/sha}","statuses_url":"https://api.github.com/repos/basho/riak_kv/statuses/{sha}","languages_url":"https://api.github.com/repos/basho/riak_kv/languages","stargazers_url":"https://api.github.com/repos/basho/riak_kv/stargazers","contributors_url":"https://api.github.com/repos/basho/riak_kv/contributors","subscribers_url":"https://api.github.com/repos/basho/riak_kv/subscribers","subscription_url":"https://api.github.com/repos/basho/riak_kv/subscription","commits_url":"https://api.github.com/repos/basho/riak_kv/commits{/sha}","git_commits_url":"https://api.github.com/repos/basho/riak_kv/git/commits{/sha}","comments_url":"https://api.github.com/repos/basho/riak_kv/comments{/number}","issue_comment_url":"https://api.github.com/repos/basho/riak_kv/issues/comments{/number}","contents_url":"https://api.github.com/repos/basho/riak_kv/contents/{+path}","compare_url":"https://api.github.com/repos/basho/riak_kv/compare/{base}...{head}","merges_url":"https://api.github.com/repos/basho/riak_kv/merges","archive_url":"https://api.github.com/repos/basho/riak_kv/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/basho/riak_kv/downloads","issues_url":"https://api.github.com/repos/basho/riak_kv/issues{/number}","pulls_url":"https://api.github.com/repos/basho/riak_kv/pulls{/number}","milestones_url":"https://api.github.com/repos/basho/riak_kv/milestones{/number}","notifications_url":"https://api.github.com/repos/basho/riak_kv/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/basho/riak_kv/labels{/name}","releases_url":"https://api.github.com/repos/basho/riak_kv/releases{/id}","deployments_url":"https://api.github.com/repos/basho/riak_kv/deployments","created_at":"2010-07-22T17:27:35Z","updated_at":"2016-12-10T12:39:12Z","pushed_at":"2017-01-09T12:28:09Z","git_url":"git://github.com/basho/riak_kv.git","ssh_url":"git@github.com:basho/riak_kv.git","clone_url":"https://github.com/basho/riak_kv.git","svn_url":"https://github.com/basho/riak_kv","homepage":"","size":13129,"stargazers_count":351,"watchers_count":351,"language":"Erlang","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":139,"mirror_url":null,"open_issues_count":181,"forks":139,"open_issues":181,"watchers":351,"default_branch":"develop"}},"base":{"label":"basho:riak_ts-develop","ref":"riak_ts-develop","sha":"87f033887a1aabf8f6033e5d595afa1e1e2ce6df","user":{"login":"basho","id":176293,"avatar_url":"https://avatars.githubusercontent.com/u/176293?v=3","gravatar_id":"","url":"https://api.github.com/users/basho","html_url":"https://github.com/basho","followers_url":"https://api.github.com/users/basho/followers","following_url":"https://api.github.com/users/basho/following{/other_user}","gists_url":"https://api.github.com/users/basho/gists{/gist_id}","starred_url":"https://api.github.com/users/basho/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/basho/subscriptions","organizations_url":"https://api.github.com/users/basho/orgs","repos_url":"https://api.github.com/users/basho/repos","events_url":"https://api.github.com/users/basho/events{/privacy}","received_events_url":"https://api.github.com/users/basho/received_events","type":"Organization","site_admin":false},"repo":{"id":791522,"name":"riak_kv","full_name":"basho/riak_kv","owner":{"login":"basho","id":176293,"avatar_url":"https://avatars.githubusercontent.com/u/176293?v=3","gravatar_id":"","url":"https://api.github.com/users/basho","html_url":"https://github.com/basho","followers_url":"https://api.github.com/users/basho/followers","following_url":"https://api.github.com/users/basho/following{/other_user}","gists_url":"https://api.github.com/users/basho/gists{/gist_id}","starred_url":"https://api.github.com/users/basho/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/basho/subscriptions","organizations_url":"https://api.github.com/users/basho/orgs","repos_url":"https://api.github.com/users/basho/repos","events_url":"https://api.github.com/users/basho/events{/privacy}","received_events_url":"https://api.github.com/users/basho/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/basho/riak_kv","description":"Riak Key/Value Store","fork":false,"url":"https://api.github.com/repos/basho/riak_kv","forks_url":"https://api.github.com/repos/basho/riak_kv/forks","keys_url":"https://api.github.com/repos/basho/riak_kv/keys{/key_id}","collaborators_url":"https://api.github.com/repos/basho/riak_kv/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/basho/riak_kv/teams","hooks_url":"https://api.github.com/repos/basho/riak_kv/hooks","issue_events_url":"https://api.github.com/repos/basho/riak_kv/issues/events{/number}","events_url":"https://api.github.com/repos/basho/riak_kv/events","assignees_url":"https://api.github.com/repos/basho/riak_kv/assignees{/user}","branches_url":"https://api.github.com/repos/basho/riak_kv/branches{/branch}","tags_url":"https://api.github.com/repos/basho/riak_kv/tags","blobs_url":"https://api.github.com/repos/basho/riak_kv/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/basho/riak_kv/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/basho/riak_kv/git/refs{/sha}","trees_url":"https://api.github.com/repos/basho/riak_kv/git/trees{/sha}","statuses_url":"https://api.github.com/repos/basho/riak_kv/statuses/{sha}","languages_url":"https://api.github.com/repos/basho/riak_kv/languages","stargazers_url":"https://api.github.com/repos/basho/riak_kv/stargazers","contributors_url":"https://api.github.com/repos/basho/riak_kv/contributors","subscribers_url":"https://api.github.com/repos/basho/riak_kv/subscribers","subscription_url":"https://api.github.com/repos/basho/riak_kv/subscription","commits_url":"https://api.github.com/repos/basho/riak_kv/commits{/sha}","git_commits_url":"https://api.github.com/repos/basho/riak_kv/git/commits{/sha}","comments_url":"https://api.github.com/repos/basho/riak_kv/comments{/number}","issue_comment_url":"https://api.github.com/repos/basho/riak_kv/issues/comments{/number}","contents_url":"https://api.github.com/repos/basho/riak_kv/contents/{+path}","compare_url":"https://api.github.com/repos/basho/riak_kv/compare/{base}...{head}","merges_url":"https://api.github.com/repos/basho/riak_kv/merges","archive_url":"https://api.github.com/repos/basho/riak_kv/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/basho/riak_kv/downloads","issues_url":"https://api.github.com/repos/basho/riak_kv/issues{/number}","pulls_url":"https://api.github.com/repos/basho/riak_kv/pulls{/number}","milestones_url":"https://api.github.com/repos/basho/riak_kv/milestones{/number}","notifications_url":"https://api.github.com/repos/basho/riak_kv/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/basho/riak_kv/labels{/name}","releases_url":"https://api.github.com/repos/basho/riak_kv/releases{/id}","deployments_url":"https://api.github.com/repos/basho/riak_kv/deployments","created_at":"2010-07-22T17:27:35Z","updated_at":"2016-12-10T12:39:12Z","pushed_at":"2017-01-09T12:28:09Z","git_url":"git://github.com/basho/riak_kv.git","ssh_url":"git@github.com:basho/riak_kv.git","clone_url":"https://github.com/basho/riak_kv.git","svn_url":"https://github.com/basho/riak_kv","homepage":"","size":13129,"stargazers_count":351,"watchers_count":351,"language":"Erlang","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":139,"mirror_url":null,"open_issues_count":181,"forks":139,"open_issues":181,"watchers":351,"default_branch":"develop"}},"_links":{"self":{"href":"https://api.github.com/repos/basho/riak_kv/pulls/1589"},"html":{"href":"https://github.com/basho/riak_kv/pull/1589"},"issue":{"href":"https://api.github.com/repos/basho/riak_kv/issues/1589"},"comments":{"href":"https://api.github.com/repos/basho/riak_kv/issues/1589/comments"},"review_comments":{"href":"https://api.github.com/repos/basho/riak_kv/pulls/1589/comments"},"review_comment":{"href":"https://api.github.com/repos/basho/riak_kv/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/basho/riak_kv/pulls/1589/commits"},"statuses":{"href":"https://api.github.com/repos/basho/riak_kv/statuses/9e0b0c45b076c9fc603f34166e9f0bc55a6fad33"}}}}
{"actor":{"display_login":"hmmr"}}