{"action":"created","comment":{"url":"https://api.github.com/repos/fabxc/tsdb/pulls/comments/107370995","pull_request_review_id":28333554,"id":107370995,"diff_hunk":"@@ -188,6 +188,25 @@ func Open(dir string, l log.Logger, r prometheus.Registerer, opts *Options) (db\n \treturn db, nil\n }\n \n+// QueryLatest will return the matching series with their latest value.\n+// Useful for federation.\n+func (db *DB) QueryLatest(cutoff int64, matchers ...labels.Matcher) []SeriesLatest {\n+\tdb.headmtx.RLock()\n+\tdefer db.headmtx.RUnlock()\n+\n+\tsl := make([]SeriesLatest, 0)\n+\tfor _, h := range db.heads {\n+\t\tm := h.Meta()\n+\t\tif m.MaxTime < cutoff {\n+\t\t\tcontinue\n+\t\t}\n+\n+\t\tsl = append(sl, h.SelectLatest(cutoff, matchers...)...)\n+\t}","path":"db.go","position":18,"original_position":18,"commit_id":"8557582074cf8efa63fc622b9df292ac28be14ff","original_commit_id":"8557582074cf8efa63fc622b9df292ac28be14ff","user":{"login":"fabxc","id":4948210,"avatar_url":"https://avatars1.githubusercontent.com/u/4948210?v=3","gravatar_id":"","url":"https://api.github.com/users/fabxc","html_url":"https://github.com/fabxc","followers_url":"https://api.github.com/users/fabxc/followers","following_url":"https://api.github.com/users/fabxc/following{/other_user}","gists_url":"https://api.github.com/users/fabxc/gists{/gist_id}","starred_url":"https://api.github.com/users/fabxc/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/fabxc/subscriptions","organizations_url":"https://api.github.com/users/fabxc/orgs","repos_url":"https://api.github.com/users/fabxc/repos","events_url":"https://api.github.com/users/fabxc/events{/privacy}","received_events_url":"https://api.github.com/users/fabxc/received_events","type":"User","site_admin":false},"body":"This logically doesn't seem correct.\r\n\r\nThe `heads` list is _not_ sharded across all our series but represents blocks sequential in time, each covering the entire series space.\r\nWhat we do here is checking every head for the last sample, which generally means we get several samples for the same series, and throwing them all into a big result.\r\n\r\nWe have to first consider only heads with `m.MaxTime >= cutoff` as you correctly do here.\r\nBut then we have to start from the last block, look which series exist there, and take their last sample.\r\nThen go to the block before that and do the same, but only add those series and samples to the result, which are not already in our result so far. Repeat for all considered blocks.\r\n\r\nGoing with slices here is fine for sketching things out, but we will have to make this all iterator based.\r\nEspecially if we want to \"merge with priority\" across blocks, with sorted iterators we can do so in O(n) whereas the naive approach will be quadratic, cubic, etc... depending on how many heads have to be considered.","created_at":"2017-03-22T09:55:50Z","updated_at":"2017-03-22T09:56:11Z","html_url":"https://github.com/fabxc/tsdb/pull/17#discussion_r107370995","pull_request_url":"https://api.github.com/repos/fabxc/tsdb/pulls/17","_links":{"self":{"href":"https://api.github.com/repos/fabxc/tsdb/pulls/comments/107370995"},"html":{"href":"https://github.com/fabxc/tsdb/pull/17#discussion_r107370995"},"pull_request":{"href":"https://api.github.com/repos/fabxc/tsdb/pulls/17"}}},"pull_request":{"url":"https://api.github.com/repos/fabxc/tsdb/pulls/17","id":111843219,"html_url":"https://github.com/fabxc/tsdb/pull/17","diff_url":"https://github.com/fabxc/tsdb/pull/17.diff","patch_url":"https://github.com/fabxc/tsdb/pull/17.patch","issue_url":"https://api.github.com/repos/fabxc/tsdb/issues/17","number":17,"state":"open","locked":false,"title":"[WIP]: Add A Method For Latest Value Per Series","user":{"login":"Gouthamve","id":7354143,"avatar_url":"https://avatars3.githubusercontent.com/u/7354143?v=3","gravatar_id":"","url":"https://api.github.com/users/Gouthamve","html_url":"https://github.com/Gouthamve","followers_url":"https://api.github.com/users/Gouthamve/followers","following_url":"https://api.github.com/users/Gouthamve/following{/other_user}","gists_url":"https://api.github.com/users/Gouthamve/gists{/gist_id}","starred_url":"https://api.github.com/users/Gouthamve/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Gouthamve/subscriptions","organizations_url":"https://api.github.com/users/Gouthamve/orgs","repos_url":"https://api.github.com/users/Gouthamve/repos","events_url":"https://api.github.com/users/Gouthamve/events{/privacy}","received_events_url":"https://api.github.com/users/Gouthamve/received_events","type":"User","site_admin":false},"body":"This is a very rough stab at #9. \r\n\r\nThis is to get my head around the code-base and will primarily be used for benchmarking the need for the latest value retrieval.\r\n\r\n@fabxc I have no clue if this works or not :) Need some pointers as to how I can test this.","created_at":"2017-03-21T18:02:02Z","updated_at":"2017-03-22T09:56:11Z","closed_at":null,"merged_at":null,"merge_commit_sha":"56fcf42365a716f0dbe436d9f57bc424a6e2dcfc","assignee":null,"assignees":[],"milestone":null,"commits_url":"https://api.github.com/repos/fabxc/tsdb/pulls/17/commits","review_comments_url":"https://api.github.com/repos/fabxc/tsdb/pulls/17/comments","review_comment_url":"https://api.github.com/repos/fabxc/tsdb/pulls/comments{/number}","comments_url":"https://api.github.com/repos/fabxc/tsdb/issues/17/comments","statuses_url":"https://api.github.com/repos/fabxc/tsdb/statuses/8557582074cf8efa63fc622b9df292ac28be14ff","head":{"label":"Gouthamve:latest-val","ref":"latest-val","sha":"8557582074cf8efa63fc622b9df292ac28be14ff","user":{"login":"Gouthamve","id":7354143,"avatar_url":"https://avatars3.githubusercontent.com/u/7354143?v=3","gravatar_id":"","url":"https://api.github.com/users/Gouthamve","html_url":"https://github.com/Gouthamve","followers_url":"https://api.github.com/users/Gouthamve/followers","following_url":"https://api.github.com/users/Gouthamve/following{/other_user}","gists_url":"https://api.github.com/users/Gouthamve/gists{/gist_id}","starred_url":"https://api.github.com/users/Gouthamve/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Gouthamve/subscriptions","organizations_url":"https://api.github.com/users/Gouthamve/orgs","repos_url":"https://api.github.com/users/Gouthamve/repos","events_url":"https://api.github.com/users/Gouthamve/events{/privacy}","received_events_url":"https://api.github.com/users/Gouthamve/received_events","type":"User","site_admin":false},"repo":{"id":85487421,"name":"tsdb","full_name":"Gouthamve/tsdb","owner":{"login":"Gouthamve","id":7354143,"avatar_url":"https://avatars3.githubusercontent.com/u/7354143?v=3","gravatar_id":"","url":"https://api.github.com/users/Gouthamve","html_url":"https://github.com/Gouthamve","followers_url":"https://api.github.com/users/Gouthamve/followers","following_url":"https://api.github.com/users/Gouthamve/following{/other_user}","gists_url":"https://api.github.com/users/Gouthamve/gists{/gist_id}","starred_url":"https://api.github.com/users/Gouthamve/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Gouthamve/subscriptions","organizations_url":"https://api.github.com/users/Gouthamve/orgs","repos_url":"https://api.github.com/users/Gouthamve/repos","events_url":"https://api.github.com/users/Gouthamve/events{/privacy}","received_events_url":"https://api.github.com/users/Gouthamve/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/Gouthamve/tsdb","description":null,"fork":true,"url":"https://api.github.com/repos/Gouthamve/tsdb","forks_url":"https://api.github.com/repos/Gouthamve/tsdb/forks","keys_url":"https://api.github.com/repos/Gouthamve/tsdb/keys{/key_id}","collaborators_url":"https://api.github.com/repos/Gouthamve/tsdb/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/Gouthamve/tsdb/teams","hooks_url":"https://api.github.com/repos/Gouthamve/tsdb/hooks","issue_events_url":"https://api.github.com/repos/Gouthamve/tsdb/issues/events{/number}","events_url":"https://api.github.com/repos/Gouthamve/tsdb/events","assignees_url":"https://api.github.com/repos/Gouthamve/tsdb/assignees{/user}","branches_url":"https://api.github.com/repos/Gouthamve/tsdb/branches{/branch}","tags_url":"https://api.github.com/repos/Gouthamve/tsdb/tags","blobs_url":"https://api.github.com/repos/Gouthamve/tsdb/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/Gouthamve/tsdb/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/Gouthamve/tsdb/git/refs{/sha}","trees_url":"https://api.github.com/repos/Gouthamve/tsdb/git/trees{/sha}","statuses_url":"https://api.github.com/repos/Gouthamve/tsdb/statuses/{sha}","languages_url":"https://api.github.com/repos/Gouthamve/tsdb/languages","stargazers_url":"https://api.github.com/repos/Gouthamve/tsdb/stargazers","contributors_url":"https://api.github.com/repos/Gouthamve/tsdb/contributors","subscribers_url":"https://api.github.com/repos/Gouthamve/tsdb/subscribers","subscription_url":"https://api.github.com/repos/Gouthamve/tsdb/subscription","commits_url":"https://api.github.com/repos/Gouthamve/tsdb/commits{/sha}","git_commits_url":"https://api.github.com/repos/Gouthamve/tsdb/git/commits{/sha}","comments_url":"https://api.github.com/repos/Gouthamve/tsdb/comments{/number}","issue_comment_url":"https://api.github.com/repos/Gouthamve/tsdb/issues/comments{/number}","contents_url":"https://api.github.com/repos/Gouthamve/tsdb/contents/{+path}","compare_url":"https://api.github.com/repos/Gouthamve/tsdb/compare/{base}...{head}","merges_url":"https://api.github.com/repos/Gouthamve/tsdb/merges","archive_url":"https://api.github.com/repos/Gouthamve/tsdb/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/Gouthamve/tsdb/downloads","issues_url":"https://api.github.com/repos/Gouthamve/tsdb/issues{/number}","pulls_url":"https://api.github.com/repos/Gouthamve/tsdb/pulls{/number}","milestones_url":"https://api.github.com/repos/Gouthamve/tsdb/milestones{/number}","notifications_url":"https://api.github.com/repos/Gouthamve/tsdb/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/Gouthamve/tsdb/labels{/name}","releases_url":"https://api.github.com/repos/Gouthamve/tsdb/releases{/id}","deployments_url":"https://api.github.com/repos/Gouthamve/tsdb/deployments","created_at":"2017-03-19T15:33:44Z","updated_at":"2017-03-19T15:33:46Z","pushed_at":"2017-03-21T17:57:05Z","git_url":"git://github.com/Gouthamve/tsdb.git","ssh_url":"git@github.com:Gouthamve/tsdb.git","clone_url":"https://github.com/Gouthamve/tsdb.git","svn_url":"https://github.com/Gouthamve/tsdb","homepage":null,"size":2612,"stargazers_count":0,"watchers_count":0,"language":"Go","has_issues":false,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":0,"forks":0,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"fabxc:master","ref":"master","sha":"70909ca8ad875aa6f01e1801d185b3f3bab2fda3","user":{"login":"fabxc","id":4948210,"avatar_url":"https://avatars1.githubusercontent.com/u/4948210?v=3","gravatar_id":"","url":"https://api.github.com/users/fabxc","html_url":"https://github.com/fabxc","followers_url":"https://api.github.com/users/fabxc/followers","following_url":"https://api.github.com/users/fabxc/following{/other_user}","gists_url":"https://api.github.com/users/fabxc/gists{/gist_id}","starred_url":"https://api.github.com/users/fabxc/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/fabxc/subscriptions","organizations_url":"https://api.github.com/users/fabxc/orgs","repos_url":"https://api.github.com/users/fabxc/repos","events_url":"https://api.github.com/users/fabxc/events{/privacy}","received_events_url":"https://api.github.com/users/fabxc/received_events","type":"User","site_admin":false},"repo":{"id":73823302,"name":"tsdb","full_name":"fabxc/tsdb","owner":{"login":"fabxc","id":4948210,"avatar_url":"https://avatars1.githubusercontent.com/u/4948210?v=3","gravatar_id":"","url":"https://api.github.com/users/fabxc","html_url":"https://github.com/fabxc","followers_url":"https://api.github.com/users/fabxc/followers","following_url":"https://api.github.com/users/fabxc/following{/other_user}","gists_url":"https://api.github.com/users/fabxc/gists{/gist_id}","starred_url":"https://api.github.com/users/fabxc/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/fabxc/subscriptions","organizations_url":"https://api.github.com/users/fabxc/orgs","repos_url":"https://api.github.com/users/fabxc/repos","events_url":"https://api.github.com/users/fabxc/events{/privacy}","received_events_url":"https://api.github.com/users/fabxc/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/fabxc/tsdb","description":null,"fork":false,"url":"https://api.github.com/repos/fabxc/tsdb","forks_url":"https://api.github.com/repos/fabxc/tsdb/forks","keys_url":"https://api.github.com/repos/fabxc/tsdb/keys{/key_id}","collaborators_url":"https://api.github.com/repos/fabxc/tsdb/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/fabxc/tsdb/teams","hooks_url":"https://api.github.com/repos/fabxc/tsdb/hooks","issue_events_url":"https://api.github.com/repos/fabxc/tsdb/issues/events{/number}","events_url":"https://api.github.com/repos/fabxc/tsdb/events","assignees_url":"https://api.github.com/repos/fabxc/tsdb/assignees{/user}","branches_url":"https://api.github.com/repos/fabxc/tsdb/branches{/branch}","tags_url":"https://api.github.com/repos/fabxc/tsdb/tags","blobs_url":"https://api.github.com/repos/fabxc/tsdb/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/fabxc/tsdb/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/fabxc/tsdb/git/refs{/sha}","trees_url":"https://api.github.com/repos/fabxc/tsdb/git/trees{/sha}","statuses_url":"https://api.github.com/repos/fabxc/tsdb/statuses/{sha}","languages_url":"https://api.github.com/repos/fabxc/tsdb/languages","stargazers_url":"https://api.github.com/repos/fabxc/tsdb/stargazers","contributors_url":"https://api.github.com/repos/fabxc/tsdb/contributors","subscribers_url":"https://api.github.com/repos/fabxc/tsdb/subscribers","subscription_url":"https://api.github.com/repos/fabxc/tsdb/subscription","commits_url":"https://api.github.com/repos/fabxc/tsdb/commits{/sha}","git_commits_url":"https://api.github.com/repos/fabxc/tsdb/git/commits{/sha}","comments_url":"https://api.github.com/repos/fabxc/tsdb/comments{/number}","issue_comment_url":"https://api.github.com/repos/fabxc/tsdb/issues/comments{/number}","contents_url":"https://api.github.com/repos/fabxc/tsdb/contents/{+path}","compare_url":"https://api.github.com/repos/fabxc/tsdb/compare/{base}...{head}","merges_url":"https://api.github.com/repos/fabxc/tsdb/merges","archive_url":"https://api.github.com/repos/fabxc/tsdb/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/fabxc/tsdb/downloads","issues_url":"https://api.github.com/repos/fabxc/tsdb/issues{/number}","pulls_url":"https://api.github.com/repos/fabxc/tsdb/pulls{/number}","milestones_url":"https://api.github.com/repos/fabxc/tsdb/milestones{/number}","notifications_url":"https://api.github.com/repos/fabxc/tsdb/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/fabxc/tsdb/labels{/name}","releases_url":"https://api.github.com/repos/fabxc/tsdb/releases{/id}","deployments_url":"https://api.github.com/repos/fabxc/tsdb/deployments","created_at":"2016-11-15T14:50:03Z","updated_at":"2017-03-19T14:47:58Z","pushed_at":"2017-03-21T18:02:03Z","git_url":"git://github.com/fabxc/tsdb.git","ssh_url":"git@github.com:fabxc/tsdb.git","clone_url":"https://github.com/fabxc/tsdb.git","svn_url":"https://github.com/fabxc/tsdb","homepage":null,"size":2593,"stargazers_count":6,"watchers_count":6,"language":"Go","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":1,"mirror_url":null,"open_issues_count":14,"forks":1,"open_issues":14,"watchers":6,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/fabxc/tsdb/pulls/17"},"html":{"href":"https://github.com/fabxc/tsdb/pull/17"},"issue":{"href":"https://api.github.com/repos/fabxc/tsdb/issues/17"},"comments":{"href":"https://api.github.com/repos/fabxc/tsdb/issues/17/comments"},"review_comments":{"href":"https://api.github.com/repos/fabxc/tsdb/pulls/17/comments"},"review_comment":{"href":"https://api.github.com/repos/fabxc/tsdb/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/fabxc/tsdb/pulls/17/commits"},"statuses":{"href":"https://api.github.com/repos/fabxc/tsdb/statuses/8557582074cf8efa63fc622b9df292ac28be14ff"}}}}
{"actor":{"display_login":"fabxc"}}