{"action":"created","issue":{"url":"https://api.github.com/repos/attic-labs/noms/issues/2942","repository_url":"https://api.github.com/repos/attic-labs/noms","labels_url":"https://api.github.com/repos/attic-labs/noms/issues/2942/labels{/name}","comments_url":"https://api.github.com/repos/attic-labs/noms/issues/2942/comments","events_url":"https://api.github.com/repos/attic-labs/noms/issues/2942/events","html_url":"https://github.com/attic-labs/noms/issues/2942","id":195234891,"number":2942,"title":"Go read-ahead is limited by the size of the N+1 cursor sequence","user":{"login":"rafael-atticlabs","id":12848669,"avatar_url":"https://avatars2.githubusercontent.com/u/12848669?v=3","gravatar_id":"","url":"https://api.github.com/users/rafael-atticlabs","html_url":"https://github.com/rafael-atticlabs","followers_url":"https://api.github.com/users/rafael-atticlabs/followers","following_url":"https://api.github.com/users/rafael-atticlabs/following{/other_user}","gists_url":"https://api.github.com/users/rafael-atticlabs/gists{/gist_id}","starred_url":"https://api.github.com/users/rafael-atticlabs/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rafael-atticlabs/subscriptions","organizations_url":"https://api.github.com/users/rafael-atticlabs/orgs","repos_url":"https://api.github.com/users/rafael-atticlabs/repos","events_url":"https://api.github.com/users/rafael-atticlabs/events{/privacy}","received_events_url":"https://api.github.com/users/rafael-atticlabs/received_events","type":"User","site_admin":false},"labels":[{"id":335117297,"url":"https://api.github.com/repos/attic-labs/noms/labels/P0","name":"P0","color":"e11d21","default":false},{"id":379950681,"url":"https://api.github.com/repos/attic-labs/noms/labels/Perf","name":"Perf","color":"ff8000","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":1,"created_at":"2016-12-13T12:00:49Z","updated_at":"2017-04-11T00:18:55Z","closed_at":null,"body":"The current read-ahead approach creates a second cursor to read ahead, but because it relies on advance(), it is blocked at the end of each leaf chunk on it's parent loading the next group of leaves. in practice, this means that it can never read ahead more than about ~150 leaf chunks (or ~500k). It experiments, I was able to see read rates nearing that of the underlying disk when reading ahead closer to 8 to 16MB.\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/attic-labs/noms/issues/comments/293114491","html_url":"https://github.com/attic-labs/noms/issues/2942#issuecomment-293114491","issue_url":"https://api.github.com/repos/attic-labs/noms/issues/2942","id":293114491,"user":{"login":"rafael-atticlabs","id":12848669,"avatar_url":"https://avatars2.githubusercontent.com/u/12848669?v=3","gravatar_id":"","url":"https://api.github.com/users/rafael-atticlabs","html_url":"https://github.com/rafael-atticlabs","followers_url":"https://api.github.com/users/rafael-atticlabs/followers","following_url":"https://api.github.com/users/rafael-atticlabs/following{/other_user}","gists_url":"https://api.github.com/users/rafael-atticlabs/gists{/gist_id}","starred_url":"https://api.github.com/users/rafael-atticlabs/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rafael-atticlabs/subscriptions","organizations_url":"https://api.github.com/users/rafael-atticlabs/orgs","repos_url":"https://api.github.com/users/rafael-atticlabs/repos","events_url":"https://api.github.com/users/rafael-atticlabs/events{/privacy}","received_events_url":"https://api.github.com/users/rafael-atticlabs/received_events","type":"User","site_admin":false},"created_at":"2017-04-11T00:18:55Z","updated_at":"2017-04-11T00:18:55Z","body":"To be more clear about this: the *distance* we read-ahead is measured as a constant number of meta-1 nodes. The problem with this is that meta nodes can contain a varying number of leaf references. If they contain 100-200, then we'll read ahead by something like 10MB, but they can contain *far* less. This isn't really a problem for reading large blob data, because the leaf nodes will tend to be fairly wide, but it could become a problem for map data if they keys are long.\r\n\r\nhttps://github.com/attic-labs/noms/issues/2763 would mitigate this problem some."}}
{"actor":{"display_login":"rafael-atticlabs"}}