{"action":"opened","number":2672,"pull_request":{"url":"https://api.github.com/repos/git-lfs/git-lfs/pulls/2672","id":147079085,"html_url":"https://github.com/git-lfs/git-lfs/pull/2672","diff_url":"https://github.com/git-lfs/git-lfs/pull/2672.diff","patch_url":"https://github.com/git-lfs/git-lfs/pull/2672.patch","issue_url":"https://api.github.com/repos/git-lfs/git-lfs/issues/2672","number":2672,"state":"open","locked":false,"title":"FastWalkGitRepo: limit number of concurrent goroutines","user":{"login":"technoweenie","id":21,"avatar_url":"https://avatars3.githubusercontent.com/u/21?v=4","gravatar_id":"","url":"https://api.github.com/users/technoweenie","html_url":"https://github.com/technoweenie","followers_url":"https://api.github.com/users/technoweenie/followers","following_url":"https://api.github.com/users/technoweenie/following{/other_user}","gists_url":"https://api.github.com/users/technoweenie/gists{/gist_id}","starred_url":"https://api.github.com/users/technoweenie/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/technoweenie/subscriptions","organizations_url":"https://api.github.com/users/technoweenie/orgs","repos_url":"https://api.github.com/users/technoweenie/repos","events_url":"https://api.github.com/users/technoweenie/events{/privacy}","received_events_url":"https://api.github.com/users/technoweenie/received_events","type":"User","site_admin":true},"body":"The `tools.FastWalkGitRepo()` mimics `filepath.Walk()`, with a few performance optimizations:\r\n\r\n* Subdirectories, in groups of up to 100, are walked in goroutines.\r\n* Files are not returned in any specific order.\r\n* `.gitignore` patterns are respected.\r\n\r\nThis PR introduces a limit to the concurrent goroutines that can be spawned, with an arbitrary default of 20x the number of CPUs that are available. That seems to be plenty, and well within system limits. It does mean that `tools.FastWalkGitRepo()` will slow down for particularly wide or deep directory structures in a repository, but hopefully it won't crash.\r\n\r\nFWIW, `tools.FastWalkGitRepo()` is used in the following commands:\r\n\r\n* `git lfs track`, when it's scanning your repository for nested `.gitattributes` files\r\n* `post-merge` and `post-checkout`, when LFS ensures lockable files not currently locked by you are marked readonly.\r\n\r\nI want to include this in LFS v2.3.4. A smaller experimental version of this patch is at https://github.com/git-lfs/git-lfs/compare/fastwalk-depth. I removed the debug messages, and went a little further extracting an internal struct type for a single `tools.FastWalkGitRepo()` call.","created_at":"2017-10-17T15:45:10Z","updated_at":"2017-10-17T15:45:10Z","closed_at":null,"merged_at":null,"merge_commit_sha":null,"assignee":null,"assignees":[],"requested_reviewers":[],"milestone":null,"commits_url":"https://api.github.com/repos/git-lfs/git-lfs/pulls/2672/commits","review_comments_url":"https://api.github.com/repos/git-lfs/git-lfs/pulls/2672/comments","review_comment_url":"https://api.github.com/repos/git-lfs/git-lfs/pulls/comments{/number}","comments_url":"https://api.github.com/repos/git-lfs/git-lfs/issues/2672/comments","statuses_url":"https://api.github.com/repos/git-lfs/git-lfs/statuses/6566be686416c97e7a9f958c92c46ec96e009bfe","head":{"label":"git-lfs:fastwalker-goroutine-limit","ref":"fastwalker-goroutine-limit","sha":"6566be686416c97e7a9f958c92c46ec96e009bfe","user":{"login":"git-lfs","id":20246716,"avatar_url":"https://avatars2.githubusercontent.com/u/20246716?v=4","gravatar_id":"","url":"https://api.github.com/users/git-lfs","html_url":"https://github.com/git-lfs","followers_url":"https://api.github.com/users/git-lfs/followers","following_url":"https://api.github.com/users/git-lfs/following{/other_user}","gists_url":"https://api.github.com/users/git-lfs/gists{/gist_id}","starred_url":"https://api.github.com/users/git-lfs/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/git-lfs/subscriptions","organizations_url":"https://api.github.com/users/git-lfs/orgs","repos_url":"https://api.github.com/users/git-lfs/repos","events_url":"https://api.github.com/users/git-lfs/events{/privacy}","received_events_url":"https://api.github.com/users/git-lfs/received_events","type":"Organization","site_admin":false},"repo":{"id":13021798,"name":"git-lfs","full_name":"git-lfs/git-lfs","owner":{"login":"git-lfs","id":20246716,"avatar_url":"https://avatars2.githubusercontent.com/u/20246716?v=4","gravatar_id":"","url":"https://api.github.com/users/git-lfs","html_url":"https://github.com/git-lfs","followers_url":"https://api.github.com/users/git-lfs/followers","following_url":"https://api.github.com/users/git-lfs/following{/other_user}","gists_url":"https://api.github.com/users/git-lfs/gists{/gist_id}","starred_url":"https://api.github.com/users/git-lfs/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/git-lfs/subscriptions","organizations_url":"https://api.github.com/users/git-lfs/orgs","repos_url":"https://api.github.com/users/git-lfs/repos","events_url":"https://api.github.com/users/git-lfs/events{/privacy}","received_events_url":"https://api.github.com/users/git-lfs/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/git-lfs/git-lfs","description":"Git extension for versioning large files","fork":false,"url":"https://api.github.com/repos/git-lfs/git-lfs","forks_url":"https://api.github.com/repos/git-lfs/git-lfs/forks","keys_url":"https://api.github.com/repos/git-lfs/git-lfs/keys{/key_id}","collaborators_url":"https://api.github.com/repos/git-lfs/git-lfs/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/git-lfs/git-lfs/teams","hooks_url":"https://api.github.com/repos/git-lfs/git-lfs/hooks","issue_events_url":"https://api.github.com/repos/git-lfs/git-lfs/issues/events{/number}","events_url":"https://api.github.com/repos/git-lfs/git-lfs/events","assignees_url":"https://api.github.com/repos/git-lfs/git-lfs/assignees{/user}","branches_url":"https://api.github.com/repos/git-lfs/git-lfs/branches{/branch}","tags_url":"https://api.github.com/repos/git-lfs/git-lfs/tags","blobs_url":"https://api.github.com/repos/git-lfs/git-lfs/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/git-lfs/git-lfs/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/git-lfs/git-lfs/git/refs{/sha}","trees_url":"https://api.github.com/repos/git-lfs/git-lfs/git/trees{/sha}","statuses_url":"https://api.github.com/repos/git-lfs/git-lfs/statuses/{sha}","languages_url":"https://api.github.com/repos/git-lfs/git-lfs/languages","stargazers_url":"https://api.github.com/repos/git-lfs/git-lfs/stargazers","contributors_url":"https://api.github.com/repos/git-lfs/git-lfs/contributors","subscribers_url":"https://api.github.com/repos/git-lfs/git-lfs/subscribers","subscription_url":"https://api.github.com/repos/git-lfs/git-lfs/subscription","commits_url":"https://api.github.com/repos/git-lfs/git-lfs/commits{/sha}","git_commits_url":"https://api.github.com/repos/git-lfs/git-lfs/git/commits{/sha}","comments_url":"https://api.github.com/repos/git-lfs/git-lfs/comments{/number}","issue_comment_url":"https://api.github.com/repos/git-lfs/git-lfs/issues/comments{/number}","contents_url":"https://api.github.com/repos/git-lfs/git-lfs/contents/{+path}","compare_url":"https://api.github.com/repos/git-lfs/git-lfs/compare/{base}...{head}","merges_url":"https://api.github.com/repos/git-lfs/git-lfs/merges","archive_url":"https://api.github.com/repos/git-lfs/git-lfs/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/git-lfs/git-lfs/downloads","issues_url":"https://api.github.com/repos/git-lfs/git-lfs/issues{/number}","pulls_url":"https://api.github.com/repos/git-lfs/git-lfs/pulls{/number}","milestones_url":"https://api.github.com/repos/git-lfs/git-lfs/milestones{/number}","notifications_url":"https://api.github.com/repos/git-lfs/git-lfs/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/git-lfs/git-lfs/labels{/name}","releases_url":"https://api.github.com/repos/git-lfs/git-lfs/releases{/id}","deployments_url":"https://api.github.com/repos/git-lfs/git-lfs/deployments","created_at":"2013-09-22T22:03:28Z","updated_at":"2017-10-17T14:07:58Z","pushed_at":"2017-10-15T03:43:13Z","git_url":"git://github.com/git-lfs/git-lfs.git","ssh_url":"git@github.com:git-lfs/git-lfs.git","clone_url":"https://github.com/git-lfs/git-lfs.git","svn_url":"https://github.com/git-lfs/git-lfs","homepage":"https://git-lfs.github.com","size":8792,"stargazers_count":5019,"watchers_count":5019,"language":"Go","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":863,"mirror_url":null,"open_issues_count":266,"forks":863,"open_issues":266,"watchers":5019,"default_branch":"master"}},"base":{"label":"git-lfs:master","ref":"master","sha":"150edd19ea727308e5b448b50d63f0c77967d3cd","user":{"login":"git-lfs","id":20246716,"avatar_url":"https://avatars2.githubusercontent.com/u/20246716?v=4","gravatar_id":"","url":"https://api.github.com/users/git-lfs","html_url":"https://github.com/git-lfs","followers_url":"https://api.github.com/users/git-lfs/followers","following_url":"https://api.github.com/users/git-lfs/following{/other_user}","gists_url":"https://api.github.com/users/git-lfs/gists{/gist_id}","starred_url":"https://api.github.com/users/git-lfs/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/git-lfs/subscriptions","organizations_url":"https://api.github.com/users/git-lfs/orgs","repos_url":"https://api.github.com/users/git-lfs/repos","events_url":"https://api.github.com/users/git-lfs/events{/privacy}","received_events_url":"https://api.github.com/users/git-lfs/received_events","type":"Organization","site_admin":false},"repo":{"id":13021798,"name":"git-lfs","full_name":"git-lfs/git-lfs","owner":{"login":"git-lfs","id":20246716,"avatar_url":"https://avatars2.githubusercontent.com/u/20246716?v=4","gravatar_id":"","url":"https://api.github.com/users/git-lfs","html_url":"https://github.com/git-lfs","followers_url":"https://api.github.com/users/git-lfs/followers","following_url":"https://api.github.com/users/git-lfs/following{/other_user}","gists_url":"https://api.github.com/users/git-lfs/gists{/gist_id}","starred_url":"https://api.github.com/users/git-lfs/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/git-lfs/subscriptions","organizations_url":"https://api.github.com/users/git-lfs/orgs","repos_url":"https://api.github.com/users/git-lfs/repos","events_url":"https://api.github.com/users/git-lfs/events{/privacy}","received_events_url":"https://api.github.com/users/git-lfs/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/git-lfs/git-lfs","description":"Git extension for versioning large files","fork":false,"url":"https://api.github.com/repos/git-lfs/git-lfs","forks_url":"https://api.github.com/repos/git-lfs/git-lfs/forks","keys_url":"https://api.github.com/repos/git-lfs/git-lfs/keys{/key_id}","collaborators_url":"https://api.github.com/repos/git-lfs/git-lfs/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/git-lfs/git-lfs/teams","hooks_url":"https://api.github.com/repos/git-lfs/git-lfs/hooks","issue_events_url":"https://api.github.com/repos/git-lfs/git-lfs/issues/events{/number}","events_url":"https://api.github.com/repos/git-lfs/git-lfs/events","assignees_url":"https://api.github.com/repos/git-lfs/git-lfs/assignees{/user}","branches_url":"https://api.github.com/repos/git-lfs/git-lfs/branches{/branch}","tags_url":"https://api.github.com/repos/git-lfs/git-lfs/tags","blobs_url":"https://api.github.com/repos/git-lfs/git-lfs/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/git-lfs/git-lfs/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/git-lfs/git-lfs/git/refs{/sha}","trees_url":"https://api.github.com/repos/git-lfs/git-lfs/git/trees{/sha}","statuses_url":"https://api.github.com/repos/git-lfs/git-lfs/statuses/{sha}","languages_url":"https://api.github.com/repos/git-lfs/git-lfs/languages","stargazers_url":"https://api.github.com/repos/git-lfs/git-lfs/stargazers","contributors_url":"https://api.github.com/repos/git-lfs/git-lfs/contributors","subscribers_url":"https://api.github.com/repos/git-lfs/git-lfs/subscribers","subscription_url":"https://api.github.com/repos/git-lfs/git-lfs/subscription","commits_url":"https://api.github.com/repos/git-lfs/git-lfs/commits{/sha}","git_commits_url":"https://api.github.com/repos/git-lfs/git-lfs/git/commits{/sha}","comments_url":"https://api.github.com/repos/git-lfs/git-lfs/comments{/number}","issue_comment_url":"https://api.github.com/repos/git-lfs/git-lfs/issues/comments{/number}","contents_url":"https://api.github.com/repos/git-lfs/git-lfs/contents/{+path}","compare_url":"https://api.github.com/repos/git-lfs/git-lfs/compare/{base}...{head}","merges_url":"https://api.github.com/repos/git-lfs/git-lfs/merges","archive_url":"https://api.github.com/repos/git-lfs/git-lfs/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/git-lfs/git-lfs/downloads","issues_url":"https://api.github.com/repos/git-lfs/git-lfs/issues{/number}","pulls_url":"https://api.github.com/repos/git-lfs/git-lfs/pulls{/number}","milestones_url":"https://api.github.com/repos/git-lfs/git-lfs/milestones{/number}","notifications_url":"https://api.github.com/repos/git-lfs/git-lfs/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/git-lfs/git-lfs/labels{/name}","releases_url":"https://api.github.com/repos/git-lfs/git-lfs/releases{/id}","deployments_url":"https://api.github.com/repos/git-lfs/git-lfs/deployments","created_at":"2013-09-22T22:03:28Z","updated_at":"2017-10-17T14:07:58Z","pushed_at":"2017-10-15T03:43:13Z","git_url":"git://github.com/git-lfs/git-lfs.git","ssh_url":"git@github.com:git-lfs/git-lfs.git","clone_url":"https://github.com/git-lfs/git-lfs.git","svn_url":"https://github.com/git-lfs/git-lfs","homepage":"https://git-lfs.github.com","size":8792,"stargazers_count":5019,"watchers_count":5019,"language":"Go","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":863,"mirror_url":null,"open_issues_count":266,"forks":863,"open_issues":266,"watchers":5019,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/git-lfs/git-lfs/pulls/2672"},"html":{"href":"https://github.com/git-lfs/git-lfs/pull/2672"},"issue":{"href":"https://api.github.com/repos/git-lfs/git-lfs/issues/2672"},"comments":{"href":"https://api.github.com/repos/git-lfs/git-lfs/issues/2672/comments"},"review_comments":{"href":"https://api.github.com/repos/git-lfs/git-lfs/pulls/2672/comments"},"review_comment":{"href":"https://api.github.com/repos/git-lfs/git-lfs/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/git-lfs/git-lfs/pulls/2672/commits"},"statuses":{"href":"https://api.github.com/repos/git-lfs/git-lfs/statuses/6566be686416c97e7a9f958c92c46ec96e009bfe"}},"author_association":"OWNER","merged":false,"mergeable":null,"rebaseable":null,"mergeable_state":"unknown","merged_by":null,"comments":0,"review_comments":0,"maintainer_can_modify":false,"commits":2,"additions":81,"deletions":53,"changed_files":2}}
{"actor":{"display_login":"technoweenie"}}