{"action":"created","issue":{"url":"https://api.github.com/repos/Franiac/TwitchLeecher/issues/144","repository_url":"https://api.github.com/repos/Franiac/TwitchLeecher","labels_url":"https://api.github.com/repos/Franiac/TwitchLeecher/issues/144/labels{/name}","comments_url":"https://api.github.com/repos/Franiac/TwitchLeecher/issues/144/comments","events_url":"https://api.github.com/repos/Franiac/TwitchLeecher/issues/144/events","html_url":"https://github.com/Franiac/TwitchLeecher/issues/144","id":229452360,"number":144,"title":"temporary folder on samba share slowdown","user":{"login":"theultramage","id":1738190,"avatar_url":"https://avatars1.githubusercontent.com/u/1738190?v=3","gravatar_id":"","url":"https://api.github.com/users/theultramage","html_url":"https://github.com/theultramage","followers_url":"https://api.github.com/users/theultramage/followers","following_url":"https://api.github.com/users/theultramage/following{/other_user}","gists_url":"https://api.github.com/users/theultramage/gists{/gist_id}","starred_url":"https://api.github.com/users/theultramage/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/theultramage/subscriptions","organizations_url":"https://api.github.com/users/theultramage/orgs","repos_url":"https://api.github.com/users/theultramage/repos","events_url":"https://api.github.com/users/theultramage/events{/privacy}","received_events_url":"https://api.github.com/users/theultramage/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":1,"created_at":"2017-05-17T18:48:51Z","updated_at":"2017-05-17T19:53:01Z","closed_at":null,"body":"While downloading a 7-hour 160p stream to a networked share, I noticed that eventually the download will slow down to a trickle and smbd will max out one cpu core. Cancelling the download will let smbd recover, after all the temporary files get deleted.\r\n\r\nAfter seeing that the temporary folder contained many thousands of small files, I'm assuming that TwitchLeecher does some inefficient activity when downloading each chunk. Something like listing all the files in the directory, or finding the highest chunk id by testing the existence of all previous files one by one. I don't see anything obvious in ProcMon when saving to a local disk, though. I did notice that when doing the processing step from a local disk to a network share, the data is written inefficiently, in many small chunks (100-1000 bytes), creating a processing speed bottleneck of 1MB/s."},"comment":{"url":"https://api.github.com/repos/Franiac/TwitchLeecher/issues/comments/302213304","html_url":"https://github.com/Franiac/TwitchLeecher/issues/144#issuecomment-302213304","issue_url":"https://api.github.com/repos/Franiac/TwitchLeecher/issues/144","id":302213304,"user":{"login":"theultramage","id":1738190,"avatar_url":"https://avatars1.githubusercontent.com/u/1738190?v=3","gravatar_id":"","url":"https://api.github.com/users/theultramage","html_url":"https://github.com/theultramage","followers_url":"https://api.github.com/users/theultramage/followers","following_url":"https://api.github.com/users/theultramage/following{/other_user}","gists_url":"https://api.github.com/users/theultramage/gists{/gist_id}","starred_url":"https://api.github.com/users/theultramage/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/theultramage/subscriptions","organizations_url":"https://api.github.com/users/theultramage/orgs","repos_url":"https://api.github.com/users/theultramage/repos","events_url":"https://api.github.com/users/theultramage/events{/privacy}","received_events_url":"https://api.github.com/users/theultramage/received_events","type":"User","site_admin":false},"created_at":"2017-05-17T19:53:01Z","updated_at":"2017-05-17T19:53:01Z","body":"Here's the first 30% of the download. I/O overload is not an issue, it very quickly drops down to maybe 500kB/s average. The client seems to be requesting some cpu-intensive activity, and things get worse over time. My initial guess is ```FileSystem.DeleteFile(part.LocalFile);```. I can see the Disposition: Open / Error: object name not found messages going back and forth. This would cause a problem if for some reason, file lookups by name in a large directory were expensive. No idea why they would be, though.\r\n![untitled](https://cloud.githubusercontent.com/assets/1738190/26172722/abbb9e1e-3b49-11e7-86d7-d7485da0fe43.png)\r\n"}}
{"actor":{"display_login":"theultramage"}}