{"action":"created","comment":{"url":"https://api.github.com/repos/wikimedia/service-runner/pulls/comments/102752029","pull_request_review_id":23518593,"id":102752029,"diff_hunk":"@@ -121,6 +121,29 @@ var streamConverter = {\n };\n var streamConverterList = Object.keys(streamConverter);\n \n+Logger.prototype._addErrorHandler = function() {\n+    var self = this;\n+    self._logger.on('error', function(err, failedStream) {\n+        // If some fatal error occurred in one of the logger streams,\n+        // we can't do much, so ignore.\n+\n+        if (err.code === 'ENOSPC') {","path":"lib/logger.js","position":20,"original_position":20,"commit_id":"1dbf95dc534c597da5bab953afb7a2a1a13ce607","original_commit_id":"1dbf95dc534c597da5bab953afb7a2a1a13ce607","user":{"login":"gwicke","id":1225898,"avatar_url":"https://avatars.githubusercontent.com/u/1225898?v=3","gravatar_id":"","url":"https://api.github.com/users/gwicke","html_url":"https://github.com/gwicke","followers_url":"https://api.github.com/users/gwicke/followers","following_url":"https://api.github.com/users/gwicke/following{/other_user}","gists_url":"https://api.github.com/users/gwicke/gists{/gist_id}","starred_url":"https://api.github.com/users/gwicke/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/gwicke/subscriptions","organizations_url":"https://api.github.com/users/gwicke/orgs","repos_url":"https://api.github.com/users/gwicke/repos","events_url":"https://api.github.com/users/gwicke/events{/privacy}","received_events_url":"https://api.github.com/users/gwicke/received_events","type":"User","site_admin":false},"body":"Is there a reason to keep going if the error is not ENOSPC? For example, a partition that's remounted read-only should not take out the service.","created_at":"2017-02-23T16:21:29Z","updated_at":"2017-02-23T16:21:29Z","html_url":"https://github.com/wikimedia/service-runner/pull/156#discussion_r102752029","pull_request_url":"https://api.github.com/repos/wikimedia/service-runner/pulls/156","_links":{"self":{"href":"https://api.github.com/repos/wikimedia/service-runner/pulls/comments/102752029"},"html":{"href":"https://github.com/wikimedia/service-runner/pull/156#discussion_r102752029"},"pull_request":{"href":"https://api.github.com/repos/wikimedia/service-runner/pulls/156"}}},"pull_request":{"url":"https://api.github.com/repos/wikimedia/service-runner/pulls/156","id":107496564,"html_url":"https://github.com/wikimedia/service-runner/pull/156","diff_url":"https://github.com/wikimedia/service-runner/pull/156.diff","patch_url":"https://github.com/wikimedia/service-runner/pull/156.patch","issue_url":"https://api.github.com/repos/wikimedia/service-runner/issues/156","number":156,"state":"closed","locked":false,"title":"Don't bring down the service if an error occurs in the logger","user":{"login":"Pchelolo","id":814754,"avatar_url":"https://avatars.githubusercontent.com/u/814754?v=3","gravatar_id":"","url":"https://api.github.com/users/Pchelolo","html_url":"https://github.com/Pchelolo","followers_url":"https://api.github.com/users/Pchelolo/followers","following_url":"https://api.github.com/users/Pchelolo/following{/other_user}","gists_url":"https://api.github.com/users/Pchelolo/gists{/gist_id}","starred_url":"https://api.github.com/users/Pchelolo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Pchelolo/subscriptions","organizations_url":"https://api.github.com/users/Pchelolo/orgs","repos_url":"https://api.github.com/users/Pchelolo/repos","events_url":"https://api.github.com/users/Pchelolo/events{/privacy}","received_events_url":"https://api.github.com/users/Pchelolo/received_events","type":"User","site_admin":false},"body":"I've been testing what happens when the disk partition we log to fills up. Currently that exception is not \r\nbeing handled, and the whole service (including master process) just dies. \r\n\r\nWe don't want to bring down the whole service because of the logger, but we as well can't really handle properly any fatal logging errors, so let's just ignore them hoping that if one stream failed, other streams would succeed.\r\n\r\nHowever, disk fill-up is a different situation. In that case every attempt to log something would result in a memory leak, quickly exhausting the process memory. To fix it up, we can remove the file logging stream and hope that some other streams would we able to deliver our error message.\r\n\r\nBug: https://phabricator.wikimedia.org/T112648","created_at":"2017-02-22T20:57:58Z","updated_at":"2017-02-23T16:21:29Z","closed_at":"2017-02-22T22:21:16Z","merged_at":"2017-02-22T22:21:16Z","merge_commit_sha":"4cfc27e73ab54a12d4a4bd7b300aedbabab910ea","assignee":null,"assignees":[],"milestone":null,"commits_url":"https://api.github.com/repos/wikimedia/service-runner/pulls/156/commits","review_comments_url":"https://api.github.com/repos/wikimedia/service-runner/pulls/156/comments","review_comment_url":"https://api.github.com/repos/wikimedia/service-runner/pulls/comments{/number}","comments_url":"https://api.github.com/repos/wikimedia/service-runner/issues/156/comments","statuses_url":"https://api.github.com/repos/wikimedia/service-runner/statuses/1dbf95dc534c597da5bab953afb7a2a1a13ce607","head":{"label":"Pchelolo:logger_err","ref":"logger_err","sha":"1dbf95dc534c597da5bab953afb7a2a1a13ce607","user":{"login":"Pchelolo","id":814754,"avatar_url":"https://avatars.githubusercontent.com/u/814754?v=3","gravatar_id":"","url":"https://api.github.com/users/Pchelolo","html_url":"https://github.com/Pchelolo","followers_url":"https://api.github.com/users/Pchelolo/followers","following_url":"https://api.github.com/users/Pchelolo/following{/other_user}","gists_url":"https://api.github.com/users/Pchelolo/gists{/gist_id}","starred_url":"https://api.github.com/users/Pchelolo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Pchelolo/subscriptions","organizations_url":"https://api.github.com/users/Pchelolo/orgs","repos_url":"https://api.github.com/users/Pchelolo/repos","events_url":"https://api.github.com/users/Pchelolo/events{/privacy}","received_events_url":"https://api.github.com/users/Pchelolo/received_events","type":"User","site_admin":false},"repo":{"id":40494448,"name":"service-runner","full_name":"Pchelolo/service-runner","owner":{"login":"Pchelolo","id":814754,"avatar_url":"https://avatars.githubusercontent.com/u/814754?v=3","gravatar_id":"","url":"https://api.github.com/users/Pchelolo","html_url":"https://github.com/Pchelolo","followers_url":"https://api.github.com/users/Pchelolo/followers","following_url":"https://api.github.com/users/Pchelolo/following{/other_user}","gists_url":"https://api.github.com/users/Pchelolo/gists{/gist_id}","starred_url":"https://api.github.com/users/Pchelolo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Pchelolo/subscriptions","organizations_url":"https://api.github.com/users/Pchelolo/orgs","repos_url":"https://api.github.com/users/Pchelolo/repos","events_url":"https://api.github.com/users/Pchelolo/events{/privacy}","received_events_url":"https://api.github.com/users/Pchelolo/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/Pchelolo/service-runner","description":"Generic nodejs service supervisor","fork":true,"url":"https://api.github.com/repos/Pchelolo/service-runner","forks_url":"https://api.github.com/repos/Pchelolo/service-runner/forks","keys_url":"https://api.github.com/repos/Pchelolo/service-runner/keys{/key_id}","collaborators_url":"https://api.github.com/repos/Pchelolo/service-runner/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/Pchelolo/service-runner/teams","hooks_url":"https://api.github.com/repos/Pchelolo/service-runner/hooks","issue_events_url":"https://api.github.com/repos/Pchelolo/service-runner/issues/events{/number}","events_url":"https://api.github.com/repos/Pchelolo/service-runner/events","assignees_url":"https://api.github.com/repos/Pchelolo/service-runner/assignees{/user}","branches_url":"https://api.github.com/repos/Pchelolo/service-runner/branches{/branch}","tags_url":"https://api.github.com/repos/Pchelolo/service-runner/tags","blobs_url":"https://api.github.com/repos/Pchelolo/service-runner/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/Pchelolo/service-runner/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/Pchelolo/service-runner/git/refs{/sha}","trees_url":"https://api.github.com/repos/Pchelolo/service-runner/git/trees{/sha}","statuses_url":"https://api.github.com/repos/Pchelolo/service-runner/statuses/{sha}","languages_url":"https://api.github.com/repos/Pchelolo/service-runner/languages","stargazers_url":"https://api.github.com/repos/Pchelolo/service-runner/stargazers","contributors_url":"https://api.github.com/repos/Pchelolo/service-runner/contributors","subscribers_url":"https://api.github.com/repos/Pchelolo/service-runner/subscribers","subscription_url":"https://api.github.com/repos/Pchelolo/service-runner/subscription","commits_url":"https://api.github.com/repos/Pchelolo/service-runner/commits{/sha}","git_commits_url":"https://api.github.com/repos/Pchelolo/service-runner/git/commits{/sha}","comments_url":"https://api.github.com/repos/Pchelolo/service-runner/comments{/number}","issue_comment_url":"https://api.github.com/repos/Pchelolo/service-runner/issues/comments{/number}","contents_url":"https://api.github.com/repos/Pchelolo/service-runner/contents/{+path}","compare_url":"https://api.github.com/repos/Pchelolo/service-runner/compare/{base}...{head}","merges_url":"https://api.github.com/repos/Pchelolo/service-runner/merges","archive_url":"https://api.github.com/repos/Pchelolo/service-runner/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/Pchelolo/service-runner/downloads","issues_url":"https://api.github.com/repos/Pchelolo/service-runner/issues{/number}","pulls_url":"https://api.github.com/repos/Pchelolo/service-runner/pulls{/number}","milestones_url":"https://api.github.com/repos/Pchelolo/service-runner/milestones{/number}","notifications_url":"https://api.github.com/repos/Pchelolo/service-runner/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/Pchelolo/service-runner/labels{/name}","releases_url":"https://api.github.com/repos/Pchelolo/service-runner/releases{/id}","deployments_url":"https://api.github.com/repos/Pchelolo/service-runner/deployments","created_at":"2015-08-10T16:54:28Z","updated_at":"2016-01-14T01:06:01Z","pushed_at":"2017-02-22T20:51:52Z","git_url":"git://github.com/Pchelolo/service-runner.git","ssh_url":"git@github.com:Pchelolo/service-runner.git","clone_url":"https://github.com/Pchelolo/service-runner.git","svn_url":"https://github.com/Pchelolo/service-runner","homepage":null,"size":311,"stargazers_count":0,"watchers_count":0,"language":"JavaScript","has_issues":false,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":0,"forks":0,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"wikimedia:master","ref":"master","sha":"d26c139de39bbeb9055cd0929d95581a812e37d2","user":{"login":"wikimedia","id":56668,"avatar_url":"https://avatars.githubusercontent.com/u/56668?v=3","gravatar_id":"","url":"https://api.github.com/users/wikimedia","html_url":"https://github.com/wikimedia","followers_url":"https://api.github.com/users/wikimedia/followers","following_url":"https://api.github.com/users/wikimedia/following{/other_user}","gists_url":"https://api.github.com/users/wikimedia/gists{/gist_id}","starred_url":"https://api.github.com/users/wikimedia/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/wikimedia/subscriptions","organizations_url":"https://api.github.com/users/wikimedia/orgs","repos_url":"https://api.github.com/users/wikimedia/repos","events_url":"https://api.github.com/users/wikimedia/events{/privacy}","received_events_url":"https://api.github.com/users/wikimedia/received_events","type":"Organization","site_admin":false},"repo":{"id":30805535,"name":"service-runner","full_name":"wikimedia/service-runner","owner":{"login":"wikimedia","id":56668,"avatar_url":"https://avatars.githubusercontent.com/u/56668?v=3","gravatar_id":"","url":"https://api.github.com/users/wikimedia","html_url":"https://github.com/wikimedia","followers_url":"https://api.github.com/users/wikimedia/followers","following_url":"https://api.github.com/users/wikimedia/following{/other_user}","gists_url":"https://api.github.com/users/wikimedia/gists{/gist_id}","starred_url":"https://api.github.com/users/wikimedia/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/wikimedia/subscriptions","organizations_url":"https://api.github.com/users/wikimedia/orgs","repos_url":"https://api.github.com/users/wikimedia/repos","events_url":"https://api.github.com/users/wikimedia/events{/privacy}","received_events_url":"https://api.github.com/users/wikimedia/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/wikimedia/service-runner","description":"Generic nodejs service supervisor","fork":false,"url":"https://api.github.com/repos/wikimedia/service-runner","forks_url":"https://api.github.com/repos/wikimedia/service-runner/forks","keys_url":"https://api.github.com/repos/wikimedia/service-runner/keys{/key_id}","collaborators_url":"https://api.github.com/repos/wikimedia/service-runner/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/wikimedia/service-runner/teams","hooks_url":"https://api.github.com/repos/wikimedia/service-runner/hooks","issue_events_url":"https://api.github.com/repos/wikimedia/service-runner/issues/events{/number}","events_url":"https://api.github.com/repos/wikimedia/service-runner/events","assignees_url":"https://api.github.com/repos/wikimedia/service-runner/assignees{/user}","branches_url":"https://api.github.com/repos/wikimedia/service-runner/branches{/branch}","tags_url":"https://api.github.com/repos/wikimedia/service-runner/tags","blobs_url":"https://api.github.com/repos/wikimedia/service-runner/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/wikimedia/service-runner/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/wikimedia/service-runner/git/refs{/sha}","trees_url":"https://api.github.com/repos/wikimedia/service-runner/git/trees{/sha}","statuses_url":"https://api.github.com/repos/wikimedia/service-runner/statuses/{sha}","languages_url":"https://api.github.com/repos/wikimedia/service-runner/languages","stargazers_url":"https://api.github.com/repos/wikimedia/service-runner/stargazers","contributors_url":"https://api.github.com/repos/wikimedia/service-runner/contributors","subscribers_url":"https://api.github.com/repos/wikimedia/service-runner/subscribers","subscription_url":"https://api.github.com/repos/wikimedia/service-runner/subscription","commits_url":"https://api.github.com/repos/wikimedia/service-runner/commits{/sha}","git_commits_url":"https://api.github.com/repos/wikimedia/service-runner/git/commits{/sha}","comments_url":"https://api.github.com/repos/wikimedia/service-runner/comments{/number}","issue_comment_url":"https://api.github.com/repos/wikimedia/service-runner/issues/comments{/number}","contents_url":"https://api.github.com/repos/wikimedia/service-runner/contents/{+path}","compare_url":"https://api.github.com/repos/wikimedia/service-runner/compare/{base}...{head}","merges_url":"https://api.github.com/repos/wikimedia/service-runner/merges","archive_url":"https://api.github.com/repos/wikimedia/service-runner/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/wikimedia/service-runner/downloads","issues_url":"https://api.github.com/repos/wikimedia/service-runner/issues{/number}","pulls_url":"https://api.github.com/repos/wikimedia/service-runner/pulls{/number}","milestones_url":"https://api.github.com/repos/wikimedia/service-runner/milestones{/number}","notifications_url":"https://api.github.com/repos/wikimedia/service-runner/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/wikimedia/service-runner/labels{/name}","releases_url":"https://api.github.com/repos/wikimedia/service-runner/releases{/id}","deployments_url":"https://api.github.com/repos/wikimedia/service-runner/deployments","created_at":"2015-02-14T18:58:54Z","updated_at":"2017-02-23T13:41:11Z","pushed_at":"2017-02-22T22:24:07Z","git_url":"git://github.com/wikimedia/service-runner.git","ssh_url":"git@github.com:wikimedia/service-runner.git","clone_url":"https://github.com/wikimedia/service-runner.git","svn_url":"https://github.com/wikimedia/service-runner","homepage":null,"size":301,"stargazers_count":11,"watchers_count":11,"language":"JavaScript","has_issues":false,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":9,"mirror_url":null,"open_issues_count":5,"forks":9,"open_issues":5,"watchers":11,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/wikimedia/service-runner/pulls/156"},"html":{"href":"https://github.com/wikimedia/service-runner/pull/156"},"issue":{"href":"https://api.github.com/repos/wikimedia/service-runner/issues/156"},"comments":{"href":"https://api.github.com/repos/wikimedia/service-runner/issues/156/comments"},"review_comments":{"href":"https://api.github.com/repos/wikimedia/service-runner/pulls/156/comments"},"review_comment":{"href":"https://api.github.com/repos/wikimedia/service-runner/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/wikimedia/service-runner/pulls/156/commits"},"statuses":{"href":"https://api.github.com/repos/wikimedia/service-runner/statuses/1dbf95dc534c597da5bab953afb7a2a1a13ce607"}}}}
{"actor":{"display_login":"gwicke"}}