{"action":"created","issue":{"url":"https://api.github.com/repos/bda-research/node-crawler/issues/222","repository_url":"https://api.github.com/repos/bda-research/node-crawler","labels_url":"https://api.github.com/repos/bda-research/node-crawler/issues/222/labels{/name}","comments_url":"https://api.github.com/repos/bda-research/node-crawler/issues/222/comments","events_url":"https://api.github.com/repos/bda-research/node-crawler/issues/222/events","html_url":"https://github.com/bda-research/node-crawler/issues/222","id":241478511,"number":222,"title":"CRAWLER Error Error: ESOCKETTIMEDOUT when fetching undefined","user":{"login":"acantepie","id":12775815,"avatar_url":"https://avatars0.githubusercontent.com/u/12775815?v=4","gravatar_id":"","url":"https://api.github.com/users/acantepie","html_url":"https://github.com/acantepie","followers_url":"https://api.github.com/users/acantepie/followers","following_url":"https://api.github.com/users/acantepie/following{/other_user}","gists_url":"https://api.github.com/users/acantepie/gists{/gist_id}","starred_url":"https://api.github.com/users/acantepie/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/acantepie/subscriptions","organizations_url":"https://api.github.com/users/acantepie/orgs","repos_url":"https://api.github.com/users/acantepie/repos","events_url":"https://api.github.com/users/acantepie/events{/privacy}","received_events_url":"https://api.github.com/users/acantepie/received_events","type":"User","site_admin":false},"labels":[{"id":126786372,"url":"https://api.github.com/repos/bda-research/node-crawler/labels/Question","name":"Question","color":"d4c5f9","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":6,"created_at":"2017-07-08T20:36:50Z","updated_at":"2017-08-07T11:36:00Z","closed_at":null,"body":"Hello,\r\n\r\nI have developed a crawler to check 404/500 on my website but after he has crawled a lot of url i have the following error : \r\n```\r\n2017-07-08T20:29:10.047Z - error: CRAWLER Error Error: ESOCKETTIMEDOUT when fetching undefined\r\n/home/acantepie/Workspace/WebCrawler/src/WCrawler.js:176\r\n            crawledUrl.href = res.request.uri.href;\r\n```\r\nIt happens randomly (not always on the same url) and after i have crawled a lot of url (~ 1000 urls).\r\n\r\nI use the following config : \r\n```\r\n            rateLimit: 0,\r\n            maxConnections: 50,\r\n            retries: 0,\r\n            jQuery: false,\r\n            skipDuplicates: true,\r\n           ....\r\n```\r\n\r\nThanks for help,"},"comment":{"url":"https://api.github.com/repos/bda-research/node-crawler/issues/comments/320640501","html_url":"https://github.com/bda-research/node-crawler/issues/222#issuecomment-320640501","issue_url":"https://api.github.com/repos/bda-research/node-crawler/issues/222","id":320640501,"user":{"login":"darrenqc","id":5037802,"avatar_url":"https://avatars2.githubusercontent.com/u/5037802?v=4","gravatar_id":"","url":"https://api.github.com/users/darrenqc","html_url":"https://github.com/darrenqc","followers_url":"https://api.github.com/users/darrenqc/followers","following_url":"https://api.github.com/users/darrenqc/following{/other_user}","gists_url":"https://api.github.com/users/darrenqc/gists{/gist_id}","starred_url":"https://api.github.com/users/darrenqc/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/darrenqc/subscriptions","organizations_url":"https://api.github.com/users/darrenqc/orgs","repos_url":"https://api.github.com/users/darrenqc/repos","events_url":"https://api.github.com/users/darrenqc/events{/privacy}","received_events_url":"https://api.github.com/users/darrenqc/received_events","type":"User","site_admin":false},"created_at":"2017-08-07T11:36:00Z","updated_at":"2017-08-07T11:36:00Z","body":"@metux Can I have a look at your test code if you don't mind. I want to reproduce the bug. Thank you."}}
{"actor":{"display_login":"darrenqc"}}