{"action":"closed","issue":{"url":"https://api.github.com/repos/aio-libs/aiohttp/issues/1811","repository_url":"https://api.github.com/repos/aio-libs/aiohttp","labels_url":"https://api.github.com/repos/aio-libs/aiohttp/issues/1811/labels{/name}","comments_url":"https://api.github.com/repos/aio-libs/aiohttp/issues/1811/comments","events_url":"https://api.github.com/repos/aio-libs/aiohttp/issues/1811/events","html_url":"https://github.com/aio-libs/aiohttp/issues/1811","id":221214250,"number":1811,"title":"Encoding detection results in performance difference with other clients","user":{"login":"serathius","id":5873433,"avatar_url":"https://avatars1.githubusercontent.com/u/5873433?v=3","gravatar_id":"","url":"https://api.github.com/users/serathius","html_url":"https://github.com/serathius","followers_url":"https://api.github.com/users/serathius/followers","following_url":"https://api.github.com/users/serathius/following{/other_user}","gists_url":"https://api.github.com/users/serathius/gists{/gist_id}","starred_url":"https://api.github.com/users/serathius/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/serathius/subscriptions","organizations_url":"https://api.github.com/users/serathius/orgs","repos_url":"https://api.github.com/users/serathius/repos","events_url":"https://api.github.com/users/serathius/events{/privacy}","received_events_url":"https://api.github.com/users/serathius/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":6,"created_at":"2017-04-12T10:10:26Z","updated_at":"2017-06-26T09:55:57Z","closed_at":"2017-06-26T09:55:57Z","body":"## Long story short\r\n\r\nWhen using aiohttp for fetching pages I found strange performance problems. We started comparing timings with other clients like curl and requests and difference was significant. For other clients fetching page was 6 times faster compared to aiohttp. After some digging we found that problem was using method \"text\" that was used encoding detection from chardet. \r\n\r\nRequests is also using chardet, but the difference is that it's skipping it if content-type contains word \"text\" by using 'ISO-8859-1'. https://github.com/kennethreitz/requests/blob/master/requests/utils.py#L362\r\n\r\n## Expected behaviour\r\n\r\nMatching behavior to other popular clients.\r\n\r\n## Actual behaviour\r\n\r\nHuge performance hit for pages without explicit encoding. For example \"Content-Type: text/html\"\r\nFor 300kB pages time difference for using encoding and not is 8s to 2s respectively. (using method \"text\" instead of \"read\"), and for 1.3MB page difference is 33s to 4.5s.\r\n\r\n## Steps to reproduce\r\n\r\nI'm sorry I cannot disclose the page that I used for testing. \r\n\r\n## Your environment\r\nI tested it on two environments\r\nLinux 4.8 Ubuntu 16.10 Python 3.6\r\naiohttp==1.0.5\r\nchardet==2.3.0\r\n\r\nLinux 4.8 Ubuntu 16.10 Python 3.5.2\r\naiohttp==2.0.6\r\nchardet==3.0.1\r\n^ for that environment problem was smaller by 30% \r\n\r\n"}}
{"actor":{"display_login":"asvetlov"}}