{"action":"created","issue":{"url":"https://api.github.com/repos/urwid/urwid/issues/262","repository_url":"https://api.github.com/repos/urwid/urwid","labels_url":"https://api.github.com/repos/urwid/urwid/issues/262/labels{/name}","comments_url":"https://api.github.com/repos/urwid/urwid/issues/262/comments","events_url":"https://api.github.com/repos/urwid/urwid/issues/262/events","html_url":"https://github.com/urwid/urwid/issues/262","id":285105767,"number":262,"title":"[perf] urwid startup time is slow (over 200ms)","user":{"login":"okayzed","id":98617,"avatar_url":"https://avatars3.githubusercontent.com/u/98617?v=4","gravatar_id":"","url":"https://api.github.com/users/okayzed","html_url":"https://github.com/okayzed","followers_url":"https://api.github.com/users/okayzed/followers","following_url":"https://api.github.com/users/okayzed/following{/other_user}","gists_url":"https://api.github.com/users/okayzed/gists{/gist_id}","starred_url":"https://api.github.com/users/okayzed/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/okayzed/subscriptions","organizations_url":"https://api.github.com/users/okayzed/orgs","repos_url":"https://api.github.com/users/okayzed/repos","events_url":"https://api.github.com/users/okayzed/events{/privacy}","received_events_url":"https://api.github.com/users/okayzed/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2017-12-29T14:55:00Z","updated_at":"2017-12-31T17:03:34Z","closed_at":"2017-12-31T17:03:34Z","author_association":"NONE","body":"For several years, one of my programs has been slow to startup. Previously, I had been content to blame python, but I've recently decided to re-investigate why one of my programs has been to slow to startup and urwid seems to be the module that is causing trouble, specifically a twisted import. The root cause is \"FileDescriptor\" from twisted package imported in main_loop.py\r\n\r\n## Initial Steps\r\n\r\nAfter using the following tools on my scripts, I found the urwid import to blame. \r\n\r\n* perf (showed nothing useful quickly)\r\n* python cProfile (showed nothing)\r\n* manual import modifications - bisected to urwid\r\n\r\n\r\n## Tracking it down in urwid\r\n\r\nI used \"import-profiler\" from pip and the following scripts:\r\n\r\n```\r\nfrom import_profiler import profile_import\r\n\r\nwith profile_import() as context:\r\n    import urwid\r\n    context.print_info()\r\n```\r\n\r\nThe output shows that twisted is clearly doing something:\r\n\r\n```\r\n# urwid has FileDescriptor import from twisted\r\nokay@chalk:~/tonka/apps/urwid$ python t.py\r\n  cumtime (ms)    intime (ms)  name\r\n         257.5            0.3  urwid\r\n          22.4            1.1  +urwid.widget\r\n           7.5            0.4  ++urwid.util\r\n           5.4            4.1  +++urwid\r\n           1.2            0.8  ++++urwid\r\n           1.6            1.4  +++locale\r\n           1.7            0.2  ++urwid\r\n           1.4            1.4  +++weakref\r\n          11.2            0.1  ++urwid.split_repr\r\n          11.1            3.3  +++inspect\r\n           1              1    ++++string\r\n           5.7            5.6  ++++tokenize\r\n           1.1            0.9  +urwid.container\r\n           1.5            0.5  +urwid.graphics\r\n         228.1            5.5  +urwid.main_loop\r\n         222.3            1.9  ++twisted.internet.abstract\r\n         208.5            1    +++zope\r\n         204            141.8  ++++pkg_resources\r\n           2.3            1.7  +++++zipfile\r\n           3.1            3    +++++platform\r\n          13.4            0.2  +++++email.parser\r\n          13.2            0.5  ++++++email.feedparser\r\n          12.5            0.4  +++++++email\r\n          11.9            0.1  ++++++++email.charset\r\n          11.4            0.1  +++++++++email.base64mime\r\n          11.3            0.8  ++++++++++email.utils\r\n           2.8            1.1  +++++++++++random\r\n           1.6            0.1  ++++++++++++hashlib\r\n           1.5            1.5  +++++++++++++_hashlib\r\n           1.9            0.6  +++++++++++socket\r\n           1.1            1.1  ++++++++++++_ssl\r\n           4.8            0.8  +++++++++++urllib\r\n           1.5            1.5  ++++++++++++urlparse\r\n           2.5            1.5  ++++++++++++ssl\r\n           1.4            0.1  +++++pkg_resources.extern\r\n           1.2            1.1  ++++++pkg_resources._vendor.six\r\n           2.9            0    +++++pkg_resources.extern.packaging.version\r\n           2.9            2.7  ++++++pkg_resources._vendor.packaging.version\r\n           6.1            0    +++++pkg_resources.extern.packaging.specifiers\r\n           6.1            6    ++++++pkg_resources._vendor.packaging.specifiers\r\n          30.3            0    +++++pkg_resources.extern.packaging.requirements\r\n          30.3            6.9  ++++++pkg_resources._vendor.packaging.requirements\r\n          21.7            0    +++++++pkg_resources.extern.pyparsing\r\n          21.7           18.4  ++++++++pkg_resources._vendor.pyparsing\r\n           1.4            1.4  +++++++markers\r\n           1.3            0.4  ++++zope.interface.interface\r\n           2              2    ++++zope.interface.interfaces\r\n           6.3            0.3  +++twisted.python\r\n           4.7            3.7  ++++cookielib\r\n           1.8            0.8  +++twisted.python\r\n           3.7            3    +++twisted.internet\r\n```\r\n\r\n\r\n## Removing FileDescriptor from main_loop.py\r\n\r\n\r\nAfter removing the FileDescriptor import, the import time for urwid goes down to normal (30ms) from the 250 - 300ms it was at before.\r\n\r\n```\r\n# urwid no longer has FileDescriptor import\r\nokay@chalk:~/tonka/apps/urwid$ python t.py\r\n  cumtime (ms)    intime (ms)  name\r\n          29.7            0.3  urwid\r\n          20              1    +urwid.widget\r\n           6.8            0.3  ++urwid.util\r\n           4.9            4    +++urwid\r\n           1.4            1.3  +++locale\r\n           1.3            0.1  ++urwid\r\n           1              1    +++weakref\r\n          10.2            0.1  ++urwid.split_repr\r\n          10.1            3.3  +++inspect\r\n           5.1            5    ++++tokenize\r\n           1.1            0.9  +urwid.container\r\n           1.3            0.4  +urwid.graphics\r\n           1.3            0.5  +urwid.vterm\r\n           2.4            0.3  +urwid\r\n           2              0.3  ++subprocess\r\n           1.7            1.5  +++pickle\r\n\r\n```\r\n\r\n## Proposed Fix\r\n\r\nNone, yet. \r\n\r\nI don't know why twisted is used, but I presume it is for good reason. However, it is messing with program startup time and adding significant delay. Can you figure out how to eliminate the slow startup time caused by twisted for urwid?"},"comment":{"url":"https://api.github.com/repos/urwid/urwid/issues/comments/354614539","html_url":"https://github.com/urwid/urwid/issues/262#issuecomment-354614539","issue_url":"https://api.github.com/repos/urwid/urwid/issues/262","id":354614539,"user":{"login":"okayzed","id":98617,"avatar_url":"https://avatars3.githubusercontent.com/u/98617?v=4","gravatar_id":"","url":"https://api.github.com/users/okayzed","html_url":"https://github.com/okayzed","followers_url":"https://api.github.com/users/okayzed/followers","following_url":"https://api.github.com/users/okayzed/following{/other_user}","gists_url":"https://api.github.com/users/okayzed/gists{/gist_id}","starred_url":"https://api.github.com/users/okayzed/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/okayzed/subscriptions","organizations_url":"https://api.github.com/users/okayzed/orgs","repos_url":"https://api.github.com/users/okayzed/repos","events_url":"https://api.github.com/users/okayzed/events{/privacy}","received_events_url":"https://api.github.com/users/okayzed/received_events","type":"User","site_admin":false},"created_at":"2017-12-31T17:03:34Z","updated_at":"2017-12-31T17:03:34Z","author_association":"NONE","body":"It seems like this issue is caused by \"twisted\" import in urwid. I was able to remove that for myself, but I still have dependencies on pkg_resources through pygments in my program.\r\n\r\nI think that other people might run into startup slowness with urwid because of pkg_resources if they have twisted installed, so hopefully this issue will help them. (Either uninstall twisted or stub it out before importing urwid)"}}
{"actor":{"display_login":"okayzed"}}