{"action":"created","issue":{"url":"https://api.github.com/repos/psycopg/psycopg2/issues/410","repository_url":"https://api.github.com/repos/psycopg/psycopg2","labels_url":"https://api.github.com/repos/psycopg/psycopg2/issues/410/labels{/name}","comments_url":"https://api.github.com/repos/psycopg/psycopg2/issues/410/comments","events_url":"https://api.github.com/repos/psycopg/psycopg2/issues/410/events","html_url":"https://github.com/psycopg/psycopg2/issues/410","id":134093884,"number":410,"title":"Python exceptions raised in a wait_callback do not always propagate","user":{"login":"dhagrow","id":987294,"avatar_url":"https://avatars2.githubusercontent.com/u/987294?v=3","gravatar_id":"","url":"https://api.github.com/users/dhagrow","html_url":"https://github.com/dhagrow","followers_url":"https://api.github.com/users/dhagrow/followers","following_url":"https://api.github.com/users/dhagrow/following{/other_user}","gists_url":"https://api.github.com/users/dhagrow/gists{/gist_id}","starred_url":"https://api.github.com/users/dhagrow/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dhagrow/subscriptions","organizations_url":"https://api.github.com/users/dhagrow/orgs","repos_url":"https://api.github.com/users/dhagrow/repos","events_url":"https://api.github.com/users/dhagrow/events{/privacy}","received_events_url":"https://api.github.com/users/dhagrow/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":{"url":"https://api.github.com/repos/psycopg/psycopg2/milestones/12","html_url":"https://github.com/psycopg/psycopg2/milestone/12","labels_url":"https://api.github.com/repos/psycopg/psycopg2/milestones/12/labels","id":2387694,"number":12,"title":"psycopg 2.7.2","description":"","creator":{"login":"dvarrazzo","id":199429,"avatar_url":"https://avatars3.githubusercontent.com/u/199429?v=3","gravatar_id":"","url":"https://api.github.com/users/dvarrazzo","html_url":"https://github.com/dvarrazzo","followers_url":"https://api.github.com/users/dvarrazzo/followers","following_url":"https://api.github.com/users/dvarrazzo/following{/other_user}","gists_url":"https://api.github.com/users/dvarrazzo/gists{/gist_id}","starred_url":"https://api.github.com/users/dvarrazzo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dvarrazzo/subscriptions","organizations_url":"https://api.github.com/users/dvarrazzo/orgs","repos_url":"https://api.github.com/users/dvarrazzo/repos","events_url":"https://api.github.com/users/dvarrazzo/events{/privacy}","received_events_url":"https://api.github.com/users/dvarrazzo/received_events","type":"User","site_admin":false},"open_issues":0,"closed_issues":3,"state":"open","created_at":"2017-03-15T12:24:50Z","updated_at":"2017-04-05T14:16:57Z","due_on":null,"closed_at":null},"comments":15,"created_at":"2016-02-16T20:52:24Z","updated_at":"2017-04-05T15:28:35Z","closed_at":"2017-04-05T14:16:57Z","body":"I haven't been able to put together a minimal example of this, so unfortunately I have only been able to get this problem to manifest within the scope of a project I am on. I will continue to see if I can put together an example, but in the meantime the best I can offer is some context.\n\nThe project is using SQLAlchemy on top of psycopg2, using psycogreen to provide this gevent wait_callback:\n\n```\ndef gevent_wait_callback(conn, timeout=None):\n    \"\"\"A wait callback useful to allow gevent to work with Psycopg.\"\"\"\n    while 1:\n        state = conn.poll()\n        if state == extensions.POLL_OK:\n            break\n        elif state == extensions.POLL_READ:\n            wait_read(conn.fileno(), timeout=timeout)\n        elif state == extensions.POLL_WRITE:\n            wait_write(conn.fileno(), timeout=timeout)\n        else:\n            raise psycopg2.OperationalError(\n                \"Bad result from poll: %r\" % state)\n```\n\nThe problem I'm seeing is that an exception that is raised within this callback is sometimes not being propagated to the Python code that is making calls to psycopg2 via SQLAlchemy. Instead, I get an `OperationalError('unknown error')`.\n\nTo be specific, I am killing greenlets, which raises a `GreenletExit` exception in the greenlet when it's execution resumes. In some cases, `GreenletExit` is getting raised in the wait_callback. Usually, `GreenletExit` is then also raised in the code that makes an SQLAlchemy call (e.g. `session.query(Model).filter().one()`). The problem is that sometimes I get the `OperationalError` instead.\n\nI've toyed around in the C layer of psycopg2, and I can see that `PyErr_Occurred()` returns `GreenletExit` after the callback is called in `green.c:psyco_wait()`. That is somehow getting lost in some cases before the C layer returns control to Python, and it ends up in `pqpath.c:pq_complete_error()` which raises `OperationalError('unknown error')`.\n\nThis seems to me like a bug along some specific code path, but I haven't been able to trace exactly where it goes astray and the exception gets lost.\n"},"comment":{"url":"https://api.github.com/repos/psycopg/psycopg2/issues/comments/291898595","html_url":"https://github.com/psycopg/psycopg2/issues/410#issuecomment-291898595","issue_url":"https://api.github.com/repos/psycopg/psycopg2/issues/410","id":291898595,"user":{"login":"underyx","id":1060436,"avatar_url":"https://avatars3.githubusercontent.com/u/1060436?v=3","gravatar_id":"","url":"https://api.github.com/users/underyx","html_url":"https://github.com/underyx","followers_url":"https://api.github.com/users/underyx/followers","following_url":"https://api.github.com/users/underyx/following{/other_user}","gists_url":"https://api.github.com/users/underyx/gists{/gist_id}","starred_url":"https://api.github.com/users/underyx/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/underyx/subscriptions","organizations_url":"https://api.github.com/users/underyx/orgs","repos_url":"https://api.github.com/users/underyx/repos","events_url":"https://api.github.com/users/underyx/events{/privacy}","received_events_url":"https://api.github.com/users/underyx/received_events","type":"User","site_admin":false},"created_at":"2017-04-05T15:28:35Z","updated_at":"2017-04-05T15:28:35Z","body":"Yesterday we got 66 of these errors over the span of around an hour. We just ran for 40 minutes with psycopg2 at 4b4d2796b7efdd0846e972fd7a300ca4e7b11ccd and we got\r\n\r\n- 0 `unknown error`s\r\n- 50 of a _different_ exception (which must've been what was masked yesterday)\r\n\r\nSo, I can confirm that this issue is fixed! Thanks again 🙃 "}}
{"actor":{"display_login":"underyx"}}