{"action":"opened","issue":{"url":"https://api.github.com/repos/pytorch/pytorch/issues/1707","repository_url":"https://api.github.com/repos/pytorch/pytorch","labels_url":"https://api.github.com/repos/pytorch/pytorch/issues/1707/labels{/name}","comments_url":"https://api.github.com/repos/pytorch/pytorch/issues/1707/comments","events_url":"https://api.github.com/repos/pytorch/pytorch/issues/1707/events","html_url":"https://github.com/pytorch/pytorch/issues/1707","id":233306871,"number":1707,"title":"Cannot share cuda.Tensor on GPU 1 among processes.","user":{"login":"yuandong-tian","id":2973937,"avatar_url":"https://avatars0.githubusercontent.com/u/2973937?v=3","gravatar_id":"","url":"https://api.github.com/users/yuandong-tian","html_url":"https://github.com/yuandong-tian","followers_url":"https://api.github.com/users/yuandong-tian/followers","following_url":"https://api.github.com/users/yuandong-tian/following{/other_user}","gists_url":"https://api.github.com/users/yuandong-tian/gists{/gist_id}","starred_url":"https://api.github.com/users/yuandong-tian/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/yuandong-tian/subscriptions","organizations_url":"https://api.github.com/users/yuandong-tian/orgs","repos_url":"https://api.github.com/users/yuandong-tian/repos","events_url":"https://api.github.com/users/yuandong-tian/events{/privacy}","received_events_url":"https://api.github.com/users/yuandong-tian/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2017-06-02T21:24:18Z","updated_at":"2017-06-02T21:24:18Z","closed_at":null,"body":"Here is a minimal example. Is there any solution?\r\n\r\nTHCudaCheck FAIL file=/py/conda-bld/pytorch_1493670682084/work/torch/csrc/generic/StorageSharing.cpp line=248 error=11 : invalid argument\r\nTraceback (most recent call last):\r\n  File \"/home/yuandong/anaconda3/lib/python3.5/multiprocessing/queues.py\", line 241, in _feed\r\n    obj = ForkingPickler.dumps(obj)\r\n  File \"/home/yuandong/anaconda3/lib/python3.5/multiprocessing/reduction.py\", line 50, in dumps\r\n    cls(buf, protocol).dump(obj)\r\n  File \"/home/yuandong/anaconda3/lib/python3.5/site-packages/torch/multiprocessing/reductions.py\", line 104, in reduce_storage\r\n    metadata = storage._share_cuda_()\r\nRuntimeError: cuda runtime error (11) : invalid argument at /py/conda-bld/pytorch_1493670682084/work/torch/csrc/generic/StorageSharing.cpp:248\r\n\r\n```\r\nimport torch\r\nimport torch.multiprocessing as _mp\r\nmp = _mp.get_context('spawn')\r\n\r\ndef process_main(idx, q, b):\r\n    m = q.get()\r\n    b.wait()\r\n    for i in range(10):\r\n        q.get()\r\n        print(\"[%d] %f, %f\" % (idx, m[\"a\"][0, 0], m[\"b\"][2, 3]))\r\n        b.wait()\r\n\r\nif __name__ == \"__main__\":\r\n    m = dict(a=torch.FloatTensor(2, 3).cuda(1), b=torch.FloatTensor(3, 4))\r\n    total_process = 3\r\n\r\n    q = mp.Queue()\r\n    b = mp.Barrier(total_process)\r\n\r\n    for i in range(total_process - 1):\r\n        proc = mp.Process(target=process_main, args=(i, q, b))\r\n        proc.start()\r\n\r\n    for i in range(total_process - 1):\r\n        q.put(m)\r\n    b.wait()\r\n\r\n    for i in range(10):\r\n        m[\"a\"][0, 0] = i\r\n        m[\"b\"][2, 3] = 2*i\r\n        for j in range(total_process-1):\r\n            q.put(1)\r\n        b.wait()\r\n```"}}
{"actor":{"display_login":"yuandong-tian"}}