{"action":"created","issue":{"url":"https://api.github.com/repos/jdidion/atropos/issues/38","repository_url":"https://api.github.com/repos/jdidion/atropos","labels_url":"https://api.github.com/repos/jdidion/atropos/issues/38/labels{/name}","comments_url":"https://api.github.com/repos/jdidion/atropos/issues/38/comments","events_url":"https://api.github.com/repos/jdidion/atropos/issues/38/events","html_url":"https://github.com/jdidion/atropos/issues/38","id":256392293,"number":38,"title":"Parallelized atropos trim produces MulticoreError","user":{"login":"cshenanigans","id":1870566,"avatar_url":"https://avatars0.githubusercontent.com/u/1870566?v=4","gravatar_id":"","url":"https://api.github.com/users/cshenanigans","html_url":"https://github.com/cshenanigans","followers_url":"https://api.github.com/users/cshenanigans/followers","following_url":"https://api.github.com/users/cshenanigans/following{/other_user}","gists_url":"https://api.github.com/users/cshenanigans/gists{/gist_id}","starred_url":"https://api.github.com/users/cshenanigans/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cshenanigans/subscriptions","organizations_url":"https://api.github.com/users/cshenanigans/orgs","repos_url":"https://api.github.com/users/cshenanigans/repos","events_url":"https://api.github.com/users/cshenanigans/events{/privacy}","received_events_url":"https://api.github.com/users/cshenanigans/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2017-09-08T23:05:41Z","updated_at":"2017-09-12T20:34:21Z","closed_at":null,"author_association":"NONE","body":"Atropos trim when parallelized with compression on worker is failing on [`finish()` command at this line](https://github.com/jdidion/atropos/blob/master/atropos/commands/trim/__init__.py#L776). \r\n Related to #30.\r\n\r\nIt appears that the code initializes `cur_batch = 1` and increments this after writing each batch, thus when all batches have been written `cur_batch == total_batches + 1`, however this check expects `cur_batch == total_batches`.\r\n\r\nWhat's unclear to me is why this only fails for some samples and not all, since this should always be true. I believe the fix should be simply updating [this line](https://github.com/jdidion/atropos/blob/master/atropos/commands/trim/__init__.py#L776) to `cur_batch == total_batches + 1`.\r\n\r\n### To reproduce\r\n```sh\r\natropos trim --log-level DEBUG -T 8 --preserve-order --no-default-adapters -u 6 -U 6 -o read1.trim.fastq.gz -p read2.trim.fastq.gz -pe1 read1.fastq.gz -pe2 read2.fastq.gz > trim.log\r\n```\r\n\r\nCauses the following error:\r\n```\r\nProcess Result process:\r\nTraceback (most recent call last):\r\n  File \"<PATH>/python3.5/multiprocessing/process.py\", line 249, in _bootstrap\r\n    self.run()\r\n  File \"<PATH>/venv/lib/python3.5/site-packages/atropos/commands/trim/__init__.py\", line 874, in run\r\n    num_batches if num_batches > 0 else None)\r\n  File \"<PATH>/venv/lib/python3.5/site-packages/atropos/commands/trim/__init__.py\", line 780, in finish\r\n    total_batches))\r\natropos.commands.multicore.MulticoreError: OrderPreservingWriterResultHandler finishing without having seen 50482 batches\r\n```"},"comment":{"url":"https://api.github.com/repos/jdidion/atropos/issues/comments/328976306","html_url":"https://github.com/jdidion/atropos/issues/38#issuecomment-328976306","issue_url":"https://api.github.com/repos/jdidion/atropos/issues/38","id":328976306,"user":{"login":"cshenanigans","id":1870566,"avatar_url":"https://avatars0.githubusercontent.com/u/1870566?v=4","gravatar_id":"","url":"https://api.github.com/users/cshenanigans","html_url":"https://github.com/cshenanigans","followers_url":"https://api.github.com/users/cshenanigans/followers","following_url":"https://api.github.com/users/cshenanigans/following{/other_user}","gists_url":"https://api.github.com/users/cshenanigans/gists{/gist_id}","starred_url":"https://api.github.com/users/cshenanigans/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cshenanigans/subscriptions","organizations_url":"https://api.github.com/users/cshenanigans/orgs","repos_url":"https://api.github.com/users/cshenanigans/repos","events_url":"https://api.github.com/users/cshenanigans/events{/privacy}","received_events_url":"https://api.github.com/users/cshenanigans/received_events","type":"User","site_admin":false},"created_at":"2017-09-12T20:34:21Z","updated_at":"2017-09-12T20:34:21Z","author_association":"NONE","body":"@jdidion What is your planned release schedule? I see this commit was added to the larger 1.3 working branch.  Are you planning to release the bug fix in it's own release prior to 1.3, or if not when is 1.3 planned for release?"}}
{"actor":{"display_login":"cshenanigans"}}