{"action":"created","issue":{"url":"https://api.github.com/repos/chapmanb/bcbio-nextgen/issues/1968","repository_url":"https://api.github.com/repos/chapmanb/bcbio-nextgen","labels_url":"https://api.github.com/repos/chapmanb/bcbio-nextgen/issues/1968/labels{/name}","comments_url":"https://api.github.com/repos/chapmanb/bcbio-nextgen/issues/1968/comments","events_url":"https://api.github.com/repos/chapmanb/bcbio-nextgen/issues/1968/events","html_url":"https://github.com/chapmanb/bcbio-nextgen/issues/1968","id":233393031,"number":1968,"title":"testing my installation","user":{"login":"MurliNair","id":12362166,"avatar_url":"https://avatars0.githubusercontent.com/u/12362166?v=3","gravatar_id":"","url":"https://api.github.com/users/MurliNair","html_url":"https://github.com/MurliNair","followers_url":"https://api.github.com/users/MurliNair/followers","following_url":"https://api.github.com/users/MurliNair/following{/other_user}","gists_url":"https://api.github.com/users/MurliNair/gists{/gist_id}","starred_url":"https://api.github.com/users/MurliNair/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/MurliNair/subscriptions","organizations_url":"https://api.github.com/users/MurliNair/orgs","repos_url":"https://api.github.com/users/MurliNair/repos","events_url":"https://api.github.com/users/MurliNair/events{/privacy}","received_events_url":"https://api.github.com/users/MurliNair/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":4,"created_at":"2017-06-03T20:20:26Z","updated_at":"2017-06-06T00:31:15Z","closed_at":"2017-06-05T09:52:29Z","body":"Hi, \r\nI am getting the following error when I test the Cancer tumor normal example.  I think there are two main errors \r\n1> \"Aligning lane 1_2014-08-13_dream-syn3 with bwa aligner cat: write error: Broken pipe\"\r\n 2>\" [E::bwa_idx_load_from_disk] fail to locate the index files ' returned non-zero exit status 1 \"\r\nIt looks like I have not provided the bwa index file. I am not sure where to put the index file, if I create one for Hg19. Could you comment on that please, and anything else I may have missed?\r\nThanks ../Murli\r\n\r\n===============================================\r\n\r\n$> bcbio_nextgen.py ../config/cancer-dream-syn3.yaml -n 8\r\n[2017-06-03T19:53Z] System YAML configuration: /mnt/workspaceXFS/local/share/bcbio/galaxy/bcbio_system.yaml\r\n[2017-06-03T19:53Z] Resource requests: bwa, sambamba, samtools; memory: 3.00, 3.00, 3.00; cores: 16, 16, 16\r\n[2017-06-03T19:53Z] Configuring 1 jobs to run, using 8 cores each with 24.1g of memory reserved for each job\r\n[2017-06-03T19:53Z] Timing: organize samples\r\n[2017-06-03T19:53Z] multiprocessing: organize_samples\r\n[2017-06-03T19:53Z] Using input YAML configuration: /mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbioExample/cancer-dream-syn3/config/cancer-dream-syn3.yaml\r\n[2017-06-03T19:53Z] Checking sample YAML configuration: /mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbioExample/cancer-dream-syn3/config/cancer-dream-syn3.yaml\r\n[2017-06-03T19:53Z] Testing minimum versions of installed programs\r\n[2017-06-03T19:53Z] Timing: alignment preparation\r\n[2017-06-03T19:53Z] multiprocessing: prep_align_inputs\r\n[2017-06-03T19:53Z] Resource requests: ; memory: 1.00; cores: 1\r\n[2017-06-03T19:53Z] Configuring 2 jobs to run, using 1 cores each with 1.00g of memory reserved for each job\r\n[2017-06-03T19:53Z] Resource requests: ; memory: 1.00; cores: 1\r\n[2017-06-03T19:53Z] Configuring 2 jobs to run, using 1 cores each with 1.00g of memory reserved for each job\r\n[2017-06-03T19:53Z] Resource requests: ; memory: 1.00; cores: 1\r\n[2017-06-03T19:53Z] Configuring 2 jobs to run, using 1 cores each with 1.00g of memory reserved for each job\r\n[2017-06-03T19:53Z] Resource requests: ; memory: 1.00; cores: 1\r\n[2017-06-03T19:53Z] Configuring 2 jobs to run, using 1 cores each with 1.00g of memory reserved for each job\r\n[2017-06-03T19:53Z] multiprocessing: disambiguate_split\r\n[2017-06-03T19:53Z] Timing: alignment\r\n[2017-06-03T19:53Z] multiprocessing: process_alignment\r\n[2017-06-03T19:53Z] Aligning lane 1_2014-08-13_dream-syn3 with bwa aligner\r\ncat: write error: Broken pipe\r\n[2017-06-03T19:53Z] bwa mem alignment from fastq: syn3-normal\r\n[2017-06-03T19:53Z] [E::bwa_idx_load_from_disk] fail to locate the index files\r\n[2017-06-03T19:53Z] Uncaught exception occurred\r\nTraceback (most recent call last):\r\n  File \"/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/provenance/do.py\", line 22, in run\r\n    _do_run(cmd, checks, log_stdout, env=env)\r\n  File \"/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/provenance/do.py\", line 102, in _do_run\r\n    raise subprocess.CalledProcessError(exitcode, error_msg)\r\nCalledProcessError: Command 'set -o pipefail; unset JAVA_HOME && /mnt/workspaceXFS/local/share/bcbio/galaxy/../anaconda/bin/bwa mem   -c 250 -M -t 8  -R '@RG\\tID:syn3-normal\\tPL:illumina\\tPU:1_2014-08-13_dream-syn3\\tSM:syn3-normal' -v 1 /mnt/workspaceXFS/local/share/bcbio/genomes/Hsapiens/GRCh37/seq/GRCh37 <(grabix grab /mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbioExample/cancer-dream-syn3/work/align_prep/syn3-normal_synthetic_challenge_set3_normal_NGv3_1.fq.gz 1 20000000) <(grabix grab /mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbioExample/cancer-dream-syn3/work/align_prep/syn3-normal_synthetic_challenge_set3_normal_NGv3_2.fq.gz 1 20000000)  | /mnt/workspaceXFS/local/share/bcbio/galaxy/../anaconda/bin/samtools sort -n -@ 8 -m 1G -O bam -T /mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbioExample/cancer-dream-syn3/work/bcbiotx/tmp9d3OlW/syn3-normal-sort-1_20000000-sorttmp-namesort -o /mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbioExample/cancer-dream-syn3/work/bcbiotx/tmp9d3OlW/syn3-normal-sort-1_20000000.bam -\r\n[E::bwa_idx_load_from_disk] fail to locate the index files\r\n' returned non-zero exit status 1\r\nTraceback (most recent call last):\r\n  File \"/home/mnair/local/bin/bcbio_nextgen.py\", line 234, in <module>\r\n    main(**kwargs)\r\n  File \"/home/mnair/local/bin/bcbio_nextgen.py\", line 43, in main\r\n    run_main(**kwargs)\r\n  File \"/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/pipeline/main.py\", line 42, in run_main\r\n    fc_dir, run_info_yaml)\r\n  File \"/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/pipeline/main.py\", line 86, in _run_toplevel\r\n    for xs in pipeline(config, run_info_yaml, parallel, dirs, samples):\r\n  File \"/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/pipeline/main.py\", line 126, in variant2pipeline\r\n    samples = run_parallel(\"process_alignment\", samples)\r\n  File \"/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/distributed/multi.py\", line 28, in run_parallel\r\n    return run_multicore(fn, items, config, parallel=parallel)\r\n  File \"/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/distributed/multi.py\", line 86, in run_multicore\r\n    for data in joblib.Parallel(parallel[\"num_jobs\"], batch_size=1)(joblib.delayed(fn)(x) for x in items):\r\n  File \"/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/joblib/parallel.py\", line 804, in __call__\r\n    while self.dispatch_one_batch(iterator):\r\n  File \"/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/joblib/parallel.py\", line 662, in dispatch_one_batch\r\n    self._dispatch(tasks)\r\n  File \"/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/joblib/parallel.py\", line 570, in _dispatch\r\n    job = ImmediateComputeBatch(batch)\r\n  File \"/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/joblib/parallel.py\", line 183, in __init__\r\n    self.results = batch()\r\n  File \"/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/joblib/parallel.py\", line 72, in __call__\r\n    return [func(*args, **kwargs) for func, args, kwargs in self.items]\r\n  File \"/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/utils.py\", line 50, in wrapper\r\n    return apply(f, *args, **kwargs)\r\n  File \"/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/distributed/multitasks.py\", line 93, in process_alignment\r\n    return sample.process_alignment(*args)\r\n  File \"/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/pipeline/sample.py\", line 116, in process_alignment\r\n    data = align_to_sort_bam(fastq1, fastq2, aligner, data)\r\n  File \"/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/pipeline/alignment.py\", line 65, in align_to_sort_bam\r\n    names, align_dir, data)\r\n  File \"/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/pipeline/alignment.py\", line 119, in _align_from_fastq\r\n    out = align_fn(fastq1, fastq2, align_ref, names, align_dir, data)\r\n  File \"/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/ngsalign/bwa.py\", line 156, in align_pipe\r\n    names, rg_info, data)\r\n  File \"/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/ngsalign/bwa.py\", line 167, in _align_mem\r\n    [do.file_nonempty(tx_out_file), do.file_reasonable_size(tx_out_file, fastq_file)])\r\n  File \"/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/provenance/do.py\", line 22, in run\r\n    _do_run(cmd, checks, log_stdout, env=env)\r\n  File \"/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/provenance/do.py\", line 102, in _do_run\r\n    raise subprocess.CalledProcessError(exitcode, error_msg)\r\nsubprocess.CalledProcessError: Command 'set -o pipefail; unset JAVA_HOME && /mnt/workspaceXFS/local/share/bcbio/galaxy/../anaconda/bin/bwa mem   -c 250 -M -t 8  -R '@RG\\tID:syn3-normal\\tPL:illumina\\tPU:1_2014-08-13_dream-syn3\\tSM:syn3-normal' -v 1 /mnt/workspaceXFS/local/share/bcbio/genomes/Hsapiens/GRCh37/seq/GRCh37 <(grabix grab /mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbioExample/cancer-dream-syn3/work/align_prep/syn3-normal_synthetic_challenge_set3_normal_NGv3_1.fq.gz 1 20000000) <(grabix grab /mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbioExample/cancer-dream-syn3/work/align_prep/syn3-normal_synthetic_challenge_set3_normal_NGv3_2.fq.gz 1 20000000)  | /mnt/workspaceXFS/local/share/bcbio/galaxy/../anaconda/bin/samtools sort -n -@ 8 -m 1G -O bam -T /mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbioExample/cancer-dream-syn3/work/bcbiotx/tmp9d3OlW/syn3-normal-sort-1_20000000-sorttmp-namesort -o /mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbioExample/cancer-dream-syn3/work/bcbiotx/tmp9d3OlW/syn3-normal-sort-1_20000000.bam -\r\n[E::bwa_idx_load_from_disk] fail to locate the index files\r\n' returned non-zero exit status 1\r\n"},"comment":{"url":"https://api.github.com/repos/chapmanb/bcbio-nextgen/issues/comments/306347808","html_url":"https://github.com/chapmanb/bcbio-nextgen/issues/1968#issuecomment-306347808","issue_url":"https://api.github.com/repos/chapmanb/bcbio-nextgen/issues/1968","id":306347808,"user":{"login":"MurliNair","id":12362166,"avatar_url":"https://avatars0.githubusercontent.com/u/12362166?v=3","gravatar_id":"","url":"https://api.github.com/users/MurliNair","html_url":"https://github.com/MurliNair","followers_url":"https://api.github.com/users/MurliNair/followers","following_url":"https://api.github.com/users/MurliNair/following{/other_user}","gists_url":"https://api.github.com/users/MurliNair/gists{/gist_id}","starred_url":"https://api.github.com/users/MurliNair/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/MurliNair/subscriptions","organizations_url":"https://api.github.com/users/MurliNair/orgs","repos_url":"https://api.github.com/users/MurliNair/repos","events_url":"https://api.github.com/users/MurliNair/events{/privacy}","received_events_url":"https://api.github.com/users/MurliNair/received_events","type":"User","site_admin":false},"created_at":"2017-06-06T00:31:15Z","updated_at":"2017-06-06T00:31:15Z","body":"Hi, \r\nI reinstalled everything all over using the command given in the \"Automated\" section. It installed cleanly. \r\nI then installed gatk using the following command\r\ngatk-register GenomeAnalysisTK-3.7-0.tar.bz2\r\nI then ran \r\nbcbio_nextgen.py upgrade --data --aligners bwa\r\n\r\nThe cancer -normal did not run cleanly, gave the same error. Also I want to point out, that \r\n(/mnt/workspaceXFS/local/share/bcbio/galaxy/tool-data ) does not contain the index files now. It has the following files\r\ngatk_sorted_picard_index.loc  picard_index.loc  sam_fa_indices.loc\r\n\r\nDid I miss anything? Kindly let me know. Thx a ton for your time. \r\nCheers../Murli\r\n\r\n\r\n"}}
{"actor":{"display_login":"MurliNair"}}