{"action":"created","issue":{"url":"https://api.github.com/repos/marbl/canu/issues/538","repository_url":"https://api.github.com/repos/marbl/canu","labels_url":"https://api.github.com/repos/marbl/canu/issues/538/labels{/name}","comments_url":"https://api.github.com/repos/marbl/canu/issues/538/comments","events_url":"https://api.github.com/repos/marbl/canu/issues/538/events","html_url":"https://github.com/marbl/canu/issues/538","id":239275714,"number":538,"title":"Canu failed","user":{"login":"moudixtc","id":13108127,"avatar_url":"https://avatars1.githubusercontent.com/u/13108127?v=3","gravatar_id":"","url":"https://api.github.com/users/moudixtc","html_url":"https://github.com/moudixtc","followers_url":"https://api.github.com/users/moudixtc/followers","following_url":"https://api.github.com/users/moudixtc/following{/other_user}","gists_url":"https://api.github.com/users/moudixtc/gists{/gist_id}","starred_url":"https://api.github.com/users/moudixtc/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/moudixtc/subscriptions","organizations_url":"https://api.github.com/users/moudixtc/orgs","repos_url":"https://api.github.com/users/moudixtc/repos","events_url":"https://api.github.com/users/moudixtc/events{/privacy}","received_events_url":"https://api.github.com/users/moudixtc/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2017-06-28T19:51:01Z","updated_at":"2017-06-28T19:55:19Z","closed_at":null,"body":"Hi, I just started using canu, and I'm sorry if this is something obvious, but please help me understand what went wrong.\r\n\r\nI'm running canu v1.5 on a grid setup using SGE, which is bootstrapped by cfncluster on AWS. \r\n\r\nGot the following error when running the command `canu -p F_vert2 -d F_vert2_auto genomeSize=50m -pacbio-raw /shared/filtered_subreads.fasta gridEngineMemoryOption=\"-l h_vmem=MEMORY\" gridEngineThreadsOption=\"-pe make THREADS\"`\r\n```\r\n-- Canu release v1.5\r\n-- Detected Java(TM) Runtime Environment '1.8.0_131' (from 'java').\r\n-- Detected gnuplot version '4.6 patchlevel 4' (from 'gnuplot') and image format 'png'.\r\n-- Detected 8 CPUs and 31 gigabytes of memory.\r\n-- Detected Sun Grid Engine in '/opt/sge/default'.\r\n-- User supplied Grid Engine environment '-pe make THREADS'.\r\n-- User supplied Grid Engine consumable '-l h_vmem=MEMORY'.\r\n-- \r\n-- Found   1 host  with   2 cores and    7 GB memory under Sun Grid Engine control.\r\n-- Found   2 hosts with   8 cores and   31 GB memory under Sun Grid Engine control.\r\n--\r\n-- Run under grid control using    7 GB and   2 CPUs for stage 'meryl'.\r\n-- Run under grid control using   13 GB and   8 CPUs for stage 'mhap (cor)'.\r\n-- Run under grid control using    7 GB and   2 CPUs for stage 'overlapper (obt)'.\r\n-- Run under grid control using    7 GB and   2 CPUs for stage 'overlapper (utg)'.\r\n-- Run under grid control using    7 GB and   2 CPUs for stage 'falcon_sense'.\r\n-- Run under grid control using    3 GB and   1 CPU  for stage 'ovStore bucketizer'.\r\n-- Run under grid control using    8 GB and   1 CPU  for stage 'ovStore sorting'.\r\n-- Run under grid control using    6 GB and   2 CPUs for stage 'read error detection'.\r\n-- Run under grid control using    2 GB and   1 CPU  for stage 'overlap error adjustment'.\r\n-- Run under grid control using   31 GB and   8 CPUs for stage 'bogart'.\r\n-- Run under grid control using    4 GB and   2 CPUs for stage 'GFA alignment and processing'.\r\n-- Run under grid control using   31 GB and   8 CPUs for stage 'consensus'.\r\n--\r\n-- Generating assembly 'F_vert2' in '/shared/F_vert2_auto'\r\n--\r\n-- Parameters:\r\n--\r\n--  genomeSize        50000000\r\n--\r\n--  Overlap Generation Limits:\r\n--    corOvlErrorRate 0.2400 ( 24.00%)\r\n--    obtOvlErrorRate 0.0450 (  4.50%)\r\n--    utgOvlErrorRate 0.0450 (  4.50%)\r\n--\r\n--  Overlap Processing Limits:\r\n--    corErrorRate    0.3000 ( 30.00%)\r\n--    obtErrorRate    0.0450 (  4.50%)\r\n--    utgErrorRate    0.0450 (  4.50%)\r\n--    cnsErrorRate    0.0450 (  4.50%)\r\n--\r\n--\r\n-- BEGIN CORRECTION\r\n--\r\n-- Meryl finished successfully.\r\n-- Finished stage 'merylCheck', reset canuIteration.\r\n--\r\n-- WARNING: gnuplot failed; no plots will appear in HTML output.\r\n--\r\n----------------------------------------\r\n--\r\n-- WARNING: gnuplot failed; no plots will appear in HTML output.\r\n--\r\n----------------------------------------\r\nUse of uninitialized value $error[0] in join or string at /usr/share/perl/5.18/Carp.pm line 301.\r\n================================================================================\r\nPlease panic.  Canu failed, and it shouldn't have.\r\n\r\nStack trace:\r\n\r\n at /home/ubuntu/canu-1.5/Linux-amd64/bin/lib/canu/Meryl.pm line 668.\r\n\tcanu::Meryl::merylProcess('F_vert2', 'cor') called at /home/ubuntu/canu-1.5/Linux-amd64/bin/canu line 536\r\n\r\n\r\nLast few lines of the relevant log file (correction/0-mercounts/F_vert2.ms16.histogram.info):\r\n\r\nmerylStreamReader()-- ERROR: ./F_vert2.ms16.mcidx is not a merylStream index file!\r\nmerylStreamReader()-- ERROR: ./F_vert2.ms16.mcdat is not a merylStream data file!\r\n\r\nCanu release v1.5 failed with:\r\n  didn't find any mers?\r\n```\r\n\r\nHere is the output of `correction/0-mercounts/F_vert2.ms16.histogram.info`:\r\n```\r\nmerylStreamReader()-- ERROR: ./F_vert2.ms16.mcidx is not a merylStream index file!\r\nmerylStreamReader()-- ERROR: ./F_vert2.ms16.mcdat is not a merylStream data file!\r\n```\r\n\r\nFound some errors in `correction/0-mercounts/meryl.1.out`\r\n```\r\nComputing 2 segments using 2 threads and 5734MB memory (3593MB if in one batch).\r\n  numMersActual      = 5321595752\r\n  mersPerBatch       = 3951034368\r\n  basesPerBatch      = 2674092031\r\n  numBuckets         = 134217728 (27 bits)\r\n  bucketPointerWidth = 32\r\n  merDataWidth       = 5\r\nComputing segment 1 of 2.\r\n Allocating 512MB for bucket pointer table (32 bits wide).\r\n Allocating 512MB for counting the size of each bucket.\r\nComputing segment 2 of 2.\r\n Allocating 512MB for bucket pointer table (32 bits wide).\r\n Allocating 512MB for counting the size of each bucket.\r\n Counting mers in buckets: 2660.75 Mmers -- 16.75 Mmers/second\r\n Creating bucket pointers.\r\n Releasing 512MB from counting the size of each bucket./second\r\n Allocating 1593MB for mer storage (5 bits wide).\r\n Counting mers in buckets: 2660.85 Mmers -- 16.48 Mmers/second\r\n Creating bucket pointers.\r\n Releasing 512MB from counting the size of each bucket./second\r\n Allocating 1593MB for mer storage (5 bits wide).\r\nterminate called after throwing an instance of 'std::bad_alloc'\r\n  what():  std::bad_alloc\r\nCan fit 7902068736 mers into table with prefix of 27 bits, using 5734.000MB (0.000MB for positions)\r\nAborted (core dumped)\r\n```\r\n\r\nAlso some errors in `correction/0-mercounts/F_vert2.ms16.estMerThresh.err`\r\n```\r\nFailed with 'Segmentation fault'; backtrace (libbacktrace):\r\nAS_UTL/AS_UTL_stackTrace.C::102 in _Z17AS_UTL_catchCrashiP7siginfoPv()\r\n(null)::0 in (null)()\r\n(null)::0 in (null)()\r\nmeryl/estimate-mer-threshold.C::113 in _Z13loadHistogramP8_IO_FILERmS1_S1_RjRPj()\r\nmeryl/estimate-mer-threshold.C::199 in main()\r\n(null)::0 in (null)()\r\n(null)::0 in (null)()\r\nSegmentation fault (core dumped)\r\n```"},"comment":{"url":"https://api.github.com/repos/marbl/canu/issues/comments/311769350","html_url":"https://github.com/marbl/canu/issues/538#issuecomment-311769350","issue_url":"https://api.github.com/repos/marbl/canu/issues/538","id":311769350,"user":{"login":"skoren","id":923478,"avatar_url":"https://avatars1.githubusercontent.com/u/923478?v=3","gravatar_id":"","url":"https://api.github.com/users/skoren","html_url":"https://github.com/skoren","followers_url":"https://api.github.com/users/skoren/followers","following_url":"https://api.github.com/users/skoren/following{/other_user}","gists_url":"https://api.github.com/users/skoren/gists{/gist_id}","starred_url":"https://api.github.com/users/skoren/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/skoren/subscriptions","organizations_url":"https://api.github.com/users/skoren/orgs","repos_url":"https://api.github.com/users/skoren/repos","events_url":"https://api.github.com/users/skoren/events{/privacy}","received_events_url":"https://api.github.com/users/skoren/received_events","type":"User","site_admin":false},"created_at":"2017-06-28T19:55:19Z","updated_at":"2017-06-28T19:55:19Z","body":"The canu job is running out of memory, usually h_vmem is not the correct parameter because it is not consumable and also isn't scaled by threads. Since canu divides the memory by the threads requested it would be under-requesting memory it needs. You can check your configuration using the ```qconf``` command:\r\n```\r\n#name               shortcut   type        relop requestable consumable default  urgency \r\n#----------------------------------------------------------------------------------------\r\nh_vmem              h_vmem     MEMORY      <=    YES         NO         0        0\r\nmem_free            mf         MEMORY      <=    YES         YES        0        0\r\n```\r\n\r\nIf you have mem_free (or another requestable/consumable) memory option I would use that for MEMORY instead of h_vmem."}}
{"actor":{"display_login":"skoren"}}