{"action":"created","issue":{"url":"https://api.github.com/repos/open-mpi/ompi/issues/1763","repository_url":"https://api.github.com/repos/open-mpi/ompi","labels_url":"https://api.github.com/repos/open-mpi/ompi/issues/1763/labels{/name}","comments_url":"https://api.github.com/repos/open-mpi/ompi/issues/1763/comments","events_url":"https://api.github.com/repos/open-mpi/ompi/issues/1763/events","html_url":"https://github.com/open-mpi/ompi/issues/1763","id":158904650,"number":1763,"title":"Tuned bcast data corruption","user":{"login":"nysal","id":8509287,"avatar_url":"https://avatars.githubusercontent.com/u/8509287?v=3","gravatar_id":"","url":"https://api.github.com/users/nysal","html_url":"https://github.com/nysal","followers_url":"https://api.github.com/users/nysal/followers","following_url":"https://api.github.com/users/nysal/following{/other_user}","gists_url":"https://api.github.com/users/nysal/gists{/gist_id}","starred_url":"https://api.github.com/users/nysal/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/nysal/subscriptions","organizations_url":"https://api.github.com/users/nysal/orgs","repos_url":"https://api.github.com/users/nysal/repos","events_url":"https://api.github.com/users/nysal/events{/privacy}","received_events_url":"https://api.github.com/users/nysal/received_events","type":"User","site_admin":false},"labels":[{"id":129984007,"url":"https://api.github.com/repos/open-mpi/ompi/labels/bug","name":"bug","color":"fc2929","default":true}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":{"url":"https://api.github.com/repos/open-mpi/ompi/milestones/17","html_url":"https://github.com/open-mpi/ompi/milestone/17","labels_url":"https://api.github.com/repos/open-mpi/ompi/milestones/17/labels","id":1604178,"number":17,"title":"v3.0.0","description":"Release managers:\r\n* Howard Pritchard\r\n* ...TBD...\r\n\r\nv3.0.0 will be a new fork from the master branch, and will have some backwards-incompatibilities with the v2.x series.","creator":{"login":"jsquyres","id":617316,"avatar_url":"https://avatars.githubusercontent.com/u/617316?v=3","gravatar_id":"","url":"https://api.github.com/users/jsquyres","html_url":"https://github.com/jsquyres","followers_url":"https://api.github.com/users/jsquyres/followers","following_url":"https://api.github.com/users/jsquyres/following{/other_user}","gists_url":"https://api.github.com/users/jsquyres/gists{/gist_id}","starred_url":"https://api.github.com/users/jsquyres/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jsquyres/subscriptions","organizations_url":"https://api.github.com/users/jsquyres/orgs","repos_url":"https://api.github.com/users/jsquyres/repos","events_url":"https://api.github.com/users/jsquyres/events{/privacy}","received_events_url":"https://api.github.com/users/jsquyres/received_events","type":"User","site_admin":false},"open_issues":30,"closed_issues":7,"state":"open","created_at":"2016-02-24T23:58:35Z","updated_at":"2017-02-21T16:21:34Z","due_on":"2017-06-17T07:00:00Z","closed_at":null},"comments":11,"created_at":"2016-06-07T11:55:55Z","updated_at":"2017-02-21T16:21:56Z","closed_at":null,"body":"There are multiple issues with bcast algorithms when the tasks involved specify different counts and datatypes for a given bcast operation. \n\nOne low hanging fix is :\n\n``` diff\n--- a/ompi/mca/coll/tuned/coll_tuned_decision_fixed.c\n+++ b/ompi/mca/coll/tuned/coll_tuned_decision_fixed.c\n@@ -252,7 +252,7 @@ int ompi_coll_tuned_bcast_intra_dec_fixed(void *buff, int count,\n\n     /* Handle messages of small and intermediate size, and\n        single-element broadcasts */\n-    if ((message_size < small_message_size) || (count <= 1)) {\n+    if (message_size < small_message_size) {\n         /* Binomial without segmentation */\n         segsize = 0;\n         return  ompi_coll_base_bcast_intra_binomial(buff, count, datatype,\n```\n\nI'm seeing issues with both the pipelined and split binary tree algorithms. This is probably a known issue with pipelined algorithms in tuned module. We should probably start discussing the right approach to solve this.\n\nHere's a reproducer:\n\n``` C\n#include <stdio.h>\n#include <stdlib.h>\n#include \"mpi.h\"\n\n#define MAXLEN 10000000\n\nint \nmain(int argc, char **argv)\n{\n   int root,*out,i,j,k;\n   int myself,tasks;\n   MPI_Datatype type;\n\n   MPI_Init(&argc,&argv);\n   MPI_Comm_rank(MPI_COMM_WORLD,&myself);\n   MPI_Comm_size(MPI_COMM_WORLD,&tasks);\n   MPI_Type_contiguous(10, MPI_INT, &type);\n   MPI_Type_commit(&type);\n   /* Add this section because some os/architectures can't allocate\n      this much memory on the stack */\n\n   out = malloc(sizeof(int) * MAXLEN);\n   if (out == NULL) {\n     printf( \"Doh!  Rank %d was not able to allocate enough memory.  MPI test aborted!\\n\", myself);\n     exit(1);\n   }\n\n   root = tasks-1;\n   for (j=1000000;j<=MAXLEN;j*=10)  {\n      if (myself == root)\n         for (i=0;i<j;i++)  \n       out[i] = i;\n      fprintf(stderr,\"bcast %d integers\\n\",j);\n      if(myself == root) {\n          MPI_Bcast(out,j,MPI_INT,root,MPI_COMM_WORLD);\n      } else {\n          MPI_Bcast(out,j/10,type,root,MPI_COMM_WORLD);\n      }\n\n      for (k=0;k<j;k++) {\n         if (out[k] != k) {  \n       printf(\"bad answer (%d) at index %d of %d (should be %d)\\n\",out[k],k,j,k); \n           exit(1);\n     }\n      }\n   }\n   MPI_Barrier(MPI_COMM_WORLD);\n   printf(\"Rank %d exiting successfully\\n\", myself);\n   MPI_Type_free(&type);\n   MPI_Finalize();\n\n   /* Clean up, 'cause it's the Right Thing to do */\n\n   free(out);\n\n   return 0;\n}\n```\n"},"comment":{"url":"https://api.github.com/repos/open-mpi/ompi/issues/comments/281394140","html_url":"https://github.com/open-mpi/ompi/issues/1763#issuecomment-281394140","issue_url":"https://api.github.com/repos/open-mpi/ompi/issues/1763","id":281394140,"user":{"login":"jsquyres","id":617316,"avatar_url":"https://avatars.githubusercontent.com/u/617316?v=3","gravatar_id":"","url":"https://api.github.com/users/jsquyres","html_url":"https://github.com/jsquyres","followers_url":"https://api.github.com/users/jsquyres/followers","following_url":"https://api.github.com/users/jsquyres/following{/other_user}","gists_url":"https://api.github.com/users/jsquyres/gists{/gist_id}","starred_url":"https://api.github.com/users/jsquyres/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jsquyres/subscriptions","organizations_url":"https://api.github.com/users/jsquyres/orgs","repos_url":"https://api.github.com/users/jsquyres/repos","events_url":"https://api.github.com/users/jsquyres/events{/privacy}","received_events_url":"https://api.github.com/users/jsquyres/received_events","type":"User","site_admin":false},"created_at":"2017-02-21T16:21:56Z","updated_at":"2017-02-21T16:21:56Z","body":"Per discussion on the 2017-02-21 call, removed the blocker label and moved to v3.0.0."}}
{"actor":{"display_login":"jsquyres"}}