title: How nodes communicate in OpenMPI
body: <p>I am able to run OpenMPI job in multiple nodes under <code>ssh</code>. Everything looks good but I find that I do not know much about what is really happening. So, how nodes communicate in OpenMPI? It's in multiple nodes, hence it can not be shared memory. It also seems not TCP or UDP because I have not configured any port. Can anyone describe what happened when a message is sent and received between 2 processes in 2 nodes? THANKS!</p>