OpenCL - Multiple GPU Buffer Synchronization
===

I have an OpenCL kernel that calculates total force on a particle exerted by other particles in the system, and then another one that integrates the particle position/velocity. I would like to parallelize these kernels across multiple GPUs, basically assigning some amount of particles to each GPU. However, I have to run this kernel multiple times, and the result from each GPU is used on every other. Let me explain that a little further:

Say you have particle 0 on GPU 0, and particle 1 on GPU 1. The force on particle 0 is changed, as is the force on particle 1, and then their positions and velocities are changed accordingly by the integrator. Then, these new positions need to be placed on each GPU (both GPUs need to know where both particle 0 and particle 1 are) and these new positions are used to calculate the forces on each particle in the next step, which is used by the integrator, whose results are used to calculate forces, etc, etc. **Essentially, all the buffers need to contain the same information by the time the force calculations roll around.**

So, the question is: **What is the best way to synchronize buffers across GPUs, given that each GPU has a different buffer**? They cannot have a single shared buffer if I want to keep parallelism, [as per my last question][1] (though, if there is a way to create a shared buffer and still keep multiple GPUs, I'm all for that). I suspect that copying the results each step will cause more slowdown than it's worth to parallelize the algorithm across GPUs.

I did find [this thread][2], but the answer was not very definitive and applied only to a single buffer across all GPUs. I would like to know, specifically, for Nvidia GPUs (more specifically, the Tesla M2090).


  [1]: http://stackoverflow.com/questions/11562543/clenqueuendrange-blocking-on-nvidia-hardware-also-multi-gpu
  [2]: http://stackoverflow.com/questions/11093826/read-write-opencl-memory-buffers-on-multiple-gpu-in-a-single-context