MPS is recommended for jobs that do not generate enough work for GPUs. If you have  multiple runs concurrently, MPS helps to balance the workload of the GPU and increases its  utilization. Also, MPI programs that have different MPI-tasks but share a GPU can see performance  improvements by using MPS to control access to the GPU between tasks. Note:  the Volta MPS server supports 48 client CUDA contexts per-device. A recommendation is to use MPS with the  EXCLUSIVE_PROCESS  mode to be sure that only the  MPS server runs on GPUs.  The CUDA program works by using MPS if the MPS control daemon runs in the system. The  program at startup attempts to connect to the MPS control daemon, which creates an MPS  server or reuses a server if it has the same user ID as the user who started the job. Therefore,  each user has its own MPS server.  The MPS server creates the shared GPU context for the different jobs in the system,  manages its clients, and calls to GPU on behalf of them. An overview of this process is shown  in Figure 9-24. To run MPS, you need to run the following commands as root: export CUDA_VISIBLE_DEVICES=0,1 #Use only first and second GPUs 1. (This step is optional.) Set the  CUDA_VISIBLE_DEVICES  environment variable to inform the  system which GPUs will be used. To use all GPUs, start from step 2 on page 294 and  issue the following command: Chapter 9. Measuring and tuning applications  293 Figure 9-24 NVIDIA MPS