Reinforcement Learning for Load-balanced Parallel Particle Tracing
  We explore an online reinforcement learning (RL) paradigm to dynamically
optimize parallel particle tracing performance in distributed-memory systems.
Our method combines three novel components: (1) a work donation algorithm, (2)
a high-order workload estimation model, and (3) a communication cost model.
First, we design an RL-based work donation algorithm. Our algorithm monitors
workloads of processes and creates RL agents to donate data blocks and
particles from high-workload processes to low-workload processes to minimize
program execution time. The agents learn the donation strategy on the fly based
on reward and cost functions designed to consider processes' workload changes
and data transfer costs of donation actions. Second, we propose a workload
estimation model, helping RL agents estimate the workload distribution of
processes in future computations. Third, we design a communication cost model
that considers both block and particle data exchange costs, helping RL agents
make effective decisions with minimized communication costs. We demonstrate
that our algorithm adapts to different flow behaviors in large-scale fluid
dynamics, ocean, and weather simulation data. Our algorithm improves parallel
particle tracing performance in terms of parallel efficiency, load balance, and
costs of I/O and communication for evaluations with up to 16,384 processors.
