CUDA out of memory when running on multiple GPUs
Hey guys,I’m running some basic tests for preparing a fine tuning job for mT5-large using aws Sagemaker. With ap3.2xlargeinstance (1 Tesla V100-GPU) and the following setting, the job runs successfully:per_device_train_batch_size=16
gradient_accumulation_steps=8
gradient_checkpointing=TrueBut if I usep3.8xlarge(4 GPUs) with the settingper_device_train_batch_size=4
gradient_accumulation_steps=8
gradient_checkpointing=TrueI run into a memory error. Any idea how I should adapt the parameters?