Switch batch size and gradient accumulation step values mid training
Hello, I’m currently training a model on A40 GPU using batch size of 8 and gradient accumulation step 4. Training the model on A100 is almost twice as fast but it has 40 GB VRAM while A40 has 45 GB VRAM. If I switch the values to batch size 4 and gradient accumulation steps 8, will the results of the training stay not affected (stay the same as if I used batch size 8 and gradient acc. 4)?