global batch size at different sequence length
This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days. This issue was closed because it has been inactive for 7 days since being marked as stale.
Are we supposed to use the same global batch size when increasing sequence_lengh?
Shouldn't the total number of tokens per global batch size remain the same?