| INFO:__main__: Optimizer = adafactor | |
| INFO:__main__: Learning rate (peak) = 0.005 | |
| INFO:__main__: Num examples = 31519126 | |
| INFO:__main__: Num tokenized group examples 36347268 | |
| INFO:__main__: Num Epochs = 10 | |
| INFO:__main__: Instantaneous batch size per device = 16 | |
| INFO:__main__: Total train batch size (w. parallel & grad accum) = 128 | |
| INFO:__main__: Steps per epoch = 283963 | |
| INFO:__main__: Total optimization steps = 2839630 |