How to determine throughput (FLOPs) and samples per second for each step
I have been trying to implement the logging info for calculating the throughput and samples per second for each step while pretraining a Megatron GPT model.

Any idea on how to include this information along with logging infos on loss, global step, consumed samples etc. for each step.