Huggingface --resume_from_checkpoint feature with deepspeed
This is an appreciation post for the feature--resume_from_checkpointwhich actually works withdeepspeedand do iterate data up-to certain iteration to mimic the same experiment.Even we can sync loss in wandb.A big thumbs up for@sgugger.