title: What does "learning rate warm-up" mean?
body: <p>In machine learning, especially deep learning, what does it mean to warm-up?</p>
  <p>I've heard sometimes that in some models, warming-up is a phase in training. But honestly, I don't know what it is because I'm very new to ML. Until now I've never used or come across it, but I want to know it because I think it might be useful for me.</p>
  <p><strong>What is learning rate warm-up and when do we need it?</strong></p>