title: How to tune hyper-parameters with validation data
body: <p>I'm training a neural network machine learning model and am a little bit confused about how to tune hyper-parameters.</p>
  
  <p>I see the training process as follows:</p>
  
  <ul>
  <li>build neural network with some hyper parameters (e.g. number of neurons in hidden layer)</li>
  <li>update parameters using training data</li>
  <li>use validation data to assess if model is not over-fitting and is improving</li>
  <li>use test data to assess performance</li>
  </ul>
  
  <p>I've been told that validation data is also used to tune hyper-parameters. I don't understand how to do this since you use the validation data <strong>after</strong> you've built the model and trained it. I can't change a hyper-parameter because the model is already built.</p>
  
  <p>Assuming I have 4 hyper-parameters which specify the architecture of a neural network. Am I supposed to go through the whole training, validation process for every hyper-parameter combination (4^4 combinations)?</p>
  
  <p>Thank you</p>