title: Python unittest: run same test over a list of values, but treat them as separate tests
body: <p>I'm writing a test to validate that my program is able to solve the problem at hand for different levels of complexity. The expected outcome is always the same (solution is complete), so a single test definition works for all problems.</p>
  
  <p><strong>How do I run the <em>same test</em> for a list of values read from a file, but tell unittest to <em>treat each of these problems as a separate test</em> so I can pinpoint all failing/passing cases?</strong> (Preferably no external libraries)</p>
  
  <p>To avoid explicitly having <code>test_solution_1, test_solution_2... test_solution_n</code>, my initial thought was to have a for loop run through each item of the list, and run the assertions one at a time:</p>
  
  <pre><code>class TestProblem(unittest.TestCase):
      def test_all(self):
          results = Counter()
          rng = 50
          for i in range(rng):
              p = Problem(matrix_index=i)  # generate instance of problem.
              p.solve()
              results[p.is_complete()] += 1  # log result of test.
              self.assertTrue(p.is_complete(), msg="Failed at %s" % str(i))
          # Now ensure all instances passed (useful when above assertion not included).
          self.assertEqual(results[True], rng, msg="SCORE: %s / %s" % (results[True], rng))
  </code></pre>
  
  <p>The problem with this approach is that the first failure stops the rest from running, so its more difficult to get a complete picture of what is wrong.</p>