title: Django: my test migration throws a Key Error but normal migration is fine. What's broken and how do I fix it?
body: <p>Django test migration throws Key Error </p>
  
  <p>I get a Key error, but only when I do <code>python manage.py test apps/userprofile</code> and not when I do <code>python manage.py makemigrations</code> or <code>python manage.py migrate</code>.</p>
  
  <p>Any suggestions on how I can fix this? </p>
  
  <hr>
  
  <pre><code>python manage.py test apps/userprofile
  Creating test database for alias 'default'...
  Got an error creating the test database: database "test_financetracker" already exists
  
  Type 'yes' if you would like to try deleting the test database 'test_financetracker', or 'no' to cancel: yes
  Destroying old test database for alias 'default'...
  Traceback (most recent call last):
    File "manage.py", line 16, in &lt;module&gt;
      execute_from_command_line(sys.argv)
    File "/Users/financetracker/myvenv/lib/python3.7/site-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
      utility.execute()
    File "/Users/financetracker/myvenv/lib/python3.7/site-packages/django/core/management/__init__.py", line 375, in execute
      self.fetch_command(subcommand).run_from_argv(self.argv)
    File "/Users/financetracker/myvenv/lib/python3.7/site-packages/django/core/management/commands/test.py", line 23, in run_from_argv
      super().run_from_argv(argv)
    File "/Users/financetracker/myvenv/lib/python3.7/site-packages/django/core/management/base.py", line 323, in run_from_argv
      self.execute(*args, **cmd_options)
    File "/Users/financetracker/myvenv/lib/python3.7/site-packages/django/core/management/base.py", line 364, in execute
      output = self.handle(*args, **options)
    File "/Users/financetracker/myvenv/lib/python3.7/site-packages/django/core/management/commands/test.py", line 53, in handle
      failures = test_runner.run_tests(test_labels)
    File "/Users/financetracker/myvenv/lib/python3.7/site-packages/django/test/runner.py", line 629, in run_tests
      old_config = self.setup_databases(aliases=databases)
    File "/Users/financetracker/myvenv/lib/python3.7/site-packages/django/test/runner.py", line 554, in setup_databases
      self.parallel, **kwargs
    File "/Users/financetracker/myvenv/lib/python3.7/site-packages/django/test/utils.py", line 174, in setup_databases
      serialize=connection.settings_dict.get('TEST', {}).get('SERIALIZE', True),
    File "/Users/financetracker/myvenv/lib/python3.7/site-packages/django/db/backends/base/creation.py", line 72, in create_test_db
      run_syncdb=True,
    File "/Users/financetracker/myvenv/lib/python3.7/site-packages/django/core/management/__init__.py", line 148, in call_command
      return command.execute(*args, **defaults)
    File "/Users/financetracker/myvenv/lib/python3.7/site-packages/django/core/management/base.py", line 364, in execute
      output = self.handle(*args, **options)
    File "/Users/financetracker/myvenv/lib/python3.7/site-packages/django/core/management/base.py", line 83, in wrapped
      res = handle_func(*args, **kwargs)
    File "/Users/financetracker/myvenv/lib/python3.7/site-packages/django/core/management/commands/migrate.py", line 234, in handle
      fake_initial=fake_initial,
    File "/Users/financetracker/myvenv/lib/python3.7/site-packages/django/db/migrations/executor.py", line 117, in migrate
      state = self._migrate_all_forwards(state, plan, full_plan, fake=fake, fake_initial=fake_initial)
    File "/Users/financetracker/myvenv/lib/python3.7/site-packages/django/db/migrations/executor.py", line 147, in _migrate_all_forwards
      state = self.apply_migration(state, migration, fake=fake, fake_initial=fake_initial)
    File "/Users/financetracker/myvenv/lib/python3.7/site-packages/django/db/migrations/executor.py", line 245, in apply_migration
      state = migration.apply(state, schema_editor)
    File "/Users/financetracker/myvenv/lib/python3.7/site-packages/django/db/migrations/migration.py", line 114, in apply
      operation.state_forwards(self.app_label, project_state)
    File "/Users/financetracker/myvenv/lib/python3.7/site-packages/django/db/migrations/operations/fields.py", line 165, in state_forwards
      for name, instance in state.models[app_label, self.model_name_lower].fields:
  KeyError: ('hourly_rate', 'hourlyrate')
  </code></pre>
  
  <p>This is the relevant (I think) <code>hourly_rates</code> python class from the models.py file:</p>
  
  <pre><code>class HourlyRate(TimeStampedModel):
      # TODO remove user PK as it will no longer be used.
      user = models.OneToOneField(
          User, blank=True, null=True, on_delete=models.CASCADE)
      upload = models.ForeignKey(
          'hourly_rate.Upload', blank=True, null=True, on_delete=models.CASCADE)
      customer = models.ForeignKey(Customer, on_delete=models.CASCADE)
      vendor = models.ForeignKey(Vendor, on_delete=models.CASCADE)
  
      name = models.CharField(max_length=255)
      timekeeper = models.CharField(verbose_name='Timekeeper ID', max_length=20,
                                    blank=True)  # is required, but blank = True and null = True for backwards compatiblity.
      classification = models.CharField(max_length=20, choices=CLASSIFICATIONS)
      years_of_experience = models.PositiveIntegerField(blank=True,
                                                        null=True)  # is required, but blank = True and null = True for backwards compatiblity.
      hourly_rate = models.PositiveIntegerField(default=1)
      currency = models.ForeignKey(Currency, on_delete=models.CASCADE)
  
      status = models.CharField(choices=UPLOAD_STATUSES,
                                default='requested', max_length=255)
      fiscal_year = models.PositiveIntegerField(choices=YEARS, default=2017)
  
      def __str__(self):
          return '{}, {}'.format(self.name, self.timekeeper)
  
      def status_and_date(self):
          """Return the status and date of the upload, if an upload is available. Else: return status. Might change this later to -"""
          if self.upload:
              return self.upload.status_and_date
          else:
              return self.status  # idea, change to: "-"
  
      class Meta:
          ordering = ('user__userprofile__name', '-id',)
          verbose_name = 'Timekeeper rate'
  </code></pre>
  
  <p>This is the directory flow, although I removed quite some files for simplicity, so if something is missing, just ask.</p>
  
  <pre><code>manage.py
  config
  ├── __init__.py
  ├── __pycache__
  │   └── __init__.cpython-37.pyc
  ├── alpha
  │   ├── nginx
  │   └── supervisor
  ├── default.conf
  ├── demo
  │   ├── nginx
  │   └── supervisor
  ├── settings
  │   ├── __init__.py
  │   ├── __pycache__
  │   │   ├── __init__.cpython-37.pyc
  │   │   ├── base.cpython-37.pyc
  │   │   ├── defaults.cpython-37.pyc
  │   │   └── development.cpython-37.pyc
  │   ├── alpha.py
  │   ├── base.py
  │   ├── defaults.py
  │   ├── demo.py
  │   ├── development.py
  │   ├── local.py
  │   ├── production.py
  │   └── staging.py
  ├── staging
  │   ├── nginx
  │   └── supervisor
  └── wsgi.py
  apps
  └── report
  └── hourly_rate
      ├── __init__.py
      ├── __pycache__
      │   ├── __init__.cpython-37.pyc
      │   ├── admin.cpython-37.pyc
      │   ├── apps.cpython-37.pyc
      │   ├── filters.cpython-37.pyc
      │   ├── forms.cpython-37.pyc
      │   ├── helpers.cpython-37.pyc
      │   ├── models.cpython-37.pyc
      │   ├── urls.cpython-37.pyc
      │   └── views.cpython-37.pyc
      ├── admin.py
      ├── filters.py
      ├── forms.py
      ├── helpers.py
      ├── migrations
      │   ├── 0001_initial.py
      │   ├── 0002_auto_20170424_1735.py
      │   ├── 0003_auto_20170424_1738.py
      │   ├── 0004_auto_20170424_1818.py
      │   ├── 0005_auto_20170424_1917.py
      │   ├── 0006_auto_20170424_2302.py
      │   ├── 0007_auto_20170504_1948.py
      │   ├── 0008_auto_20170513_1412.py
      │   ├── 0009_auto_20170613_1726.py
      │   ├── 0010_auto_20171115_1051.py
      │   ├── 0011_auto_20180311_1223.py
      │   ├── 0012_auto_20190801_1703.py
      │   ├── __init__.py
      │   └── __pycache__
      │       ├── 0001_initial.cpython-37.pyc
      │       ├── 0002_auto_20170424_1735.cpython-37.pyc
      │       ├── 0003_auto_20170424_1738.cpython-37.pyc
      │       ├── 0004_auto_20170424_1818.cpython-37.pyc
      │       ├── 0005_auto_20170424_1917.cpython-37.pyc
      │       ├── 0006_auto_20170424_2302.cpython-37.pyc
      │       ├── 0007_auto_20170504_1948.cpython-37.pyc
      │       ├── 0008_auto_20170513_1412.cpython-37.pyc
      │       ├── 0009_auto_20170613_1726.cpython-37.pyc
      │       ├── 0010_auto_20171115_1051.cpython-37.pyc
      │       ├── 0011_auto_20180311_1223.cpython-37.pyc
      │       ├── 0012_auto_20190801_1703.cpython-37.pyc
      │       └── __init__.cpython-37.pyc
      ├── models.py
      ├── urls.py
      └── views.py
  </code></pre>
  
  <p><strong>Django version is 2.2.9</strong></p>
  
  <p>Migration files, many automatic ones. Here are the first three of twelve: </p>
  
  <p><strong>initial</strong></p>
  
  <pre><code># -*- coding: utf-8 -*-
  from __future__ import unicode_literals
  
  from django.conf import settings
  from django.db import migrations, models
  
  
  class Migration(migrations.Migration):
      dependencies = [
          ('customer', '0002_historicalbillingaddress_historicalentity'),
          migrations.swappable_dependency(settings.AUTH_USER_MODEL),
      ]
  
      operations = [
          migrations.CreateModel(
              name='Upload',
              fields=[
                  ('id', models.AutoField(verbose_name='ID',
                                          serialize=False, auto_created=True, primary_key=True)),
                  ('created', models.DateTimeField(null=True, editable=False)),
                  ('updated', models.DateTimeField(null=True, editable=False)),
                  ('excel_file', models.FileField(help_text=b'Kies het door u ingevulde bestand.', upload_to=b'uploads',
                                                  verbose_name=b'Upload')),
                  ('period', models.DateField(null=True, blank=True)),
                  ('status', models.CharField(default=b'upload failed', max_length=255,
                                              choices=[(b'upload failed', b'upload failed'), (b'requested', b'requested'),
                                                       (b'rejected', b'rejected'), (b'approved', b'approved')])),
                  ('customer', models.ForeignKey(
                      to='customer.Customer', on_delete=models.CASCADE)),
                  ('user', models.ForeignKey(verbose_name=b'uploaded by user',
                                             to=settings.AUTH_USER_MODEL, on_delete=models.CASCADE)),
              ],
              options={
                  'abstract': False,
              },
          ),
      ]
  
  
  </code></pre>
  
  <p><strong>second, automatic from now on, but this one was edited</strong></p>
  
  <pre><code>
  # -*- coding: utf-8 -*-
  from __future__ import unicode_literals
  
  from django.conf import settings
  from django.db import migrations, models
  
  
  class Migration(migrations.Migration):
      dependencies = [
          ('vendor', '0008_auto_20170417_1656'),
          ('customer', '0002_historicalbillingaddress_historicalentity'),
          migrations.swappable_dependency(settings.AUTH_USER_MODEL),
          ('currency', '0001_initial'),
          ('hourly_rate', '0001_initial'),
      ]
  
      operations = [
          migrations.CreateModel(
              name='HourlyRate',
              fields=[
                  ('id', models.AutoField(verbose_name='ID', serialize=False, auto_created=True, primary_key=True)),
                  ('created', models.DateTimeField(null=True, editable=False)),
                  ('updated', models.DateTimeField(null=True, editable=False)),
                  ('name', models.CharField(max_length=255, blank=True)),
                  ('classification', models.CharField(max_length=20,
                                                      choices=[(b'AS', b'AS'), (b'OC', b'OC'), (b'OT', b'OT'),
                                                               (b'PL', b'PL'), (b'PT', b'PT')])),
                  ('hourly_rate', models.PositiveIntegerField()),
                  ('status', models.CharField(default=b'approved', max_length=255,
                                              choices=[(b'upload failed', b'upload failed'), (b'requested', b'requested'),
                                                       (b'rejected', b'rejected'), (b'approved', b'approved')])),
                  ('period', models.DateField(null=True, blank=True)),
                  ('currency', models.ForeignKey(to='currency.Currency', on_delete=models.CASCADE)),
                  ('customer', models.ForeignKey(default=1, to='customer.Customer', on_delete=models.CASCADE)),
              ],
              options={
                  'ordering': ('user__userprofile__name', '-id'),
              },
          ),
          migrations.AlterModelOptions(
              name='upload',
              options={'ordering': ['-id']},
          ),
          migrations.AddField(
              model_name='hourlyrate',
              name='upload',
              field=models.ForeignKey(blank=True, to='hourly_rate.Upload', null=True, on_delete=models.CASCADE),
          ),
          migrations.AddField(
              model_name='hourlyrate',
              name='user',
              field=models.OneToOneField(null=True, blank=True, to=settings.AUTH_USER_MODEL, on_delete=models.CASCADE),
          ),
          migrations.AddField(
              model_name='hourlyrate',
              name='vendor',
              field=models.ForeignKey(blank=True, to='vendor.Vendor', null=True, on_delete=models.CASCADE),
          ),
      ]
  
      def apply(self, project_state, schema_editor, collect_sql=False):
          return project_state.clone()
  
      def unapply(self, project_state, schema_editor, collect_sql=False):
          return project_state.clone()
  
  </code></pre>
  
  <p><strong>second as before the edit</strong></p>
  
  <pre><code># -*- coding: utf-8 -*-
  from __future__ import unicode_literals
  
  from django.db import migrations, models
  from django.conf import settings
  
  
  class Migration(migrations.Migration):
  
      dependencies = [
          ('vendor', '0008_auto_20170417_1656'),
          ('customer', '0002_historicalbillingaddress_historicalentity'),
          migrations.swappable_dependency(settings.AUTH_USER_MODEL),
          ('currency', '0001_initial'),
          ('hourly_rate', '0001_initial'),
      ]
  
      operations = [
          migrations.CreateModel(
              name='HourlyRate',
              fields=[
                  ('id', models.AutoField(verbose_name='ID', serialize=False, auto_created=True, primary_key=True)),
                  ('created', models.DateTimeField(null=True, editable=False)),
                  ('updated', models.DateTimeField(null=True, editable=False)),
                  ('name', models.CharField(max_length=255, blank=True)),
                  ('classification', models.CharField(max_length=20, choices=[(b'AS', b'AS'), (b'OC', b'OC'), (b'OT', b'OT'), (b'PL', b'PL'), (b'PT', b'PT')])),
                  ('hourly_rate', models.PositiveIntegerField()),
                  ('status', models.CharField(default=b'approved', max_length=255, choices=[(b'upload failed', b'upload failed'), (b'requested', b'requested'), (b'rejected', b'rejected'), (b'approved', b'approved')])),
                  ('period', models.DateField(null=True, blank=True)),
                  ('currency', models.ForeignKey(to='currency.Currency')),
                  ('customer', models.ForeignKey(default=1, to='customer.Customer')),
              ],
              options={
                  'ordering': ('user__userprofile__name', '-id'),
              },
          ),
          migrations.AlterModelOptions(
              name='upload',
              options={'ordering': ['-id']},
          ),
          migrations.AddField(
              model_name='hourlyrate',
              name='upload',
              field=models.ForeignKey(blank=True, to='hourly_rate.Upload', null=True),
          ),
          migrations.AddField(
              model_name='hourlyrate',
              name='user',
              field=models.OneToOneField(null=True, blank=True, to=settings.AUTH_USER_MODEL),
          ),
          migrations.AddField(
              model_name='hourlyrate',
              name='vendor',
              field=models.ForeignKey(blank=True, to='vendor.Vendor', null=True),
          ),
      ]
  
      def apply(self, project_state, schema_editor, collect_sql=False):
          return project_state.clone()
  
      def unapply(self, project_state, schema_editor, collect_sql=False):
          return project_state.clone()
  </code></pre>
  
  <p><strong>third</strong></p>
  
  <pre><code># -*- coding: utf-8 -*-
  from __future__ import unicode_literals
  
  from django.db import migrations, models
  
  
  class Migration(migrations.Migration):
      dependencies = [
          ('hourly_rate', '0002_auto_20170424_1735'),
      ]
  
      operations = [
          migrations.RemoveField(
              model_name='hourlyrate',
              name='period',
          ),
          migrations.RemoveField(
              model_name='upload',
              name='period',
          ),
          migrations.AddField(
              model_name='hourlyrate',
              name='fiscal_year',
              field=models.PositiveIntegerField(default=2017,
                                                choices=[(2017, 2017), (2018, 2018), (2019, 2019), (2020, 2020),
                                                         (2021, 2021), (2022, 2022), (2023, 2023), (2024, 2024)]),
          ),
          migrations.AddField(
              model_name='upload',
              name='fiscal_year',
              field=models.PositiveIntegerField(default=2017,
                                                choices=[(2017, 2017), (2018, 2018), (2019, 2019), (2020, 2020),
                                                         (2021, 2021), (2022, 2022), (2023, 2023), (2024, 2024)]),
          ),
      ]
  
  </code></pre>
  
  <p><a href="https://i.stack.imgur.com/Q1w8g.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/Q1w8g.png" alt="debugging on the line it breaks at"></a></p>
  
  <p>Debugged the line that throws the error.</p>
  
  <p><a href="https://i.stack.imgur.com/C99ty.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/C99ty.png" alt="enter image description here"></a>
  Is it the migrations? Is it the code?</p>
  
  <p>Thanks</p>