title: this.Controls.RemoveByKey doesn't remove as expected
body: <p>Here's my code:</p>
  
  <pre><code>private void modeRadioChecked(object sender, EventArgs e)
      {
          RadioButton me = (sender as RadioButton);
  
          if (me == modeVerifyBtn)
          {
              label4.Visible = false;
              categoryField.Visible = false;
  
              Button button = new Button();
              button.Text = "Select document..";
              button.Name = "selectDocBtn";
              button.Location = label4.Location;
  
              this.mode = SearchMode.VerifyCategories;
              this.Controls.Add(button);
          }
          else if(me == modeGscBtn)
          {
              this.Controls.RemoveByKey("selectDocBtn");
              this.mode = SearchMode.GetSkuCategories;
              label4.Visible = !label4.Visible;
              categoryField.Visible = !categoryField.Visible;
              categoryField.Enabled = false;
              dataGridView1.DataSource = this.skuResultsContainer;
          }
          else if (me == modeSearchCategoryBtn)
          {
              this.Controls.RemoveByKey("selectDocBtn");
              this.mode = SearchMode.SearchCategory;
              label4.Visible = !label4.Visible;
              categoryField.Visible = !categoryField.Visible;
              categoryField.Enabled = true;
              dataGridView1.DataSource = this.resultsContainer;
          }
      }
  </code></pre>
  
  <p>Checking the modeVerifyBtn creates the radiobutton as you would expect. However, when I check one of the other two radio buttons, it doesn't remove the radio button until I've clicked yet another radio button. i.e.</p>
  
  <p>Step 1) Click modeVerifyButton (button is there, as expected
  Step 2) Click modeGscBtn (button is still there, do not want this)
  Step 3) Click modeSearchCategoryBtn (button disappears)</p>
  
  <p>What's up with this?</p>