title: le.transform() ValueError: y contains previously unseen labels: [1, 2, 3, 4]
body: <p>I'm running a very basic code to create encoder classes, and then use the same classes to encode a new dataframe. In this code, I don't need to use <code>np.save</code> and <code>np.load</code>, however in my actual implementation, I will need to re-load the encoder to transform a new dataframe. I'm trying to understand how I can create an encoder class on one dataframe, and then in another script, load that encoder and transform a new dataframe.</p>
  <pre><code>from sklearn.preprocessing import LabelEncoder
  import pickle as cPickle
  import numpy as np
  
  df_test = pd.DataFrame({'A': [1, 2, 3, 4],
                     'B': [&quot;Yes&quot;, &quot;No&quot;, &quot;Yes&quot;, &quot;Yes&quot;],
                     'C': [&quot;Yes&quot;, &quot;No&quot;, &quot;No&quot;, &quot;Yes&quot;],
                     'D': [&quot;No&quot;, &quot;Yes&quot;, &quot;No&quot;, &quot;Yes&quot;]})
  
  le = LabelEncoder()
  df_test['A'] = le.fit_transform(df_test['A'])
  le_dict = dict(zip(le.classes_, le.transform(le.classes_)))
  class_name = 'classes_' +'A' + '.npy'
  np.save(class_name, le_dict, allow_pickle=True)
  
  print(df_test)
  print(le.classes_)
  
  le.classes_ = np.load('classes_A.npy', allow_pickle = True)
  
  print(le.classes_)
  
  df_new = pd.DataFrame({'A': [1, 2, 3, 4],
                     'B': [&quot;Yes&quot;, &quot;No&quot;, &quot;Yes&quot;, &quot;Yes&quot;],
                     'C': [&quot;Yes&quot;, &quot;No&quot;, &quot;No&quot;, &quot;Yes&quot;],
                     'D': [&quot;No&quot;, &quot;Yes&quot;, &quot;No&quot;, &quot;Yes&quot;]})
  
  df_new['A'] = le.transform(df_new['A'])
  </code></pre>
  <p>This is giving me the following error:</p>
  <pre><code>
    File &quot;&lt;ipython-input-42-a1aa630ec7e8&gt;&quot;, line 1, in &lt;module&gt;
      df_new['A'] = le.transform(df_new['A'])
  
    File &quot;/Users/usr/opt/anaconda3/envs/signals_gcp_py36/lib/python3.6/site-packages/sklearn/preprocessing/label.py&quot;, line 257, in transform
      _, y = _encode(y, uniques=self.classes_, encode=True)
  
    File &quot;/Users/usr/opt/anaconda3/envs/signals_gcp_py36/lib/python3.6/site-packages/sklearn/preprocessing/label.py&quot;, line 110, in _encode
      return _encode_numpy(values, uniques, encode)
  
    File &quot;/Users/usr/opt/anaconda3/envs/signals_gcp_py36/lib/python3.6/site-packages/sklearn/preprocessing/label.py&quot;, line 49, in _encode_numpy
      % str(diff))
  
  ValueError: y contains previously unseen labels: [1, 2, 3, 4]
  </code></pre>
  <p>When I print le.classes_ before loading it from memory, this is what it looks like:
  <code>array([1, 2, 3, 4])</code></p>
  <p>But when I print it after np.load(), this is what it looks like:
  <code>{1: 0, 2: 1, 3: 2, 4: 3}</code></p>
  <p>Here's some more information on le.classes after <code>np.load()</code>:</p>
  <pre><code>In []: le.classes_
  Out[]: array({1: 0, 2: 1, 3: 2, 4: 3}, dtype=object)
  
  In []: type(le.classes_)
  Out[]: numpy.ndarray
  
  In []: print(le.classes_)
  Out[]: {1: 0, 2: 1, 3: 2, 4: 3}
  </code></pre>
  <p>I'm trying to understand what is happening how these functions work. I ran the same code, but for col.B, and I'm getting yet another error.</p>
  <pre><code>from sklearn.preprocessing import LabelEncoder
  import pickle as cPickle
  import numpy as np
  
  df_test = pd.DataFrame({'A': [1, 2, 3, 4],
                     'B': [&quot;Yes&quot;, &quot;No&quot;, &quot;Yes&quot;, &quot;Yes&quot;],
                     'C': [&quot;Yes&quot;, &quot;No&quot;, &quot;No&quot;, &quot;Yes&quot;],
                     'D': [&quot;No&quot;, &quot;Yes&quot;, &quot;No&quot;, &quot;Yes&quot;]})
  
  le = LabelEncoder()
  df_test['B'] = le.fit_transform(df_test['B'])
  le_dict = dict(zip(le.classes_, le.transform(le.classes_)))
  class_name = 'classes_' +'B' + '.npy'
  
  np.save(class_name, le_dict, allow_pickle=True)
  
  print(df_test)
  print(le.classes_)
  
  le.classes_ = np.load('classes_B.npy', allow_pickle = True)
  
  print(le.classes_)
  
  df_new = pd.DataFrame({'A': [1, 2, 3, 4],
                     'B': [&quot;Yes&quot;, &quot;No&quot;, &quot;Yes&quot;, &quot;Yes&quot;],
                     'C': [&quot;Yes&quot;, &quot;No&quot;, &quot;No&quot;, &quot;Yes&quot;],
                     'D': [&quot;No&quot;, &quot;Yes&quot;, &quot;No&quot;, &quot;Yes&quot;]})
  
  df_new['B'] = le.transform(df_new['B'])
  </code></pre>
  <p>The error with this one is <code>TypeError: argument must be a string or number</code>.</p>
  <p>Here's the full stack:</p>
  <pre><code>Traceback (most recent call last):
  
    File &quot;&lt;ipython-input-71-a0243d411c34&gt;&quot;, line 1, in &lt;module&gt;
      df_new['B'] = le.transform(df_new['B'])
  
    File &quot;/Users/usr/opt/anaconda3/envs/signals_gcp_py36/lib/python3.6/site-packages/sklearn/preprocessing/label.py&quot;, line 257, in transform
      _, y = _encode(y, uniques=self.classes_, encode=True)
  
    File &quot;/Users/usr/opt/anaconda3/envs/signals_gcp_py36/lib/python3.6/site-packages/sklearn/preprocessing/label.py&quot;, line 107, in _encode
      raise TypeError(&quot;argument must be a string or number&quot;)
  
  TypeError: argument must be a string or number
  </code></pre>
  <p>Any help is appreciated!</p>