title: How to use sklearn LDA coef_ for feature selection
body: <p>I have a set of features Train = [a1, a2, ..., aN], and Train_label = [y1, y2, ..., yN]. I have 3 classes. I applied LDA </p>
  
  <pre><code>clf = LDA(store_covariance = True)
  clf.fit(Train, Train_label) 
  </code></pre>
  
  <p>and tried to get the importance of these features by </p>
  
  <pre><code>clf.coef_. 
  </code></pre>
  
  <p>The result is a 3XN array. The range of returned values are from around negative 800 to around positive 800. The plot shows the weights of one classclf.coef_[1,:]</p>
  
  <p><a href="https://i.stack.imgur.com/wz9G3.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/wz9G3.png" alt="weights of one classclf.coef_[1,:]"></a>
  What do the negative values mean?</p>
  
  <p>I select the features for each class with the first 3 largest coef_ value to do classification (cross validation and training and test) again. </p>
  
  <pre><code>selected_features = [a27, a22, a5, a23, a10, a42]  
  </code></pre>
  
  <p>where, a27, a22, a5 are selected 3 features with largest coef for the first class, a27, a23, a10 for the second class, and a27, a10, a42 for the third class.</p>
  
  <p>But the classification accuracy decreased a lot! The results are even worse than a few features I selected before.I tried with 6 selected features with largest coef_ value and results were also bad. </p>
  
  <p>Can any one tell me where is wrong and explain to me what is the correct way to do feature selection using LDA? </p>