title: How does distances weighting work in KNN?
body: <p>I'm writing KNN classifier in R. I want to add weighting scheme, e. g. inverted indices 1/d.  As it is, for Iris dataset I get almost perfect 66% accuracy (no matter the metric used) since value no. 3 ("virginica") almost never shows up and I want to make it better with weighting. My question is: what exactly and how do I weight? I've read that I should weight classes of K nearest neighbours with those distances.</p>
  
  <p>I've tried creating vectors of classes and distances to K nearest neighbours and then taking weighted mean from it:</p>
  
  <pre><code>inverted &lt;- function(vals, distances)
  {
    inv_distances &lt;- 1 / distances
    # eliminate division-by-zero errors
    inv_distances &lt;- ifelse((inv_distances &lt; 0.01), 0.01, inv_distances)
  
    weighted.mean(vals, inv_distances)
  }
  </code></pre>
  
  <p>My results are weird: for correct vectors vals (classes) and distances I sometimes get NaN (Not a Number) or NA values. Also my weights don't sum to 1, and... they probably should? I'm not sure. I just need someone to clear this weighting scheme for me.</p>
  
  <p><strong>EDIT:</strong>
  I've debugged above code, since it multiplied by weight too late (therefore not eliminating distance 0 and causing NaNs). I've also changed it to harmonic series weights, not using distance (so first neighbour has weight 1, second 1/2, third 1/3 etc.). I still don't know exactly how it works and what other weights may be.</p>
  
  <pre><code>inverted &lt;- function(vals)
  {
    weights &lt;- 1 / seq(length(vals))
  
    res &lt;- weighted.mean(vals, weights)
    res
  }
  </code></pre>