title: Sklearn.neighbors Classifier - Order of results in case of a tie
body: <p>I am having an ordering problem with the Sklearn.neighbors classifier. Namely, what I want is in case of a tie the classifier to predict the first result appearing on the dataframe.</p>
  
  <p>The documentation of Sklearn.neighbors says:</p>
  
  <blockquote>
    <p>''Warning Regarding the Nearest Neighbors algorithms, if two
    neighbors, neighbor k+1 and k, have identical distances but different
    labels, the results will depend on the ordering of the training
    data.''</p>
  </blockquote>
  
  <p>However I found out that this is true only in case I use as an algorithm ''brute force'' i.e:  </p>
  
  <pre><code>clf = NearestNeighbors(n_neighbors=2, algorithm='brute').fit(X)
  </code></pre>
  
  <p>My problem is that even by using brute as an algorithm, the order gets lost after a specific number of predicting neighbors. Namely:</p>
  
  <pre><code>clf.kneighbors(np.array(y), n_neighbors=1) -&gt; Predicts 0 row which is correct in terms of ordering
  clf.kneighbors(np.array(y), n_neighbors=2) -&gt; Predicts 0 and 101 rows which are correct in terms of ordering
  clf.kneighbors(np.array(y), n_neighbors=3) -&gt; Predicts 0, 101 and 115 rows which are correct in terms of ordering
  </code></pre>
  
  <p>But then when I use n_neighbors=4 I get:</p>
  
  <pre><code>clf.kneighbors(np.array(y), n_neighbors=4) -&gt; Predicts 4416, 223, 7626 and 2047 rows which are not correct in terms of ordering
  </code></pre>
  
  <p>And it keeps going like this for any number>3. The order is completely lost.</p>
  
  <p>Anyone that has dealt with this problem before?</p>