title: Selecting data points on one side of separating hyperplane ONLY
body: <p>I implemented a custom undersampler function, that's based on <code>SVC</code>. What the function does is fit an <code>SVC</code> on binary class data, determine the support vector for the majority class, and undersample the majority be choosing data points closer to its support vectors only.</p>
  <p>Code:</p>
  <pre><code>from collections import Counter
  from sklearn.svm import SVC
  import numpy as np 
  from math import dist # python &gt;= 3.8
  
  def svm_resample(X, y):
      # class distribution
      counter = Counter(y)
      maj_class = counter.most_common()[0][0]
      min_class = counter.most_common()[-1][0]
      # number of minority examples
      num_minority = len(X[ y == min_class])
      svc = SVC(kernel='linear', random_state=32)
      svc.fit(X,y)
      # majority class support vectors
      maj_sup_vectors = svc.support_vectors_[maj_class]
      # compute distances to support vectors' point
      distances = []
      for i, x in enumerate(X[y == maj_class]):
        #input(f'sv: {maj_sup_vectors}, x: {x}') # check value passed
        d = dist(maj_sup_vectors, x)
        distances.append((i, d))
      # sort distances (ascending)
      distances.sort(key=lambda tup: tup[1])
      index = [i for i, d in distances][:num_minority]
      X_ds = np.concatenate((X[y == maj_class][index], X[y == min_class]))
      y_ds = np.concatenate((y[y == maj_class][index], y[y == min_class]))
  
      return X_ds, y_ds
  </code></pre>
  <p>So using this function I can undersample the following sample data selecting only such instances of the majority as:</p>
  <pre><code>import matplotlib.pyplot as plt
  from sklearn.datasets import make_blobs
  from sklearn.inspection import DecisionBoundaryDisplay
  
  # create two clusters of random points
  n_samples_1 = 1000
  n_samples_2 = 200
  centers = [[0.0, 0.0], [2.0, 2.0]]
  clusters_std = [1.5, 0.5]
  X, y = make_blobs(n_samples=[n_samples_1, n_samples_2],
      centers=centers, cluster_std=clusters_std,
      random_state=0, shuffle=False,)
  
  # fit the model and get the separating hyperplane
  clf = SVC(kernel=&quot;rbf&quot;, C=1.0)
  clf.fit(X, y)
  
  # my function's resampled data points (chosen)
  Xs, ys, sdist = svm_resample(X, y)
  
  # plot the samples
  plt.scatter(X[:, 0], X[:, 1], c=y, cmap=plt.cm.Paired, edgecolors=&quot;c&quot;)
  plt.scatter(Xs[:, 0], Xs[:, 1], c=ys, cmap=plt.cm.Paired, edgecolors=&quot;b&quot;)
  
  # plot decision function
  ax = plt.gca()
  disp = DecisionBoundaryDisplay.from_estimator(
      clf, X, plot_method=&quot;contour&quot;, colors=&quot;k&quot;,
      levels=[0], alpha=0.5, linestyles=[&quot;-&quot;], ax=ax)
  
  plt.show()
  </code></pre>
  <p>Figure:</p>
  <p><a href="https://i.stack.imgur.com/q0Vpx.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/q0Vpx.png" alt="enter image description here" /></a></p>
  <p>The majority class data points are the blue + cyan points. The blue ones are those selected by the function, and the minority in read.</p>
  <p>There's overlap in data points (looking closely, we see blue data point's with the minority red region).</p>
  <p>Is there a way to modify my function, such that it selects majority data points on the other side of the hyperplane only, igoring those in overlapped region?</p>