title: How to perform (modified) t-test for multiple variables and multiple models
body: <p>I have created and analyzed around 16 machine learning models using WEKA. Right now, I have a CSV file which shows the models' metrics (such as percent_correct, F-measure, recall, precision, etc.). I am trying to conduct a (modified) student's t-test on these models. I am able to conduct one (according to THIS link) where I compare only ONE variable common to only TWO models. I want to perform a (or multiple) t-tests with MULTIPLE variables and MULTIPLE models at once.</p>
  
  <p>As mentioned, I can only perform the test with one variable (let's say F-measure) among two models (let's say decision table and neural net).</p>
  
  <p>Here's the code for that. I am performing a Kolmogorov-Smirnov test (modified t):</p>
  
  <pre><code>from matplotlib import pyplot
  from pandas import read_csv, DataFrame
  from scipy.stats import ks_2samp
  
  results = DataFrame()
  results['A'] = read_csv('LMT (f-measure).csv', header=None).values[:, 0]
  results['B'] = read_csv('LWL (f-measure).csv', header=None).values[:, 0]
  print(results.describe())
  results.boxplot()
  pyplot.show()
  results.hist()
  pyplot.show()
  
  value, pvalue = ks_2samp(results['A'], results['B'])
  alpha = 0.05
  print(value, pvalue)
  if pvalue &gt; alpha:
      print('Samples are likely drawn from the same distributions (fail to reject H0)')
  else:
      print('Samples are likely drawn from different distributions (reject H0)')
  </code></pre>
  
  <p>Any ideas?</p>