title: Weka how to predict new unseen Instance using Java Code?
body: <p>I wrote a WEKA java code to train 4 classifiers. I saved the classifiers models and want to use them to predict new unseen instances (think about it as someone who wants to test whether a tweet is positive or negative). </p>
  
  <p>I used <em>StringToWordsVector</em> filter on the training data. And to avoid the "<em>Src and Dest differ in # of attributes</em>" error I used the following code to train the filter using the trained data before applying the filter on the new instance to try and predict whether a new instance is positive or negative. And I just can't get it right.</p>
  
  <pre><code>Classifier cls = (Classifier) weka.core.SerializationHelper.read("models/myModel.model"); //reading one of the trained classifiers
  
  
      BufferedReader datafile = readDataFile("Tweets/tone1.ARFF"); //read training data
  
      Instances data = new Instances(datafile);
      data.setClassIndex(data.numAttributes() - 1);
  
      Filter filter = new StringToWordVector(50);//keep 50 words
      filter.setInputFormat(data);
      Instances filteredData = Filter.useFilter(data, filter);
  
      // rebuild classifier
      cls.buildClassifier(filteredData);
  
  
      String testInstance= "Text that I want to use as an unseen instance and predict whether it's positive or negative";
      System.out.println("&gt;create test instance"); 
      FastVector attributes = new FastVector(2); 
      attributes.addElement(new Attribute("text", (FastVector) null)); 
  
  
      // Add class attribute. 
      FastVector classValues = new FastVector(2); 
      classValues.addElement("Negative"); 
      classValues.addElement("Positive"); 
  
      attributes.addElement(new Attribute("Tone", classValues)); 
      // Create dataset with initial capacity of 100, and set index of class. 
      Instances tests = new Instances("test istance", attributes, 100); 
      tests.setClassIndex(tests.numAttributes() - 1); 
  
      Instance test = new Instance(2); 
      // Set value for message attribute 
      Attribute messageAtt = tests.attribute("text"); 
      test.setValue(messageAtt, messageAtt.addStringValue(testInstance)); 
  
      test.setDataset(tests); 
  
      Filter filter2 = new StringToWordVector(50);
      filter2.setInputFormat(tests);
      Instances filteredTests = Filter.useFilter(tests, filter2);
  
      System.out.println("&gt;train Test filter using training data"); 
      Standardize sfilter = new Standardize(); //Match the number of attributes between src and dest.
      sfilter.setInputFormat(filteredData);  // initializing the filter with training set 
      filteredTests = Filter.useFilter(filteredData, sfilter);    // create new test set
  
  ArffSaver saver = new ArffSaver(); //save test data to ARFF file
  saver.setInstances(filteredTests); 
          File unseenFile = new File ("Tweets/unseen.ARFF");
          saver.setFile(unseenFile); 
          saver.writeBatch();    
  </code></pre>
  
  <p>When I try to Standardize the Input data using the filtered training data I get a new ARFF file (unseen.ARFF) but with 2000 (same number of training data) instances where most of the values are negative. I don't understand why or how to remove those instances.</p>
  
  <pre><code>    System.out.println("&gt;Evaluation"); //without the following 2 lines I get ArrayIndexOutOfBoundException.
      filteredData.setClassIndex(filteredData.numAttributes() - 1);
      filteredTests.setClassIndex(filteredTests.numAttributes() - 1);
  
      Evaluation eval = new Evaluation(filteredData); 
      eval.evaluateModel(cls, filteredTests); 
      System.out.println(eval.toSummaryString("\nResults\n======\n", false)); 
  </code></pre>
  
  <p>Printing the evaluation results I want to see for example a percentage of how positive or negative this instance is but instead I get the following. I also want to see 1 instance instead of 2000. Any help on how to do this will be great.</p>
  
  <pre><code>&gt; Results
  ======
  
  Correlation coefficient                  0.0285
  Mean absolute error                      0.8765
  Root mean squared error                  1.2185
  Relative absolute error                409.4123 %
  Root relative squared error            121.8754 %
  Total Number of Instances             2000     
  </code></pre>
  
  <p>Thanks</p>