title: How can I create a TF-IDF for Text Classification using Spark?
body: <p>I have a CSV file with the following format :</p>
  
  <pre><code>product_id1,product_title1
  product_id2,product_title2
  product_id3,product_title3
  product_id4,product_title4
  product_id5,product_title5
  [...]
  </code></pre>
  
  <p>The product_idX is a integer and the product_titleX is a String, example :</p>
  
  <pre><code>453478692, Apple iPhone 4 8Go
  </code></pre>
  
  <p>I'm trying to create the TF-IDF from my file so I can use it for a Naive Bayes Classifier in MLlib.</p>
  
  <p>I am using Spark for Scala so far and using the tutorials I have found on the official page and the Berkley AmpCamp <a href="http://ampcamp.berkeley.edu/3/exercises/index.html">3</a> and <a href="http://ampcamp.berkeley.edu/big-data-mini-course/movie-recommendation-with-mllib.html">4</a>.</p>
  
  <p>So I'm reading the file :</p>
  
  <pre><code>val file = sc.textFile("offers.csv")
  </code></pre>
  
  <p>Then I'm mapping it in tuples <code>RDD[Array[String]]</code></p>
  
  <pre><code>val tuples = file.map(line =&gt; line.split(",")).cache
  </code></pre>
  
  <p>and after I'm transforming the tuples into pairs <code>RDD[(Int, String)]</code></p>
  
  <pre><code>val pairs = tuples.(line =&gt; (line(0),line(1)))
  </code></pre>
  
  <p>But I'm stuck here and I don't know how to create the Vector from it to turn it into TFIDF.</p>
  
  <p>Thanks</p>