title: CountVectorizer, using same vocabulary in a second time
body: <p>Here my dataset:</p>
  
  <pre><code>anger,happy food food
  anger,dog food food
  disgust,food happy food
  disgust,food dog food
  neutral,food food happy
  neutral,food food dog
  </code></pre>
  
  <p>Second, this is my code, where I perform a bag of words using CountVectorizer class. </p>
  
  <pre><code>classValues =  {'anger':    '0',
              'disgust':  '1',
              'fear':     '2',
              'happiness':'3',
              'sadness':  '4',
              'surprise': '5',
              'neutral':  '6'}
  
  def getClass(line):
      parts = line.split(',')
      return float(classValues[parts[0]])
  
  def getTags(line):
      parts = line.split(',')
      return parts[1].split(" ")
  
  conf= SparkConf()
  conf.setAppName("NaiveBaye")
  conf.set('spark.driver.memory','6g')
  conf.set('spark.executor.memory','6g')
  conf.set('spark.cores.max',156)
  
  sc = SparkContext(conf= conf)
  sqlContext = SQLContext(sc)
  
  data = sc.textFile('dataset.txt')
  
  classes = data.map(getClass).collect()
  tags = data.map(getTags).collect()
  
  d = {
      'tags' : tags,
      'classes' : classes
  }
  
  df = sqlContext.createDataFrame(pd.DataFrame(data=d))
  cv = CountVectorizer(inputCol="tags", outputCol="vectors")
  model = cv.fit(df)
  model.transform(df).show(truncate=False)
  
  vocabulary =  sorted(map(str, model.vocabulary))
  print vocabulary
  </code></pre>
  
  <p>As you can see here: <code>model.transform(df).show(truncate=False)</code> and <code>print vocabulary</code> works perfectly.</p>
  
  <pre><code>+-------+-------------------+-------------------+
  |classes|tags               |vectors            |
  +-------+-------------------+-------------------+
  |0.0    |[happy, food, food]|(3,[0,1],[2.0,1.0])|
  |0.0    |[dog, food, food]  |(3,[0,2],[2.0,1.0])|
  |1.0    |[food, happy, food]|(3,[0,1],[2.0,1.0])|
  |1.0    |[food, dog, food]  |(3,[0,2],[2.0,1.0])|
  |6.0    |[food, food, happy]|(3,[0,1],[2.0,1.0])|
  |6.0    |[food, food, dog]  |(3,[0,2],[2.0,1.0])|
  +-------+-------------------+-------------------+
  ['dog', 'food', 'happy']
  </code></pre>
  
  <p>Now, if I want in a second time, perform a vectorizer of a new element using the same vocabulary how can I do this in python? </p>
  
  <p>For example </p>
  
  <pre><code>anger, happy dog food
  </code></pre>
  
  <p>will be</p>
  
  <pre><code>|0.0    |[happy, dog, food]|(3,[0,1, 2],[1.0,1.0,1.0])|
  </code></pre>
  
  <p>I've read in the documentation that exist CountVectorizerModel that permit load an exist vocabulary. But there's not any documantation about this. </p>
  
  <p>This is very important to me because if I need to classify a new element I need the same order of the vectors, in order to use the same model of my classificator. </p>
  
  <p>I've tried somethings like this: </p>
  
  <pre><code>CountVectorizerModel(vocabulary)
  </code></pre>
  
  <p>but doesn't work.</p>
  
  <h2>Edit 1</h2>
  
  <p>I'm currently using <strong>Spark 1.6.1</strong></p>