title: How to deal with more than one categorical feature in a decision tree?
body: <p>I read a piece of code about binary decision tree from a book. It has only one categorical feature, which is field(3), in the raw data, and is converted to one-of-k(one-hot encoding). </p>
  
  <pre><code>def PrepareData(sc: SparkContext): (RDD[LabeledPoint], RDD[LabeledPoint], RDD[LabeledPoint], Map[String, Int]) = {
  
    val rawDataWithHeader = sc.textFile("data/train.tsv")
    val rawData = rawDataWithHeader.mapPartitionsWithIndex { (idx, iter) =&gt; if (idx == 0) iter.drop(1) else iter }
    val lines = rawData.map(_.split("\t"))
  
  
    val categoriesMap = lines.map(fields =&gt; fields(3)).distinct.collect.zipWithIndex.toMap
    val labelpointRDD = lines.map { fields =&gt;
       val trFields = fields.map(_.replaceAll("\"", ""))
       val categoryFeaturesArray = Array.ofDim[Double](categoriesMap.size)
       val categoryIdx = categoriesMap(fields(3))
       categoryFeaturesArray(categoryIdx) = 1
       val numericalFeatures = trFields.slice(4, fields.size - 1).map(d =&gt; if (d == "?") 0.0 else d.toDouble)
       val label = trFields(fields.size - 1).toInt
       LabeledPoint(label, Vectors.dense(categoryFeaturesArray ++ numericalFeatures))
    }
  
    val Array(trainData, validationData, testData) = labelpointRDD.randomSplit(Array(8, 1, 1))
    return (trainData, validationData, testData, categoriesMap)
  }
  </code></pre>
  
  <p>I wonder how to revise the code if there are several categorical features in the raw data, let's say field(3), field(5), field(7) are all categorical features.</p>
  
  <p>I revised the first line:</p>
  
  <pre><code>def PrepareData(sc: SparkContext): (RDD[LabeledPoint], RDD[LabeledPoint], RDD[LabeledPoint], Map[String, Int], Map[String, Int], Map[String, Int], Map[String, Int]) =......
  </code></pre>
  
  <p>Then, I converted another two fields into 1-of-k encoding as it was done like:</p>
  
  <pre><code>val categoriesMap5 = lines.map(fields =&gt; fields(5)).distinct.collect.zipWithIndex.toMap
  val categoriesMap7 = lines.map(fields =&gt; fields(7)).distinct.collect.zipWithIndex.toMap
  val categoryFeaturesArray5 = Array.ofDim[Double](categoriesMap5.size)
  val categoryFeaturesArray7 = Array.ofDim[Double](categoriesMap7.size)
  val categoryIdx3 = categoriesMap5(fields(5))
  val categoryIdx5 = categoriesMap7(fields(7))
  categoryFeaturesArray5(categoryIdx5) = 1
  categoryFeaturesArray7(categoryIdx7) = 1
  </code></pre>
  
  <p>Finally, I revised LabeledPoint and return like:</p>
  
  <pre><code>LabeledPoint(label, Vectors.dense(categoryFeaturesArray ++ categoryFeaturesArray5 ++ categoryFeaturesArray7 ++ numericalFeatures))
  return (trainData, validationData, testData, categoriesMap, categoriesMap5, categoriesMap7)
  </code></pre>
  
  <p>Is it correct?</p>
  
  <p>==================================================</p>
  
  <p>The second problem I encountered is: the following code from that book, in the trainModel, it uses </p>
  
  <pre><code>  DecisionTree.trainRegressor(trainingData, categoricalFeaturesInfo, impurity, maxDepth, maxBins)
  </code></pre>
  
  <p>Here is the code:</p>
  
  <pre><code>def trainModel(trainData: RDD[LabeledPoint], impurity: String, maxDepth: Int, maxBins: Int): (DecisionTreeModel, Double) = {
     val startTime = new DateTime()
     val model = DecisionTree.trainClassifier(trainData, 2, Map[Int, Int](), impurity, maxDepth, maxBins)
     val endTime = new DateTime()
     val duration = new Duration(startTime, endTime)
     (model, duration.getMillis())
  }
  </code></pre>
  
  <p>The question is: how do I pass the categoricalFeaturesInfo into this method if it has three categorical features mentioned previously?</p>
  
  <p>I just want to follow the step on the book to build up a prediction system on my own by using a decision tree. To be more specific, the data sets I chose has several categorical features like :
  Gender: male, female</p>
  
  <p>Education: HS-grad, Bachelors, Master, PH.D, ......</p>
  
  <p>Country: US, Canada, England, Australia, ......</p>
  
  <p>But I don't know how to merge them into one single <code>categoryFeatures ++ numericalFeatures</code> to put into <code>Vector.dense()</code>, and one single  <code>categoricalFeaturesInfo</code> to put into <code>DecisionTree.trainRegressor()</code></p>