Unifying Decision Trees Split Criteria Using Tsallis Entropy
  The construction of efficient and effective decision trees remains a key
topic in machine learning because of their simplicity and flexibility. A lot of
heuristic algorithms have been proposed to construct near-optimal decision
trees. ID3, C4.5 and CART are classical decision tree algorithms and the split
criteria they used are Shannon entropy, Gain Ratio and Gini index respectively.
All the split criteria seem to be independent, actually, they can be unified in
a Tsallis entropy framework. Tsallis entropy is a generalization of Shannon
entropy and provides a new approach to enhance decision trees' performance with
an adjustable parameter $q$. In this paper, a Tsallis Entropy Criterion (TEC)
algorithm is proposed to unify Shannon entropy, Gain Ratio and Gini index,
which generalizes the split criteria of decision trees. More importantly, we
reveal the relations between Tsallis entropy with different $q$ and other split
criteria. Experimental results on UCI data sets indicate that the TEC algorithm
achieves statistically significant improvement over the classical algorithms.
