Incremental personalized E-mail spam filter using novel TFDCR feature
  selection with dynamic feature update
  Communication through e-mails remains to be highly formalized, conventional
and indispensable method for the exchange of information over the Internet. An
ever-increasing ratio and adversary nature of spam e-mails have posed a great
many challenges such as uneven class distribution, unequal error cost, frequent
change of content and personalized context-sensitive discrimination. In this
research, we propose a novel and distinctive approach to develop an incremental
personalized e-mail spam filter. The proposed work is described using three
significant contributions. First, we applied a novel term frequency difference
and category ratio based feature selection function TFDCR to select the most
discriminating features irrespective of the number of samples in each class.
Second, an incremental learning model is used which enables the classifier to
update the discriminant function dynamically. Third, a heuristic function
called selectionRankWeight is introduced to upgrade the existing feature set
that determines new features carrying strong discriminating ability from an
incoming set of e-mails. Three public e-mail datasets possessing different
characteristics are used to evaluate the filter performance. Experiments are
conducted to compare the feature selection efficiency of TFDCR and to observe
the filter performance under both the batch and the incremental learning mode.
The results demonstrate the superiority of TFDCR as the most effective f eature
selection function. The incremental learning model incorporating dynamic
feature update function overcomes the problem of drifting concepts. The
proposed filter validates its efficiency and feasibility by substantially
improving the classification accuracy and reducing the false positive error of
misclassifying legitimate e-mail as spam.
