{'fraction_non_alphanumeric': 0.02243211334120425, 'fraction_numerical': 0.002066115702479339, 'mean_word_length': 4.8633217993079585, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
A Text Classifier Model for Categorizing Feed Contents Consumed by a Web Aggregator

This paper presents a method of using a Text Classifier to automatically categorize the content of web feeds consumed by a web aggregator. The pre-defined category of the feed to be consumed by the aggregator does not always match the content being consumed and categorizing the content using the pre-defined category of the feed curtails user experience as users would not see all the contents belonging to their category of interest. A web aggregator was developed and this was integrated with the SVM classifier to automatically categorize feed content being consumed. The experimental results showed that the text classifier performs well in categorizing the content of feed being consumed and it also affirmed the disparity in the pre-defined category of the source feed and appropriate category of the consumed content. Keywords—feed; aggregator; text classifier; svm I. INTRODUCTION Web feeds provide a way for websites especially those that are frequently updated to provide up to date information to their users. Feeds are provided in either RSS or Atom format. Users who are interested in consuming the content of feeds use an aggregator software called feed reader. Aggregator software can either be a windows or a web application and it collects feed contents from various sources in one view. With a feed reader, a user can have the latest content of his/her favourite website in one place; thereby reducing time spent checking different websites. A spin-off of feed readers is web aggregation sites. A web aggregation site is a website that has content from various feeds in one place. This makes it easier for users to view contents from various websites at once. It also removes the overhead of having to build the content of a feed aggregator by the user. Popular aggregation websites include newsnow.com, kicknews.com. When aggregators have to categorize the content consumed from feeds, they either use a predefined category that has been registered for the source of the feed or try to get the category from the meta-data supplied with the feed content. Using the predefined category of the source brings up scenario in which the category does not match the actual content being consumed. In some cases also, the category supplied in the meta-data would not match any of the categories set up in the aggregator. The categorization of content from feeds can be achieved via the use of Text Classifiers. Text Classifiers are algorithms that are used to carry out Text Categorization (TC). In formal terms, taking a document di from a set of documents D and categories {c1, c2, c3}, text categorization is assigning a category ci to document di (11). Example of text categorization algorithms include; K Nearest Neighbour (KNN), Naive Bayes (NB), Support Vector Machines (SVM). In TC, documents may be classified in such a way that it can only belong to one category (single-label categorization) or can belong to multiple categories (multi-label categorization) (15). Multi-label categorization is better suited to aggregators because the content consumed from a feed can belong to multiple categories. Example, a story about a Nigerian footballer getting married to a Nollywood (Nigerian movie industry) actress can rightly belong to categories about sports, gossip and entertainment.