{'fraction_non_alphanumeric': 0.02233009708737864, 'fraction_numerical': 0.002912621359223301, 'mean_word_length': 4.922413793103448, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Filtering Methods on a Very Sparse Reddit Recommendation Dataset

The research question in this thesis concerns how accurately it is possible to estimate users’ future votes based on their voting history when votes can only be for or against (upvotes or downvotes). This estimation is done on a large and very sparse dataset of over 23 million votes gathered from the website Reddit, which is a very popular social news and entertainment website where users may vote content up or down. Since over 84% of all the votes are upvotes, downvote estimation is made even more difficult. Similar to the Netflix Prize, collaborative filtering (CF) methods can be used to approach this problem. The two main approaches in CF are neighborhood models and latent factor models. Models using both approaches are implemented and tested in this thesis. Neighborhood approach is implemented by using a k-nearest-neighbor (k-NN) classifier with 3 different types of feature vectors and latent factor models are represented by the classical Singular Value Decomposition (SVD) and more advanced Variational Bayesian Principal Component Analysis (VBPCA). While both SVD and VBPCA are implemented with regularization, VBPCA also uses Bayesian inference methods by adding the noise term into the model and introducing prior distributions over the model parameters. For the experiments, the full Reddit dataset is preprocessed into 3 different sized datasets (k-cores), where the biggest dataset represents the full dataset after removing some noise and outliers and the smallest dataset represents the core of the data, i.e., the most active users and the most voted links. The middle dataset has some properties from both the small and the big dataset. All models are tested for each dataset and the results are measured using different metrics, most notably RMSE. VBPCA and linkbased k-NN model are shown to perform best on all datasets. While VBPCA is better in terms of downvote estimation for the small and the middle datasets, link-based k-NN performs best on the big dataset.