Reenvisioning Collaborative Filtering vs Matrix Factorization
  Collaborative filtering models based on matrix factorization and learned
similarities using Artificial Neural Networks (ANNs) have gained significant
attention in recent years. This is, in part, because ANNs have demonstrated
good results in a wide variety of recommendation tasks. The introduction of
ANNs within the recommendation ecosystem has been recently questioned, raising
several comparisons in terms of efficiency and effectiveness. One aspect most
of these comparisons have in common is their focus on accuracy, neglecting
other evaluation dimensions important for the recommendation, such as novelty,
diversity, or accounting for biases. We replicate experiments from three papers
that compare Neural Collaborative Filtering (NCF) and Matrix Factorization
(MF), to extend the analysis to other evaluation dimensions. Our contribution
shows that the experiments are entirely reproducible, and we extend the study
including other accuracy metrics and two statistical hypothesis tests. We
investigated the Diversity and Novelty of the recommendations, showing that MF
provides a better accuracy also on the long tail, although NCF provides a
better item coverage and more diversified recommendations. We discuss the bias
effect generated by the tested methods. They show a relatively small bias, but
other recommendation baselines, with competitive accuracy performance,
consistently show to be less affected by this issue. This is the first work, to
the best of our knowledge, where several evaluation dimensions have been
explored for an array of SOTA algorithms covering recent adaptations of ANNs
and MF. Hence, we show the potential these techniques may have on
beyond-accuracy evaluation while analyzing the effect on reproducibility these
complementary dimensions may spark. Available at
github.com/sisinflab/Reenvisioning-the-comparison-between-Neural-Collaborative-Filtering-and-Matrix-Factorization
