A Multimodal Recommender System for Large-scale Assortment Generation in
  E-commerce
  E-commerce platforms surface interesting products largely through product
recommendations that capture users' styles and aesthetic preferences. Curating
recommendations as a complete complementary set, or assortment, is critical for
a successful e-commerce experience, especially for product categories such as
furniture, where items are selected together with the overall theme, style or
ambiance of a space in mind. In this paper, we propose two visually-aware
recommender systems that can automatically curate an assortment of living room
furniture around a couple of pre-selected seed pieces for the room. The first
system aims to maximize the visual-based style compatibility of the entire
selection by making use of transfer learning and topic modeling. The second
system extends the first by incorporating text data and applying polylingual
topic modeling to infer style over both modalities. We review the production
pipeline for surfacing these visually-aware recommender systems and compare
them through offline validations and large-scale online A/B tests on Overstock.
Our experimental results show that complimentary style is best discovered over
product sets when both visual and textual data are incorporated.
