OutfitTransformer: Learning Outfit Representations for Fashion
  Recommendation
  Learning an effective outfit-level representation is critical for predicting
the compatibility of items in an outfit, and retrieving complementary items for
a partial outfit. We present a framework, OutfitTransformer, that uses the
proposed task-specific tokens and leverages the self-attention mechanism to
learn effective outfit-level representations encoding the compatibility
relationships between all items in the entire outfit for addressing both
compatibility prediction and complementary item retrieval tasks. For
compatibility prediction, we design an outfit token to capture a global outfit
representation and train the framework using a classification loss. For
complementary item retrieval, we design a target item token that additionally
takes the target item specification (in the form of a category or text
description) into consideration. We train our framework using a proposed
set-wise outfit ranking loss to generate a target item embedding given an
outfit, and a target item specification as inputs. The generated target item
embedding is then used to retrieve compatible items that match the rest of the
outfit. Additionally, we adopt a pre-training approach and a curriculum
learning strategy to improve retrieval performance. Since our framework learns
at an outfit-level, it allows us to learn a single embedding capturing
higher-order relations among multiple items in the outfit more effectively than
pairwise methods. Experiments demonstrate that our approach outperforms
state-of-the-art methods on compatibility prediction, fill-in-the-blank, and
complementary item retrieval tasks. We further validate the quality of our
retrieval results with a user study.
