Learning Colour Representations of Search Queries
  Image search engines rely on appropriately designed ranking features that
capture various aspects of the content semantics as well as the historic
popularity. In this work, we consider the role of colour in this relevance
matching process. Our work is motivated by the observation that a significant
fraction of user queries have an inherent colour associated with them. While
some queries contain explicit colour mentions (such as 'black car' and 'yellow
daisies'), other queries have implicit notions of colour (such as 'sky' and
'grass'). Furthermore, grounding queries in colour is not a mapping to a single
colour, but a distribution in colour space. For instance, a search for 'trees'
tends to have a bimodal distribution around the colours green and brown. We
leverage historical clickthrough data to produce a colour representation for
search queries and propose a recurrent neural network architecture to encode
unseen queries into colour space. We also show how this embedding can be learnt
alongside a cross-modal relevance ranker from impression logs where a subset of
the result images were clicked. We demonstrate that the use of a query-image
colour distance feature leads to an improvement in the ranker performance as
measured by users' preferences of clicked versus skipped images.
