Predicting Human Similarity Judgments Using Large Language Models
  Similarity judgments provide a well-established method for accessing mental
representations, with applications in psychology, neuroscience and machine
learning. However, collecting similarity judgments can be prohibitively
expensive for naturalistic datasets as the number of comparisons grows
quadratically in the number of stimuli. One way to tackle this problem is to
construct approximation procedures that rely on more accessible proxies for
predicting similarity. Here we leverage recent advances in language models and
online recruitment, proposing an efficient domain-general procedure for
predicting human similarity judgments based on text descriptions. Intuitively,
similar stimuli are likely to evoke similar descriptions, allowing us to use
description similarity to predict pairwise similarity judgments. Crucially, the
number of descriptions required grows only linearly with the number of stimuli,
drastically reducing the amount of data required. We test this procedure on six
datasets of naturalistic images and show that our models outperform previous
approaches based on visual information.
