Attacks against Ranking Algorithms with Text Embeddings: a Case Study on
  Recruitment Algorithms
  Recently, some studies have shown that text classification tasks are
vulnerable to poisoning and evasion attacks. However, little work has
investigated attacks against decision making algorithms that use text
embeddings, and their output is a ranking. In this paper, we focus on ranking
algorithms for recruitment process, that employ text embeddings for ranking
applicants resumes when compared to a job description. We demonstrate both
white box and black box attacks that identify text items, that based on their
location in embedding space, have significant contribution in increasing the
similarity score between a resume and a job description. The adversary then
uses these text items to improve the ranking of their resume among others. We
tested recruitment algorithms that use the similarity scores obtained from
Universal Sentence Encoder (USE) and Term Frequency Inverse Document Frequency
(TF IDF) vectors. Our results show that in both adversarial settings, on
average the attacker is successful. We also found that attacks against TF IDF
is more successful compared to USE.
