title: How to index input words that has more than one word per line in Python
body: <p>The purpose of this code is to read through the sentences in <code>test.txt</code> and check if each word in <code>test_oov.txt</code> has an example sentence is <code>test.txt</code></p>
  <p>an example input file would be</p>
  <pre><code>joe biden is the us president
  john mccain was a congressman
  jpmorgan chase is the way to go
  this is an irrelevant sentence
  kanye west is an artist
  </code></pre>
  <p>and test_oov.txt would be</p>
  <pre><code>joe biden
  john mccain
  johns hopkins university
  johnson johnson
  jpmorgan chase
  justice department
  kamala harris
  kanye west
  </code></pre>
  <p>and its supposed to output</p>
  <pre><code>joe biden : joe biden is the us president
  john mccain : john mccain was a congressman
  etc.
  </code></pre>
  <p>However, it does not work when the <code>test_oov.txt</code> input has more than one word per line. It works good when the word is just one word. However, when it's words like, <code>Joe Biden, or John mccain etc,</code> it doesn't work.</p>
  <p>Code is below</p>
  <pre><code>from nltk.tokenize import sent_tokenize, word_tokenize
  
  # 1. create a reusable word index like {'worda': [2, 4, 10], 'wordb': [1, 9]}
  with open(&quot;test.txt&quot;, &quot;r&quot;, encoding=&quot;utf8&quot;) as fp:
      news_sentences = sent_tokenize(fp.read())
  
  index = {}
  for i, sentence in enumerate(news_sentences):
      for word in word_tokenize(sentence):
          word = word.lower()
          if word not in index:
              index[word] = []
          index[word].append(i)
  
  # 2. look up words from that index and retrieve the associated sentences
  with open(&quot;test_oov.txt&quot;, &quot;r&quot;, encoding=&quot;utf8&quot;) as fp:
      oov_ner_data = [l.strip() for l in fp.readlines()]
  
  matches = {}
  
  for word in oov_ner_data:
      word = word.lower()
      if word in index:
          matches[word] = [news_sentences[i] for i in index[word]]
  
  for (key, value) in matches.items():
      print(key + &quot; : &quot; + str(value))
  </code></pre>
  <p>How do I fix this?</p>