title: tokenize string based on self-defined dictionary
body: <p>I would like to tokenize a list of strings according to my self-defined dictionary.</p>
  
  <p>The list of string looks like this:</p>
  
  <pre><code>lst = ['vitamin c juice', 'organic supplement'] 
  </code></pre>
  
  <p>The self-defined dictionary: </p>
  
  <pre><code>dct = {0: 'organic', 1: 'juice', 2: 'supplement', 3: 'vitamin c'}
  </code></pre>
  
  <p>My expected result:</p>
  
  <p>vitamin c juice --> <code>[(3,1), (1,1)]</code>
  organic supplement --> <code>[(0,1), (2,1)]</code></p>
  
  <p>My current code:</p>
  
  <pre><code>import gensim
  import gensim.corpora as corpora
  from gensim.utils import tokenize
  dct = corpora.Dictionary([list(x) for x in tup_list]) 
  corpus = [dct.doc2bow(text) for text in [s for s in lst]]
  </code></pre>
  
  <p>The error message I got is <code>TypeError: doc2bow expects an array of unicode tokens on input, not a single string</code> However, I do not want to simply tokenize "vitamin c" as <code>vitamin</code> and <code>c</code>. Instead, I want to tokenize based on my existing <code>dct</code> words. That is to say, it should be <code>vitamin c</code>.</p>