title: What does numpy's vectorize do?
body: <p>I have a function for cleaning text of a set stopwords:</p>
  
  <pre><code>def clean_text(raw_text, stopwords_set):
      # removing everything which is not a letter
      letters_only = re.sub("[^a-zA-Z]", " ", raw_text)
      # lower case + split --&gt; list of words
      words = letters_only.lower().split()             
      # now remove the stop words
      meaningful_words = [w for w in words if not w in stopwords_set]
      # join the remaining words together to get the cleaned tweet
      return " ".join(meaningful_words)
  </code></pre>
  
  <p>And a dataset of 1.6 million twitter tweet in a <code>pandas</code> dataframe. If I simply <code>apply</code> this function to the dataframe like this:</p>
  
  <pre><code>dataframe['clean_text'] = dataframe.apply(
      lambda text: clean_text(text, set(stopwords.words('english'))),
      axis = 1)
  </code></pre>
  
  <p>The computation needs 2 minutes to finish (approximately). However, when I use <code>np.vectorize</code> like this:</p>
  
  <pre><code>dataframe['clean_text'] = np.vectorize(clean_text)(
      dataframe['text'], set(stopwords.words('english')))
  </code></pre>
  
  <p>The computation finishes after 10 seconds (approximately).</p>
  
  <p>That in itself would not be surprising, if not both methods only used one core on my machine. I was assuming, that with vectorize, it would automatically use multiple cores to finish faster and this way gaining more speed, but it seems to do something different instead.</p>
  
  <p>What kind of "magic" does <code>numpy</code>'s ´vectorize` do?</p>