title: Fuzzy Vlookup on Pandas
body: <p>I have two dataframes with around 300,000 rows and roughly 11 columns each.</p>
  <p>The first dataframe contains raw data, with names that are not homogeneous strings. The other has official data (and homogeneous strings). First of all, I tried running a fuzzy matching between the 'raw data' and the official data via a code, which seems to be working well for a dataframe with roughly 200 entries. When I try running the code on my &gt;300,000 rows, the kernel is slower (it's been running for over an hour now)... Any ideas on how I could do speed up this process? I was thinking of splitting the dataframes into smaller ones, but I would need the entire dataset to make sure the matching is correct.</p>
  <p>Thanks!</p>
  <p>P.S. Current Code:</p>
  <pre><code>def fuzzy_merge(df_1, df_2, key1, key2, threshold=90, limit=2):
      &quot;&quot;&quot;
      :param df_1: the left table to join
      :param df_2: the right table to join
      :param key1: key column of the left table
      :param key2: key column of the right table
      :param threshold: how close the matches should be to return a match, based on Levenshtein distance
      :param limit: the amount of matches that will get returned, these are sorted high to low
      :return: dataframe with boths keys and matches
      &quot;&quot;&quot;
      s = df_2[key2].tolist()
  
      m = df_1[key1].apply(lambda x: process.extract(x, s, limit=limit))    
      df_1['Example1'] = m
  
      m2 = df_1['CompanyName'].apply(lambda x: ', '.join([i[0] for i in x if i[1] &gt;= threshold]))
      df_1['Example2'] = m2
  
      return df_1
  
  from fuzzywuzzy import fuzz
  from fuzzywuzzy import process
  
  merged_df = fuzzy_merge(df1, df2, 'Example1', 'Example2', threshold=90)
  print(merged_df)```
  </code></pre>