title: Apache Beam ETL dimension table loading , any example?
body: <p>I am thinking of Loading File into one Dimension table. My solution is:</p>
  
  <ol>
  <li>Beam.read the file</li>
  <li>Create the side input from the DB about existing data.</li>
  <li>in a ParDo: filter the records which are already in the side input</li>
  <li>biquerySink into DB.</li>
  </ol>
  
  <p>and want to inquire if someone has implement this ? and can you give me some example for this ?
  Thanks</p>
  
  <pre><code>can you give me some example about coGroupByKey. I understand that it may look like below : Sorry,I am newbie to Dataflow,and watching codes is the best way to me 
  
  step 1: sourcedata = beam.ReadFromText(...)
  step 2: existing_table = beam.pvalue.AsDict(p
                                      | beam.Read(beam.BigQuerySource(my_query)
                                      | beam.Map(format_rows)
  
  I assume the structure of sourcedata and existing data is the same :&lt;k,v&gt;                       
  step 3:  source_existing_Data=  {sourcedata,existing_table}
                                  |'coGroupBy' &gt;&gt; beam.coGroupByKey()
  
  
  step4:  new_Data = source_existing_Data | beam.filter(lamada (name,(existing,source)):source is NONE))
  
  step 5:  bigQuerySink(new_Data)
  </code></pre>