title: Airflow, how to pass variables from BashOperator task to another
body: <p>I'm trying to figure out the most generic/effective way to pass variables from one BashOperator task to another. I came up with a solution, where I push the output of the first BashOperator task to xcom. This string is then pulled by a PythonOperator task that parses the string to a key-value pairs that are then pushed to xcom. Finally these k-v pairs can be pulled and used by the second BashOperator task.</p>
  
  <p>I would like to have some comments from the more experienced airflow users, is this approach overly complicated or is it ok?</p>
  
  <pre><code>from airflow import DAG
  from airflow.operators.bash_operator import BashOperator
  from airflow.operators.python_operator import PythonOperator
  import datetime as dt 
  from airflow.utils.dates import days_ago    
  
  default_args = {
      'owner': 'airflow',
      'depends_on_past': False,
      'start_date': dt.datetime(2019,10,14,10,0),
      'email': ['airflow@example.com'],
      'email_on_failure': False,
      'email_on_retry': False,
      'retries': 1,
      'retry_delay': dt.timedelta(minutes=1),
  }
  
  dag = DAG('communication-between-tasks-v1',
        catchup=False,
        default_args=default_args,
        schedule_interval='*/5 * * * *')
  
  def parse_function(**context):
      ti = context['ti']
      msg = ti.xcom_pull('start_task')
      parsed_output_parameters = dict([x.split('=') for x in msg.split()])
  
      # Option1: Dictionary parsed from the string is pushed to xcom
      ti.xcom_push(key='parameters', value=parsed_output_parameters)
  
      # Option2: We can push every key value separately
      for k, v in parsed_output_parameters.items():
          ti.xcom_push(key=k, value=v)
  
  
  start_task = BashOperator(
      task_id='start_task',
      bash_command='echo "FILE1=file1.h5 FILE2=file2.txt VARIABLE=400"',
      xcom_push=True,
      dag=dag)
  
  parse = PythonOperator(
      task_id='parse',
      python_callable=parse_function,
      provide_context=True,
      dag=dag
  )
  
  end_task = BashOperator(
      task_id='end_task',
      bash_command='echo start_task params: {{ ti.xcom_pull(task_ids="parse", 
      key="parameters")["FILE1"] }} \
      {{ ti.xcom_pull(task_ids="parse", key="FILE2") }}',
      dag=dag
  )
  
  start_task &gt;&gt; parse &gt;&gt; end_task
  </code></pre>