title: How to set up logback MDC in apache beam and dataflow?
body: <p>We are using apache beam and would like to setup the logback MDC.  logback MDC is a great GREAT resource when you have a request come in and you store let's say a userId (in our case, it's custId, fileId, requestId), then anytime a developer logs, it magically stamps that information on to the developers log.  the developer no longer forgets to add it every log statement he adds.</p>
  <p>I am starting in an end to end integration type test with apache beam direct runner embedded in our microservice for testing (in production, the microservice calls dataflow).  currently, I am see that the MDC is good up until after the expand() methods are called.  Once the processElement methods are called, the context is of course gone since I am in another thread.</p>
  <p>So, trying to fix this piece first.  Where should I put this context such that I can restore it at the beginning of this thread.</p>
  <p>As an example, if I have an Executor.execute(runnable), then I simply transfer context using that runnable like so</p>
  <pre><code>    public class MDCContextRunnable implements Runnable {
      private final Map&lt;String, String&gt; mdcSnapshot;
      private Runnable runnable;
  
      public MDCContextRunnable(Runnable runnable) {
          this.runnable = runnable;
          mdcSnapshot = MDC.getCopyOfContextMap();
      }
  
  
      @Override
      public void run() {
          try {
              MDC.setContextMap(mdcSnapshot);
  
              runnable.run();
              
          } Catch {
              //Must log errors before mdc is cleared
              log.error(&quot;message&quot;, e);.  /// Logs error and MDC
          } finally {
              MDC.clear();
          }
  
      }
  }
  </code></pre>
  <p>so I need to do the same with apache beam basically.  I need to</p>
  <ol>
  <li>Have a point to capture the MDC</li>
  <li>Have a point to restore the MDC</li>
  <li>Have a point to clear out the MDC to prevent it leaking to another request(really in case I missed something which seems to happen now and then)</li>
  </ol>
  <p>Any ideas on how to do this?</p>
  <p>oh, bonus points if it the MDC can be there when any exceptions are logged by the framework!!!!  (ie. ideally, frameworks are supposed to do this for you but apache beam seems like it is not doing this.  Most web frameworks have this built in).</p>
  <p>thanks,
  Dean</p>