title: How to Merge files using For each activity in Azure Data Factory
body: <p>I am using ADF to copy the files from a file server to Azure Blob storage. The files in the directory have the same structure without headers and I need to merge them to a single file in the Blob storage.</p>
  <p><a href="https://i.stack.imgur.com/zbSlz.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/zbSlz.png" alt="enter image description here" /></a></p>
  <p>I created a ADF pipeline which uses get metadata to fetch the childItems and For each activity to loop through the files one by one</p>
  <p><a href="https://i.stack.imgur.com/NzDuD.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/NzDuD.png" alt="enter image description here" /></a></p>
  <p>Inside the For each activity there is a copy data activity where I use the file name from the get metadata activity</p>
  <p><a href="https://i.stack.imgur.com/avjJo.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/avjJo.png" alt="enter image description here" /></a></p>
  <p>In the sink setting, I use Merge files as the copy behaviour</p>
  <p><a href="https://i.stack.imgur.com/N21WO.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/N21WO.png" alt="enter image description here" /></a></p>
  <p>When I execute the pipeline, the copy activity gets executed for 3 times and the file in the blob storage gets overwritten with the last file. How do I merge all the 3 files ?</p>
  <p>I know we can use a wildcard pattern to select files. Suppose I have 3 files to begin with, when I run the get metadata activity and by the time the control comes to copy job activity and by this time if there is an addition of 4th file in the folder then with the wildcard pattern I will process all the 4 files and get metadata activity gives me the file names of the 3 files which I will use for archiving which is not right</p>
  <p>Any help is appreciated</p>