title: designing a 10 threaded process
body: <p>I have some 500 odd data-files lying in a directory. I am processing these files using a command . Problem is it picks up each serially , of course  as would any utility 
  For the moment what my process does is spawns some 500 processes , staggers them in batches of 10 each interspersed by a wait . Just like this . 1 file == 1 process per this design.</p>
  
  <pre><code>command1 &lt;parameters&gt; &gt; log 2&amp;&gt;1&amp;
  command2 &lt;parameters&gt; &gt; log 2&amp;&gt;1&amp;
  command3 &lt;parameters&gt; &gt; log 2&amp;&gt;1&amp;
  command4 &lt;parameters&gt; &gt; log 2&amp;&gt;1&amp;  /# ....till 10
  wait 
  command11 &lt;parameters&gt; &gt; log 2&amp;&gt;1&amp;
  command1 &lt;parameters&gt; &gt; log 2&amp;&gt;1&amp;
  command1 &lt;parameters&gt; &gt; log 2&amp;&gt;1&amp; /# ...till 20 
  wait
  command1 &lt;parameters&gt; &gt; log 2&amp;&gt;1&amp; 
  </code></pre>
  
  <p>So I dont pound the system away at the same time. The command is actually another series of shells and c code that parses the data files and does some checks, which involve connecting to Oracle. 
  I'd like to re-design this so that every-time a command runs I don't open a new DB connection. So lets say there are 100 files . I want to open just 10 DB connections instead of 100 separate connections.
  The way I want to do it is first count how many files I have. Divide them by 10. Each division is a 'batch' of 10 files-so instead of 100 there are  150  files, the process should automatically create 15 directories . Each directories should move in it 10 files and then 
  I can run </p>
  
  <pre><code> command1 &lt;parameters&gt; Directorypath  &gt; log 2&amp;&gt;1&amp; 
  /# command can pick up all files in a directory or a single file
  /# ct till command 10 ...
  ....
   command10 &lt;parameters&gt; Directorypath  &gt; log 2&amp;&gt;1&amp; 
  </code></pre>
  
  <p>10 files== 1 process by this design.</p>
  
  <p>Any suggestions visa a vis commands about how this can be done.
  ty </p>