title: shell script or grunt tool to delete lines in a file, following a rule
body: <p>Ok, so i have a huge file that results in the concatenation of a lot of files.</p>
  
  <p>Not that it matters, but it's a compilation of 3 google-closure dependency files, and a lot of unit testing files.  </p>
  
  <p>My problem is that each unit test file is requiring some closure modules and because they are now all in the same file the same dependencies are being required more than once when i run it with phantomjs.<br>
  What i need is a script that goes through the file, and deletes the lines (after the first) that are repeated and start with <code>goog.require</code>(.</p>
  
  <p>So, the rules are:</p>
  
  <ul>
  <li>starts with <code>goog.require(</code></li>
  <li>the exact same line (ignoring trailing and preceding spaces) exists somewhere before</li>
  </ul>
  
  <p>Example:</p>
  
  <pre><code>goog.provide( "app.tests.mvc.PMVCUnitTestBootstrap" );
  
  goog.require( "app.tests.mvc.jsunit" );
  goog.require( "app.Bootstrap" );
  goog.require( "pmvc.patterns.observer.Observer" );
  goog.require( "app.tests.mvc.ApplicationResources" );
  goog.require( "app.notifications.SystemNotes" );
  goog.require( "lib.puremvc.definitions.notifications.NotificationMapper" );
  goog.require( "lib.puremvc.definitions.notifications.NotificationMap" );
  ...
  ...
  ...
  goog.require( "app.tests.mvc.jsunit" ); //this line is repeated
  goog.require( "app.Bootstrap" ); //this line is repeated
  goog.require( "lib.unittests.helpers.ExpectedFailures" );
  goog.require( "lib.filesystem.ConstructableFile" );
  goog.require( "lib.filesystem.FileReference" );
  goog.require( "lib.utils.ArrayUtil" );
  goog.require( "easy.util.GUIDUtil" );
  ...
  ...
  </code></pre>
  
  <p>would become</p>
  
  <pre><code> goog.provide( "app.tests.mvc.PMVCUnitTestBootstrap" );
  
   goog.require( "app.tests.mvc.jsunit" );
   goog.require( "app.Bootstrap" );
   goog.require( "pmvc.patterns.observer.Observer" );
   goog.require( "app.tests.mvc.ApplicationResources" );
   goog.require( "app.notifications.SystemNotes" );
   goog.require( "lib.puremvc.definitions.notifications.NotificationMapper" );
   goog.require( "lib.puremvc.definitions.notifications.NotificationMap" );
   ...
   ...
   ...
   goog.require( "lib.unittests.helpers.ExpectedFailures" );
   goog.require( "lib.filesystem.ConstructableFile" );
   goog.require( "lib.filesystem.FileReference" );
   goog.require( "lib.utils.ArrayUtil" );
   goog.require( "easy.util.GUIDUtil" );
   ...
   ...
  </code></pre>
  
  <p>Any help will be welcome</p>