title: Removing SQL Comments with Regular Expressions in R, while Preserving Special Tags
body: <p>I need to remove comments beginning with <code>--</code> from strings of SQL statements in R. I am trying to do this with with regular expressions and <code>gsub</code> (though I am open to other suggestions). The complication is that the strings can contain special tags that begin with <code>---&gt;&gt;&gt;</code> and end with <code>&lt;&lt;&lt;---</code>, which I need to preserve for further processing.</p>
  
  <p>Using lookaheads and lookbehinds, I have made some progress for those comments/tags that start at the beginning of the line:</p>
  
  <pre><code>&gt; re &lt;- "^(?!=&lt;&lt;&lt;-){1}--(?!-&gt;&gt;&gt;){1}.*$"
  &gt;
  &gt; gsub(re, "", "-- test", perl=TRUE)      # should be ""
  [1] ""
  &gt; gsub(re, "", "---&gt;&gt;&gt; test", perl=TRUE)   # should be "---&gt;&gt;&gt; test"
  [1] "---&gt;&gt;&gt; test"
  &gt; gsub(re, "", "&lt;&lt;&lt;--- test", perl=TRUE)   # should be "&lt;&lt;&lt;--- test"
  [1] "&lt;&lt;&lt;--- test"
  &gt; gsub(re, "", "---&gt;&gt;&gt;-&gt;&gt;&gt; test", perl=TRUE) # should be ---&gt;&gt;&gt;-&gt;&gt;&gt; test
  [1] "---&gt;&gt;&gt;-&gt;&gt;&gt; test"
  &gt; gsub(re, "", "----&gt;&gt;&gt; test", perl=TRUE)    # should be ""
  [1] ""
  &gt; gsub(re, "", "test ---&gt;&gt;&gt; test", perl=TRUE) # should be "test ---&gt;&gt;&gt; test"
  [1] "test ---&gt;&gt;&gt; test"
  &gt; gsub(re, "", "test ---&gt;&gt;&gt; test &lt;&lt;&lt;---", perl=TRUE) # should be "test ---&gt;&gt;&gt; test &lt;&lt;&lt;---"
  [1] "test ---&gt;&gt;&gt; test &lt;&lt;&lt;---"
  </code></pre>
  
  <p>But this doesn't, obviously, work with comments elsewhere in the string:</p>
  
  <pre><code>&gt; gsub(re, "", "test1 -- test", perl=TRUE) # should be "test1"
  [1] "test1 -- test"  # WRONG
  </code></pre>
  
  <p>Removing the <code>^</code> at the beginning of the regular expression breaks most of test cases:</p>
  
  <pre><code>&gt; re &lt;- "(?!=&lt;&lt;&lt;-){1}--(?!-&gt;&gt;&gt;){1}.*$"
  &gt; gsub(re, "", "-- test", perl=TRUE)      # should be ""
  [1] ""
  &gt; gsub(re, "", "test1 -- test", perl=TRUE) # should be "test1"
  [1] "test1 "
  &gt; gsub(re, "", "---&gt;&gt;&gt; test", perl=TRUE)   # should be "---&gt;&gt;&gt; test"
  [1] "-"  # WRONG
  &gt; gsub(re, "", "&lt;&lt;&lt;--- test", perl=TRUE)   # should be "&lt;&lt;&lt;--- test"
  [1] "&lt;&lt;&lt;"  # WRONG
  &gt; gsub(re, "", "---&gt;&gt;&gt;-&gt;&gt;&gt; test", perl=TRUE) # should be ---&gt;&gt;&gt;-&gt;&gt;&gt; test
  [1] "-"  # WRONG
  &gt; gsub(re, "", "----&gt;&gt;&gt; test", perl=TRUE)    # should be ""
  [1] ""
  &gt; gsub(re, "", "test ---&gt;&gt;&gt; test", perl=TRUE) # should be "test ---&gt;&gt;&gt; test"
  [1] "test -"  # WRONG
  &gt; gsub(re, "", "test ---&gt;&gt;&gt; test &lt;&lt;&lt;---", perl=TRUE) # should be "test ---&gt;&gt;&gt; test &lt;&lt;&lt;---"
  [1] "test -"  # WRONG
  </code></pre>
  
  <p>Anyone have suggestions how I can do this? I am open to any suggestions, but am constrained to R and have to keep the special tags, <code>---&gt;&gt;&gt;</code> and <code>&lt;&lt;&lt;---</code>.</p>
  
  <p><strong>Edit</strong></p>
  
  <p>As mentioned in the comments, this is also a test case:</p>
  
  <pre><code>&gt; gsub(re, "", "--&gt;&gt;&gt; test", perl=TRUE)    # should be ""
  </code></pre>