title: use JavaCC to format my input file; how to handle comments?
body: <p>I'm parsing SQL using the excellen JSQLParser library, which uses JavaCC internally.</p>
  <p>The grammar consists of <code>TOKEN</code>s and <code>SPECIAL_TOKEN</code>s. The latter is used to remove the single and multi-line comments from the token stream before the parser is called, like this:</p>
  <pre><code>SPECIAL_TOKEN:
  {
     &lt; LINE_COMMENT: (&quot;--&quot; | &quot;//&quot;) (~[&quot;\r&quot;,&quot;\n&quot;])*&gt;
  |  &lt; MULTI_LINE_COMMENT: &quot;/*&quot; (~[&quot;*&quot;])* &quot;*&quot; (&quot;*&quot; | (~[&quot;*&quot;,&quot;/&quot;] (~[&quot;*&quot;])* &quot;*&quot;))* &quot;/&quot;&gt;
  }
  </code></pre>
  <p>I can use the AST to find all the SPECIAL_TOKENS by just using <code>.next</code> on the root node and resulting nodes, but then I lose the structure. This gives me just the contents without the parse-context.</p>
  <p>I would like to use the context to implement a code-formatter.</p>
  <p>My example:</p>
  <pre><code>                -- 1. this is 
                  -- 2. an example 
  
                  SELECT * /* All cols */  FROM aap -- two
                     JOIN b ON a.c=d.c
                     WHERE /* inline comment */ true
                  -- an example
                 ;
  </code></pre>
  <p>I want it to be formatted somewhat like this:</p>
  <pre><code>                -- 1. this is 
                  -- 2. an example 
  
                  SELECT 
                      * /* All cols */  
                  FROM 
                      aap -- two
                  JOIN 
                      b ON a.c=d.c
                  WHERE /* inline comment */ true
                  -- an example
                  ;
  </code></pre>
  <p>What is the correct approach using javacc?</p>