title: Using awk how do i pull out matched strings and other data in one command
body: <p>I'm trying to parse over a number of files in a path for a certain string patterns (for e.g. new File()), which can occur over multiple lines in that file.</p>
  <p>The information I'm trying to return is;</p>
  <p>1 Filename/path<br />
  2  Number of occurrences of string pattern in the
  file<br />
  3 Code found i.e new File()<br />
  4 Line number code found on</p>
  <p>Here is an example file contents of test.txt;</p>
  <pre><code>    new
      File()
      new File()
      new
      
      
      
      File()
      Fil[![1]][1]e() new
      new File() test new File()
  </code></pre>
  <p>Here is a picture of the file in notepad++
  <a href="https://i.stack.imgur.com/xCHV1.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/xCHV1.png" alt="test.txt" /></a></p>
  <p>A more realistic real world example would be (made up code, not compilable);</p>
  <pre><code>package gw.plugin.document.impl
  
  
  @Export
  abstract class BaseLocalDocumentContentSource implements 
  InitializablePlugin
  {
  
    private static var DOCUMENTS_PATH = &quot;documents.path&quot;
  
    public property get DemoDocumentsURL() : URL {
      return new URL(&quot;file&quot;, &quot;&quot;, DemoDocumentsPath)
    }
  
  
    construct() {
    }
  
    protected function buildDocumentsPath(documentRootDir : String, 
    documentTmpDir : String) {
      if (DocumentsPathParameter.HasContent) {
        DemoDocumentsPath = getAbsolutePath(DocumentsPathParameter, 
  documentRootDir)
        if (!new test 
        File(DemoDocumentsPath).equals(new File(DocumentsPathParameter))) {
            Logger.DOCUMENT.warn((typeof this).RelativeName + &quot; has a 
  relative path specified for its documents.path parameter, so it will store 
  documents in the app container's temporary directory. For production use, 
  the configuration should be changed to a full directory path, not a 
  relative path&quot;)
            DocumentsPath = getAbsolutePath(DocumentsPathParameter, documentTmpDir)
        var file = new File(DocumentsPath)
        if (!file.exists() &amp;&amp; file.isDirectory()) {
            file.mkdirs()
        }
    } else {
        DocumentsPath = DemoDocumentsPath
    }
  }
  Logger.DOCUMENT.info(&quot;Documents path: &quot; + DocumentsPath)
    }
  
    protected function updateDocument(strDocUID : String, isDocument : InputStream) {
  try {
      var file = getDocumentFile(strDocUID)
      if (!FileUtil.isFile( file ) || file.isReservedFileName()) {
          throw new IllegalArgumentException(&quot;Document ${strDocUID} does not exist!&quot;)
      }
      var backupFile = new File(file.getPath() + &quot;.bak&quot;)
      if (not file.renameTo(backupFile) ) { // renamed physical file, 'file' still has previous name
        throw new RuntimeException(&quot;Failed to rename file to ${backupFile}&quot;)
          }
      copyToFile(isDocument, file)
      try {
        backupFile.delete()
      }
      catch (e : Throwable) {
        Logger.DOCUMENT.warn(&quot;DocMgmt failed to delete '${backupFile}'&quot;)
      }
  } catch (e : Exception) {
      throw new RuntimeException(&quot;Exception encountered trying to update document with doc UID: ${strDocUID}&quot;, e)
  }
    }
  
  protected function getDocumentFile(relativePath : String, checkDemoFolder : boolean) : File {
  var file = new File(getDocumentsDir(), relativePath)
  if (!file.exists() &amp;&amp; checkDemoFolder) {
      file = new File(getDemoDocumentsDir(), relativePath)
  }
  return file
  }
  
    protected function makeSubDirPath(diw : IDocumentInfoWrapper) : String {
    var subDirPath = diw.getSubDirForDocument()
    var dirDoc = new File(getDocumentsDir() + subDirPath)
    if (not dirDoc.Directory) {
        dirDoc.mkdirs()
    }
    return subDirPath
    }
  
  
   private static function getAbsolutePath(path : String, rootPath : String) : String {
      var retVal = path
      if (path.startsWith(&quot;\\&quot;) || path.startsWith(&quot;/&quot;) || (path.length() &gt; 1 &amp;&amp; path.charAt(1) == &quot;:&quot; as char)) {
      retVal = path
      } else {
      retVal = rootPath + File.separator + path
      }
      try {
      retVal = (new File(retVal)).getCanonicalPath()
      } catch (e : IOException) {
      throw new RuntimeException(&quot;Could not get absolute path from relative path: ${path}&quot;, e)
      }
      return retVal.replaceAll(&quot;\\\\&quot;,&quot;/&quot;)
    }
  
  }
  </code></pre>
  <p>I have looked at grep, pcregrep, sed and awk. The folder I'm searching is very large so I'm trying to return all data required in one command instead of running four commands and having to traverse the folder more than once.</p>
  <p>I've found awk the most applicable but have very limited experience in all of the programs I mentioned and I'm I don't have authorisation to install pcregrep in the env so can't use that.</p>
  <p>Here's my attempt for awk so far, it is wrong and probably poorly done so be gentle :)</p>
  <pre><code>    awk '{
         if(/new[[:space:]]*/) {
           line1=NR;
           code1=$0;
         } if(/File\(\)/) { 
           count[$0]++; 
           line2=NR; 
           if(line1 != line2) {
             code2=$0;
             printf &quot;Found on lines %d, %d, code = %s %s \nNumber of occurrences = %d&quot;, line1, line2, code1, code2, count[$0]
           } else { 
             printf &quot;Found on line %d, code = %s \nNumber of occurrences = %d&quot;, line1, code1, count[$0]
           } 
         }
      }' test.txt 
  </code></pre>
  <p>I know that my count of occurences is incorrect as I'm counting the occurrences per match as opposed to the total in the file. I'm getting some weird output such as the below;</p>
  <pre><code>     File()n lines 1, 2, code = new
       Number of occurrences = 1
       ound on line 3, code = new File()
      Number of occurrences = 1
       File()n lines 4, 8, code = new
       Number of occurrences = 2
       ound on line 9, code = File() new
      Number of occurrences = 1
  </code></pre>
  <p>Where code2 is overwriting the first few words of the print statement and not printed where I'd expect.</p>
  <p>Expected output would be something like;</p>
  <pre><code>    test.txt (Filename) 
      5 (number of occurrences of new File() pattern) 
      new File() Found on lines 1 &amp; 2 
      new File() Found on line 3 
      new File() Found on lines 4 &amp; 9 
      new File() Found on line 10 
      new File() Found on line 10 
  </code></pre>
  <p>Or something similar to this</p>
  <p>Output of cat -vte test.txt is;</p>
  <pre><code>    new^M$
      File()^M$
      new File()^M$
      new ^M$
      ^M$
      ^M$
      ^M$
      File()^M$
      File() new^M$
      new File() test new File()
  </code></pre>
  <p>Any help would be appreciated.</p>