title: How to check if a file is a CSV, XML, or JSON without relying on the file extension?
body: <p>I have a program that accepts a combination of multiple input files, and also can take directories to scan for those input files.</p>
  <p>Normally, I only accept files with valid file extensions for CSV, XML, and JSON - but as I've learned over the years, you can't really trust the user to give the correct files.</p>
  <p>For example, if a user has a file with an XML extension, but the actual contents are JSON, I would want to warn the user and treat the file as a JSON file.
  Another situation is the on Unix-based operating systems, file extensions aren't used for identifying file types like Windows does, which can open up more situations of incorrect file type compared to the file contents.</p>
  <p>Right now, my program collects the list of files, filter them into only the ones with acceptable file extensions, and then I create a <code>concurrent.futures.ThreadPoolExecutor</code> to be able to check all the files simultaneously. The actual check looks like so:</p>
  <pre class="lang-py prettyprint-override"><code>with open(the_file, &quot;r&quot;) as f:
    check = f.read(1)
    if check == &quot;{&quot; or check == &quot;[&quot;:
      file_type = &quot;json&quot;
    elif check == &quot;&lt;&quot;:
      file_type = &quot;xml&quot;
    else:
      file_type = &quot;csv&quot;
  </code></pre>
  <p>There are two major issues with this:</p>
  <ol>
  <li>If the file is supposed to be a CSV, and the first character for it is either <code>[</code> or <code>{</code> or <code>&lt;</code>, then the program will assign it the wrong file type.</li>
  <li>Conversely, if the file is not any of the acceptable types, and it does not start with either <code>[</code> or <code>{</code> or <code>&lt;</code>, then the program will just assume that the file is supposed to be a CSV.</li>
  </ol>
  <p>I have the following idea to remedy this:</p>
  <pre class="lang-py prettyprint-override"><code>import csv
  import json
  import defusedxml.ElementTree as xml
  
  try:
    loaded = xml.parse(the_file)
    return &quot;XML&quot;
  except xml.ParseError:
    del loaded
    try:
      loaded = json.load(the_file)
      return &quot;JSON&quot;
    except json.JSONDecodeError:
      del loaded
      try:
        with open(the_file, &quot;r&quot;) as csv_file:
          loaded = csv.DictReader(csv_file)
        return &quot;CSV&quot;
      except csv.Error:
        print(&quot;The file is not in any acceptable format&quot;)
  </code></pre>
  <p>The problem with this is that since my program tries to run this check on multiple files simultaneously, the memory usage can get dangerously high with so many files opened at the same time. The other minor issue is the I/O cost of opening and closing so many files at once just to check for the file type.</p>
  <p>IS there some more efficient alternative for what I'm trying to do?</p>