title: Solr: how to specify a schema during JSON and CSV import?
body: <p>I'm new to Solr and I'm trying to test its functionalities. I come from RDBMS world and was wondering how Solr would perform with my data.</p>
  
  <p>I created a new core:</p>
  
  <p><code>$ bin/solr create -c test</code></p>
  
  <p>and successfully loaded a JSON file using:</p>
  
  <p><code>$ bin/post -c test file.json</code></p>
  
  <p>The first record of <code>file.json</code> looks like this:</p>
  
  <p><code>{"attr":"01234"}</code></p>
  
  <p>but Solr stores it as:</p>
  
  <p><code>{"attr":1234}</code></p>
  
  <p>I began defining a Data Import Handler following <a href="https://www.youtube.com/watch?v=2VkFQTqrRYo" rel="nofollow noreferrer">this tutorial (Youtube video)</a> in order to correctly store my data, and found that JSON can't be processed by DIH. I'm stuck at the definition of <code>data-config.xml</code> because the tutorial treats XML files using the <code>XPathEntityProcessor</code> but can't find a JSON or even a CSV processor (I can easily retrieve a CSV version of <code>file.json</code>, so loading a CSV or a JSON is the same for me). The official documentation is a bit of a mess and doesn't provide many useful examples. The solely processors that probably treat JSON and CSV documents are <code>LineEntityProcessor</code> and <code>PlainTextEntityProcessor</code> ( <a href="https://cwiki.apache.org/confluence/display/solr/Uploading+Structured+Data+Store+Data+with+the+Data+Import+Handler#UploadingStructuredDataStoreDatawiththeDataImportHandler-EntityProcessors" rel="nofollow noreferrer">Official Documentation</a>).</p>
  
  <p><a href="https://wiki.apache.org/solr/DataImportHandler#SolrEntityProcessor" rel="nofollow noreferrer">This other link</a> from the Solr Wiki states:</p>
  
  <blockquote>
    <p><strong>Goals</strong></p>
    
    <blockquote>
      <p>...</p>
      
      <p>Make it possible to plugin any kind of datasource (ftp,scp etc) and any other format of user choice (<strong>JSON</strong>,<strong>csv</strong> etc) </p>
    </blockquote>
  </blockquote>
  
  <p>so I guess it is really possible, but HOW?</p>
  
  <p>I found a <a href="https://stackoverflow.com/questions/34798194/solr-how-to-import-json-files-using-dataconfig">similar question</a> posted in 2014 that no one answered here, so was wondering if in 2016, with the newer versions of Solar, there is a well known solution to this problem.</p>
  
  <p>So the question is: how to import JSON and CSV documents using a specific data schema?</p>
  
  <h2><strong>UPDATE</strong></h2>
  
  <p>Executing <code>http://localhost:8983/solr/test/dihupdate?command=full-import</code> doesn't trigger any error but doesn't load any document. Here are the various xml files located in the core directory:</p>
  
  <p><strong><code>solrconfig.xml</code></strong></p>
  
  <pre><code>...
  &lt;schemaFactory class="ClassicIndexSchemaFactory" /&gt;
  ...
  &lt;requestHandler name="/dihupdate" class="org.apache.solr.handler.dataimport.DataImportHandler" startup="lazy"&gt;
    &lt;lst name="defaults"&gt;
      &lt;str name="config"&gt;data-config.xml&lt;/str&gt;
    &lt;/lst&gt;
  &lt;/requestHandler&gt;
  ...
  </code></pre>
  
  <p><strong><code>schema.xml</code></strong></p>
  
  <pre><code>...
  &lt;field name="id" type="long" indexed="true" stored="true" required="true" multiValued="false" /&gt;
  &lt;field name="attr1" type="string" indexed="true" stored="true" required="true" multiValued="true" /&gt;
  &lt;field name="date" type="date" indexed="true" stored="false" multiValued="true" /&gt;
  &lt;field name="attr2" type="string" indexed="true" stored="true"  multiValued="true" /&gt;
  &lt;field name="attr3" type="string" indexed="true" stored="true" multiValued="true" /&gt;
  &lt;field name="attr4" type="int" indexed="false" stored="true" multiValued="true" /&gt;
  &lt;uniqueKey&gt;id&lt;/uniqueKey&gt;
  ...
  </code></pre>
  
  <p><strong><code>data-config.xml</code></strong></p>
  
  <pre><code>&lt;dataConfig&gt;
      &lt;dataSource type="FileDataSource" /&gt;
      &lt;document&gt;
          &lt;entity name="f" processor="FileListEntityProcessor"
                  fileName="test.json"
                  rootEntity="false"
                  dataSource="null"
                  recursive="true"
                  baseDir="/path/to/data/"/&gt;
      &lt;/document&gt;
  &lt;/dataConfig&gt;
  </code></pre>