title: Binary One Hot Encoding For Dimension Reduction
body: <p><strong>Introduction:</strong></p>
  
  <p>I've been studying machine learning this last couple of months, more specifically neural networks. I found out that categorical data, such as cities, could be transformed to integer data, but shouldn't because there's no linearity. What I mean by this is: there is no reason why New York should be 1 or 100. A better solution to this is one hot encoding. However, it greatly increases the dimensionality of the problem (Picture a network input with the number of cities as input nodes).</p>
  
  <p><strong>Example:</strong></p>
  
  <p>New York = [1, 0, 0]</p>
  
  <p>Buenos Aires = [0, 1, 0]</p>
  
  <p>Tokio = [0, 0, 1]</p>
  
  <p>You need N inputs, N being the number of cities.</p>
  
  <p><strong>Possible Solution:</strong></p>
  
  <p>I don't really know if this already exists or if it could work, I haven't tested it yet. That being said, in the example above, I'm using <em>decimal</em> inputs. Could you use <strong>binary inputs</strong>, in order to reduce the dimensionality of the problem?</p>
  
  <p><strong>Example using binary inputs</strong></p>
  
  <p>New York = [0, 1]</p>
  
  <p>Buenos Aires = [1, 0]</p>
  
  <p>Tokio = [1, 1]</p>
  
  <p>In this way, you only need ≈ <strong>log(N) (base 2)</strong> number of inputs which greatly increases, specially if there are a great number of features. For example: if you have 1000 categorical data inputs, it would only result in 10 inputs for a neural network.</p>
  
  <p>Thank you in advance. Remember I'm only learning.</p>