title: Keras Tokenizer
body: <p>anyone have idea how exactly does kera.preprocessing.text.Tockenizer work?</p>
  
  <pre><code>word_index = tokenizer.word_index
  print('Found %s unique tokens.' % len(word_index))
  data = pad_sequences(sequences, maxlen=maxlen)
  labels = np.asarray(labels)
  print('Shape of data tensor:', data.shape)
  print('Shape of label tensor:', labels.shape)
  indices = np.arange(data.shape[0])
  np.random.shuffle(indices)
  data = data[indices]
  labels = labels[indices]
  x_train = data[:training_samples]
  y_train = labels[:training_samples]
  x_val = data[training_samples: training_samples + validation_samples]
  y_val = labels[training_samples: training_samples + validation_samples]
  </code></pre>
  
  <blockquote>
    <p>Found 88413 unique tokens. Shape of data tensor: (24984, 100) Shape of
    label tensor: (24984,)</p>
  </blockquote>
  
  <pre><code>tokenizer.texts_to_sequences('You are Amrock!')
  </code></pre>
  
  <blockquote>
    <p>Out[18]:
    [[5128],  [1601],  [1205],  [],  [3],  [1480],  [962],  [],  [3], 
    [1978],  [1480],  [1601],  [1144],  [2292],  []]</p>
  </blockquote>
  
  <p>What does Out[18] mean exactly?</p>