title: How to *actually* read CSV data in TensorFlow?
body: <p>I'm relatively new to the world of TensorFlow, and pretty perplexed by how you'd <strong><em>actually</em></strong> read CSV data into a usable example/label tensors in TensorFlow. The example from the <a href="https://www.tensorflow.org/versions/r0.7/how_tos/reading_data/index.html#reading-from-files" rel="noreferrer">TensorFlow tutorial on reading CSV data</a> is pretty fragmented and only gets you part of the way to being able to train on CSV data.</p>
  
  <p>Here's my code that I've pieced together, based off that CSV tutorial:</p>
  
  <pre><code>from __future__ import print_function
  import tensorflow as tf
  
  def file_len(fname):
      with open(fname) as f:
          for i, l in enumerate(f):
              pass
      return i + 1
  
  filename = "csv_test_data.csv"
  
  # setup text reader
  file_length = file_len(filename)
  filename_queue = tf.train.string_input_producer([filename])
  reader = tf.TextLineReader(skip_header_lines=1)
  _, csv_row = reader.read(filename_queue)
  
  # setup CSV decoding
  record_defaults = [[0],[0],[0],[0],[0]]
  col1,col2,col3,col4,col5 = tf.decode_csv(csv_row, record_defaults=record_defaults)
  
  # turn features back into a tensor
  features = tf.stack([col1,col2,col3,col4])
  
  print("loading, " + str(file_length) + " line(s)\n")
  with tf.Session() as sess:
    tf.initialize_all_variables().run()
  
    # start populating filename queue
    coord = tf.train.Coordinator()
    threads = tf.train.start_queue_runners(coord=coord)
  
    for i in range(file_length):
      # retrieve a single instance
      example, label = sess.run([features, col5])
      print(example, label)
  
    coord.request_stop()
    coord.join(threads)
    print("\ndone loading")
  </code></pre>
  
  <p>And here is an brief example from the CSV file I'm loading - pretty basic data - 4 feature columns, and 1 label column:</p>
  
  <pre><code>0,0,0,0,0
  0,15,0,0,0
  0,30,0,0,0
  0,45,0,0,0
  </code></pre>
  
  <p>All the code above does is <strong>print each example from the CSV file, one by one</strong>, which, while nice, is pretty darn useless for training.</p>
  
  <p>What I'm struggling with here is how you'd actually turn those individual examples, loaded one-by-one, into a training dataset. For example, <a href="https://github.com/rringham/deep-learning-notebooks/blob/master/notMNIST_four_hidden_layers.ipynb" rel="noreferrer">here's a notebook</a> I was working on in the Udacity Deep Learning course. I basically want to take the CSV data I'm loading, and plop it into something like <strong>train_dataset</strong> and <strong>train_labels</strong>:</p>
  
  <pre><code>def reformat(dataset, labels):
    dataset = dataset.reshape((-1, image_size * image_size)).astype(np.float32)
    # Map 2 to [0.0, 1.0, 0.0 ...], 3 to [0.0, 0.0, 1.0 ...]
    labels = (np.arange(num_labels) == labels[:,None]).astype(np.float32)
    return dataset, labels
  train_dataset, train_labels = reformat(train_dataset, train_labels)
  valid_dataset, valid_labels = reformat(valid_dataset, valid_labels)
  test_dataset, test_labels = reformat(test_dataset, test_labels)
  print('Training set', train_dataset.shape, train_labels.shape)
  print('Validation set', valid_dataset.shape, valid_labels.shape)
  print('Test set', test_dataset.shape, test_labels.shape)
  </code></pre>
  
  <p>I've tried using <code>tf.train.shuffle_batch</code>, like this, but it just inexplicably hangs:</p>
  
  <pre><code>  for i in range(file_length):
      # retrieve a single instance
      example, label = sess.run([features, colRelevant])
      example_batch, label_batch = tf.train.shuffle_batch([example, label], batch_size=file_length, capacity=file_length, min_after_dequeue=10000)
      print(example, label)
  </code></pre>
  
  <p>So to sum up, here are my questions:</p>
  
  <ul>
  <li><strong>What am I missing about this process?</strong>
  
  <ul>
  <li>It feels like there is some key intuition that I'm missing about how to properly build an input pipeline.</li>
  </ul></li>
  <li><strong>Is there a way to avoid having to know the length of the CSV file?</strong>
  
  <ul>
  <li>It feels pretty inelegant to have to know the number of lines you want to process (the <code>for i in range(file_length)</code> line of code above)</li>
  </ul></li>
  </ul>
  
  <hr>
  
  <p><strong>Edit:</strong>
  As soon as Yaroslav pointed out that I was likely mixing up imperative and graph-construction parts here, it started to become clearer. I was able to pull together the following code, which I think is closer to what would typically done when training a model from CSV (excluding any model training code):</p>
  
  <pre><code>from __future__ import print_function
  import numpy as np
  import tensorflow as tf
  import math as math
  import argparse
  
  parser = argparse.ArgumentParser()
  parser.add_argument('dataset')
  args = parser.parse_args()
  
  def file_len(fname):
      with open(fname) as f:
          for i, l in enumerate(f):
              pass
      return i + 1
  
  def read_from_csv(filename_queue):
    reader = tf.TextLineReader(skip_header_lines=1)
    _, csv_row = reader.read(filename_queue)
    record_defaults = [[0],[0],[0],[0],[0]]
    colHour,colQuarter,colAction,colUser,colLabel = tf.decode_csv(csv_row, record_defaults=record_defaults)
    features = tf.stack([colHour,colQuarter,colAction,colUser])  
    label = tf.stack([colLabel])  
    return features, label
  
  def input_pipeline(batch_size, num_epochs=None):
    filename_queue = tf.train.string_input_producer([args.dataset], num_epochs=num_epochs, shuffle=True)  
    example, label = read_from_csv(filename_queue)
    min_after_dequeue = 10000
    capacity = min_after_dequeue + 3 * batch_size
    example_batch, label_batch = tf.train.shuffle_batch(
        [example, label], batch_size=batch_size, capacity=capacity,
        min_after_dequeue=min_after_dequeue)
    return example_batch, label_batch
  
  file_length = file_len(args.dataset) - 1
  examples, labels = input_pipeline(file_length, 1)
  
  with tf.Session() as sess:
    tf.initialize_all_variables().run()
  
    # start populating filename queue
    coord = tf.train.Coordinator()
    threads = tf.train.start_queue_runners(coord=coord)
  
    try:
      while not coord.should_stop():
        example_batch, label_batch = sess.run([examples, labels])
        print(example_batch)
    except tf.errors.OutOfRangeError:
      print('Done training, epoch reached')
    finally:
      coord.request_stop()
  
    coord.join(threads) 
  </code></pre>