title: Understanding how a TensorFlow model as a class and a TensorFlow Session interact
body: <p>I have been using TensorFlow for a reasonable length of time now. and believed I had a thorough understanding of how a TensorFlow graph works and executes within a session. However, I have written all of my TensorFlow models in a script-like fashion as such:</p>
  
  <pre><code>import tensorflow as tf
  import DataWorker
  import Constants
  
  x = tf.placeholder(tf.float32, [None, Constants.sequenceLength, DataWorker.numFeatures])
  y = tf.placeholder(tf.float32, [None, 1])
  xTensors = tf.unstack(x, axis=1)   # [seqLength tensors of shape (batchSize, numFeatures)]
  
  W = tf.Variable(tf.random_normal([Constants.numHidden, 1]))     # Weighted matrix
  b = tf.Variable(tf.random_normal([1]))                          # Bias
  
  cell = tf.contrib.rnn.BasicLSTMCell(Constants.numHidden, forget_bias=Constants.forgetBias)
  outputs, finalState = tf.nn.static_rnn(cell, xTensors, dtype=tf.float32)
  # predictions = [tf.add(tf.matmul(output, W), b) for output in outputs]             # List of predictions after each time step
  prediction = tf.add(tf.matmul(outputs[-1], W), b)                                   # Prediction after final time step
  prediction = tf.tanh(prediction)                                                    # Activation
  mse = tf.losses.mean_squared_error(predictions=prediction, labels=y)                # Mean loss over entire batch
  accuracy = tf.reduce_mean(1 - (tf.abs(y - prediction) / DataWorker.labelRange))     # Accuracy over entire batch
  optimiser = tf.train.AdamOptimizer(Constants.learningRate).minimize(mse)            # Backpropagation
  
  with tf.Session() as session:
      session.run(tf.global_variables_initializer())
      # #############################################
      # TRAINING
      # #############################################
      for epoch in range(Constants.numEpochs):
          print("***** EPOCH:", epoch + 1, "*****\n")
          IDPointer, TSPointer = 0, 0         # Pointers to current ID and timestamp
          epochComplete = False
          batchNum = 0
          while not epochComplete:
              batchNum += 1
              batchX, batchY, IDPointer, TSPointer, epochComplete = DataWorker.generateBatch(IDPointer, TSPointer, isTraining=True)
              dict = {x: batchX, y: batchY}
              session.run(optimiser, dict)
              if batchNum % 1000 == 0 or epochComplete:
                  batchLoss = session.run(mse, dict)
                  batchAccuracy = session.run(accuracy, dict)
                  print("Iteration:", batchNum)
                  print(batchLoss)
                  print(str("%.2f" % (batchAccuracy * 100) + "%\n"))
  
      # #############################################
      # TESTING
      # #############################################
      testX, testY, _, _, _ = DataWorker.generateBatch(0, 0, isTraining=False)
      testAccuracy = session.run(accuracy, {x: testX, y: testY})
      print("Testing Accuracy:", str("%.2f" % (testAccuracy * 100) + "%"))
  </code></pre>
  
  <p>But now, for practicality and readability, I want to implement my model as a class, but have encountered many problems with initializing my variables, etc.</p>
  
  <p>This is the closest I have got to implementing the above example using my own <code>LSTM</code> class</p>
  
  <p><strong>Model.py</strong></p>
  
  <pre><code>import tensorflow as tf
  import Constants
  import DataWorker       # Remove this dependency
  
  
  class LSTM():
      """docstring."""
  
      def __init__(self,
                   inputDimensionList,
                   outputDimensionList,
                   numLayers=Constants.numLayers,
                   numHidden=Constants.numHidden,
                   learningRate=Constants.learningRate,
                   forgetBias=Constants.forgetBias
                   ):
          """docstring."""
          self.batchInputs = tf.placeholder(tf.float32, [None] + inputDimensionList)
          self.batchLabels = tf.placeholder(tf.float32, [None] + outputDimensionList)
          self.weightedMatrix = tf.Variable(tf.random_normal([numHidden] + outputDimensionList))
          self.biasMatrix = tf.Variable(tf.random_normal(outputDimensionList))
          self.cell = tf.contrib.rnn.BasicLSTMCell(numHidden, forget_bias=forgetBias)
          self.numLayers = numLayers
          self.numHidden = numHidden
          self.learningRate = learningRate
          self.forgetBias = forgetBias
          self.batchDict = {}
          self.batchInputTensors = None
          self.batchOutputs = None    # All needed as instance variables?
          self.batchFinalStates = None
          self.batchPredictions = None
          self.batchLoss = None
          self.batchAccuracy = None
          self.initialised = False
          self.session = tf.Session()
          # Take in activation, loss and optimiser FUNCTIONS as args
  
      def execute(self, command):
          """docstring."""
          return self.session.run(command, self.batchDict)
  
      def setBatchDict(self, inputs, labels):
          """docstring."""
          self.batchDict = {self.batchInputs: inputs, self.batchLabels: labels}
          self.batchInputTensors = tf.unstack(self.batchInputs, axis=1)
  
      def processBatch(self):
          """docstring."""
          self.batchOutputs, self.batchFinalState = tf.nn.static_rnn(self.cell, self.batchInputTensors, dtype=tf.float32)
          pred = tf.tanh(tf.add(tf.matmul(self.batchOutputs[-1], self.weightedMatrix), self.biasMatrix))
          mse = tf.losses.mean_squared_error(predictions=pred, labels=self.batchLabels)
          optimiser = tf.train.AdamOptimizer(self.learningRate).minimize(mse)
  
          if not self.initialised:
              self.session.run(tf.global_variables_initializer())
              self.initialised = True
  
          with tf.variable_scope("model") as scope:
              if self.initialised:
                  scope.reuse_variables()
              self.execute(optimiser)
              self.batchPredictions = self.execute(pred)
              self.batchLoss = self.execute(tf.losses.mean_squared_error(predictions=self.batchPredictions, labels=self.batchLabels))
              self.batchAccuracy = self.execute(tf.reduce_mean(1 - (tf.abs(self.batchLabels - self.batchPredictions) / DataWorker.labelRange)))
              return self.batchPredictions, self.batchLabels, self.batchLoss, self.batchAccuracy
  
      def kill(self):
          """docstring."""
          self.session.close()
  </code></pre>
  
  <p>This class is quite messy, especially <code>processBatch()</code> as I have just been trying to get it to work before refining it.</p>
  
  <p>I then run my model here:</p>
  
  <p><strong>Main.py</strong></p>
  
  <pre><code>import DataWorker
  import Constants
  from Model import LSTM
  
  inputDim = [Constants.sequenceLength, DataWorker.numFeatures]
  outputDim = [1]
  
  lstm = LSTM(inputDimensionList=inputDim, outputDimensionList=outputDim)
  
  # #############################################
  # TRAINING
  # #############################################
  for epoch in range(Constants.numEpochs):
      print("***** EPOCH:", epoch + 1, "*****\n")
      IDPointer, TSPointer = 0, 0         # Pointers to current ID and timestamp
      epochComplete = False
      batchNum = 0
      while not epochComplete:
          batchNum += 1
          batchX, batchY, IDPointer, TSPointer, epochComplete = DataWorker.generateBatch(IDPointer, TSPointer, isTraining=True)
          lstm.setBatchDict(batchX, batchY)
          batchPredictions, batchLabels, batchLoss, batchAccuracy = lstm.runBatch()
          if batchNum % 1000 == 0 or epochComplete:
              print("Iteration:", batchNum)
              print("Pred:", batchPredictions[-1], "\tLabel:", batchLabels[-1])
              print("Loss:", batchLoss)
              print("Accuracy:", str("%.2f" % (batchAccuracy * 100) + "%\n"))
  
  # #############################################
  # TESTING
  # #############################################
  testX, testY, _, _, _ = DataWorker.generateBatch(0, 0, isTraining=False)
  lstm.setBatchDict(testX, testY)
  _, _, _, testAccuracy = lstm.runBatch()
  print("Testing Accuracy:", str("%.2f" % (testAccuracy * 100) + "%"))
  
  lstm.kill()
  </code></pre>
  
  <p>A single passthrough of the graph is executed fine, when all the variables are initialized, but it is on the second iteration where I get the error</p>
  
  <pre><code>ValueError: Variable rnn/basic_lstm_cell/kernel/Adam/ already exists, disallowed. Did you mean to set reuse=True in VarScope? Originally defined at:
  optimiser = tf.train.AdamOptimizer(self.learningRate).minimize(mse)
  </code></pre>
  
  <p>I Googled this problem and learned that using <code>scope.reuse_variables()</code> should stop it trying to initialize the AdamOptimizer a second time, but cleary this isn't working how I have implemented it. How can I fix this issue?</p>
  
  <p>As a side note, is my method of creating the TensorFlow session as an instance variable within my <code>LSTM</code> class acceptable, or should I create the session in <code>Main</code> and then pass it into the <code>LSTM</code> instance?</p>