title: Locally load saved tensorflow model .pb from google cloud machine learning engine
body: <p>I'd like to take the tensorflow model i've trained online and run it locally with a python program I distribute.</p>
  
  <p>After training, I get a directory /model with two files /saved_model.pb and a folder /variables. What is the simplest way to deploy this locally?</p>
  
  <p>I was following <a href="https://codelabs.developers.google.com/codelabs/tensorflow-for-poets/index.html#4" rel="nofollow noreferrer">here</a> for deploying frozen models, but I can't quite read in the .pb. I downloaded saved_model.pb to my working directly and tried</p>
  
  <pre><code>with tf.gfile.GFile("saved_model.pb", "rb") as f:
      graph_def = tf.GraphDef()
      graph_def.ParseFromString(f.read())
  
  google.protobuf.message.DecodeError: Truncated message.
  </code></pre>
  
  <p>Looking on SO <a href="https://stackoverflow.com/questions/35351760/tf-save-restore-graph-fails-at-tf-graphdef-parsefromstring">here</a>, they suggested a different route.</p>
  
  <pre><code>with tf.gfile.GFile("saved_model.pb", "rb") as f:
      proto_b=f.read()
      graph_def = tf.GraphDef()
      text_format.Merge(proto_b, graph_def) 
  
  builtins.TypeError: a bytes-like object is required, not 'str'
  </code></pre>
  
  <p>I find this confusing since</p>
  
  <pre><code>type(proto_b)
  &lt;class 'bytes'&gt;
  type(graph_def)
  &lt;class 'tensorflow.core.framework.graph_pb2.GraphDef'&gt;
  </code></pre>
  
  <p>Why the error, neither are strings?</p>
  
  <p>What's the best way to deploy a cloud trained model? </p>
  
  <p>Full code</p>
  
  <pre><code>import tensorflow as tf
  import sys
  from google.protobuf import text_format
  
  
  # change this as you see fit
  #image_path = sys.argv[1]
  image_path="test.jpg"
  
  # Read in the image_data
  image_data = tf.gfile.FastGFile(image_path, 'rb').read()
  
  # Loads label file, strips off carriage return
  label_lines = [line.rstrip() for line 
                 in tf.gfile.GFile("dict.txt")]
  
  # Unpersists graph from file
  with tf.gfile.GFile("saved_model.pb", "rb") as f:
      proto_b=f.read()
      graph_def = tf.GraphDef()
      text_format.Merge(proto_b, graph_def) 
  
  with tf.Session() as sess:
      # Feed the image_data as input to the graph and get first prediction
      softmax_tensor = sess.graph.get_tensor_by_name('conv1/weights:0')
  
      predictions = sess.run(softmax_tensor, \
                             {'DecodeJpeg/contents:0': image_data})
  
      # Sort to show labels of first prediction in order of confidence
      top_k = predictions[0].argsort()[-len(predictions[0]):][::-1]
  
      for node_id in top_k:
          human_string = label_lines[node_id]
          score = predictions[0][node_id]
          print('%s (score = %.5f)' % (human_string, score))
  </code></pre>