title: Tensorflow-gpu Error: InvalidArgumentError: Cannot assign a device for operation 'MatMul'
body: <p>I'm testing tensorflow with GPU in anaconda using the test code given on the tensorflow website: </p>
  
  <pre><code>import tensorflow as tf
  with tf.device('/device:GPU:0'):
    a = tf.constant([1,2,3,4,5,6],shape=[2,3],name='a')
    b = tf.constant([1,2,3,4,5,6],shape=[3,2],name='b')
    c = tf.matmul(a,b)
  sess = tf.Session(config=tf.ConfigProto(log_device_placement=True))
  print(sess.run(c))
  </code></pre>
  
  <p>I created anaconda environment and installed tensorflow+gpu using <code>pip install tensorflow-gpu</code>. Ipython notebook was used to execute the code above and keep getting the <strong>error</strong>:</p>
  
  <pre><code>InvalidArgumentError: Cannot assign a device for operation 'MatMul': Could not satisfy explicit device specification '/device:GPU:0' because no supported kernel for GPU devices is available.
       [[Node: MatMul = MatMul[T=DT_INT32, transpose_a=false, transpose_b=false, _device="/device:GPU:0"](a, b)]]
  </code></pre>
  
  <p>It seems the <code>MatMul</code> operator is unable to be loaded on the GPU. I have no idea why there is no supported kernel for GPU devices because cuda and cudNN are properly installed. Otherwise, the tensorflow message shows the gpu is recognized: </p>
  
  <pre><code>name: GeForce GTX 1080 Ti
  major: 6 minor: 1 memoryClockRate (GHz) 1.683
  pciBusID 0000:02:00.0
  Total memory: 10.91GiB
  Free memory: 10.75GiB
  2017-11-17 19:12:50.212054: W tensorflow/stream_executor/cuda/cuda_driver.cc:523] A non-primary context 0x55a56f0c2420 exists before initializing the StreamExecutor. We haven't verified StreamExecutor works with that.
  2017-11-17 19:12:50.213035: I tensorflow/core/common_runtime/gpu/gpu_device.cc:955] Found device 1 with properties:
  name: GeForce GTX 1080 Ti
  major: 6 minor: 1 memoryClockRate (GHz) 1.683
  pciBusID 0000:82:00.0
  Total memory: 10.91GiB
  Free memory: 10.75GiB
  2017-11-17 19:12:50.213089: I tensorflow/core/common_runtime/gpu/gpu_device.cc:847] Peer access not supported between device ordinals 0 and 1
  2017-11-17 19:12:50.213108: I tensorflow/core/common_runtime/gpu/gpu_device.cc:847] Peer access not supported between device ordinals 1 and 0
  2017-11-17 19:12:50.213132: I tensorflow/core/common_runtime/gpu/gpu_device.cc:976] DMA: 0 1
  2017-11-17 19:12:50.213148: I tensorflow/core/common_runtime/gpu/gpu_device.cc:986] 0:   Y N
  2017-11-17 19:12:50.213156: I tensorflow/core/common_runtime/gpu/gpu_device.cc:986] 1:   N Y
  2017-11-17 19:12:50.213169: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1045] Creating TensorFlow device (/gpu:0) -&gt; (device: 0, name: GeForce GTX 1080 Ti, pci bus id: 0000:02:00.0)
  2017-11-17 19:12:50.213179: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1045] Creating TensorFlow device (/gpu:1) -&gt; (device: 1, name: GeForce GTX 1080 Ti, pci bus id: 0000:82:00.0)
  Device mapping:
  /job:localhost/replica:0/task:0/gpu:0 -&gt; device: 0, name: GeForce GTX 1080 Ti, pci bus id: 0000:02:00.0
  /job:localhost/replica:0/task:0/gpu:1 -&gt; device: 1, name: GeForce GTX 1080 Ti, pci bus id: 0000:82:00.0
  2017-11-17 19:12:50.471348: I tensorflow/core/common_runtime/direct_session.cc:300] Device mapping:
  /job:localhost/replica:0/task:0/gpu:0 -&gt; device: 0, name: GeForce GTX 1080 Ti, pci bus id: 0000:02:00.0
  /job:localhost/replica:0/task:0/gpu:1 -&gt; device: 1, name: GeForce GTX 1080 Ti, pci bus id: 0000:82:00.0
  </code></pre>
  
  <p>There are two gpus and both of them encountered the same problem. The cuda and cudnn library are installed properly and environment variable is set up in the anaconda. The cuda sample (deviceQuery) code is able to be compiled and run without error and  shows the <code>result = pass</code>. Otherwise, <code>Matmul</code> can be load on CPU and finish computing. The variable <code>a</code> and <code>b</code> in the program is able to be loaded on the GPU device. tensorflow message was given: </p>
  
  <pre><code>2017-11-17 20:27:25.965655: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1045] Creating TensorFlow device (/gpu:0) -&gt; (device: 0, name: GeForce GTX 1080 Ti, pci bus id: 0000:02:00.0)
  2017-11-17 20:27:25.965665: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1045] Creating TensorFlow device (/gpu:1) -&gt; (device: 1, name: GeForce GTX 1080 Ti, pci bus id: 0000:82:00.0)
  Device mapping:
  /job:localhost/replica:0/task:0/gpu:0 -&gt; device: 0, name: GeForce GTX 1080 Ti, pci bus id: 0000:02:00.0
  /job:localhost/replica:0/task:0/gpu:1 -&gt; device: 1, name: GeForce GTX 1080 Ti, pci bus id: 0000:82:00.0
  2017-11-17 20:27:26.228395: I tensorflow/core/common_runtime/direct_session.cc:300] Device mapping:
  /job:localhost/replica:0/task:0/gpu:0 -&gt; device: 0, name: GeForce GTX 1080 Ti, pci bus id: 0000:02:00.0
  /job:localhost/replica:0/task:0/gpu:1 -&gt; device: 1, name: GeForce GTX 1080 Ti, pci bus id: 0000:82:00.0
  
  MatMul: (MatMul): /job:localhost/replica:0/task:0/cpu:0
  2017-11-17 20:27:26.229489: I tensorflow/core/common_runtime/simple_placer.cc:872] MatMul: (MatMul)/job:localhost/replica:0/task:0/cpu:0
  b: (Const): /job:localhost/replica:0/task:0/gpu:0
  2017-11-17 20:27:26.229512: I tensorflow/core/common_runtime/simple_placer.cc:872] b: (Const)/job:localhost/replica:0/task:0/gpu:0
  a: (Const): /job:localhost/replica:0/task:0/gpu:0
  2017-11-17 20:27:26.229526: I tensorflow/core/common_runtime/simple_placer.cc:872] a: (Const)/job:localhost/replica:0/task:0/gpu:0
  </code></pre>
  
  <p>I reinstalled the nvidia driver, cuda and anaconda several times but never solve this problem. It would be great if there are any suggestion.  </p>
  
  <ul>
  <li><strong>OS Platform and Distribution</strong>: Linux Ubuntu 16.04</li>
  <li><strong>TensorFlow installed from</strong>: binary</li>
  <li><strong>TensorFlow version</strong>: 1.3</li>
  <li><strong>Python version</strong>: 2.7.14 </li>
  <li><strong>GCC/Compiler version (if compiling from source)</strong>: 5.4.0</li>
  <li><strong>NVIDIA Driver</strong>: 384.98</li>
  <li><strong>CUDA/cuDNN version</strong>: cuda 8.0/cuDNN 6.0</li>
  <li><strong>GPU model and memory</strong>: Geforce 1080Ti</li>
  </ul>