title: AWS EC2 instance role attached but not working
body: <p>I have an ec2 instances running that is suppose to start / stop another ec2 instance. It was working fine but for some reasons it is not working anymore.</p>
  <p>I have Role attached to this instance with the following policy:</p>
  <pre><code>{
      &quot;Version&quot;: &quot;2012-10-17&quot;,
      &quot;Statement&quot;: [
          {
              &quot;Effect&quot;: &quot;Allow&quot;,
              &quot;Action&quot;: [
                  &quot;ec2:DescribeInstances&quot;,
                  &quot;ec2:StartInstances&quot;,
                  &quot;ec2:StopInstances&quot;
              ],
              &quot;Resource&quot;: &quot;*&quot;
          }
      ]
  }
  </code></pre>
  <p>I even try to give to attach admin right for debug but it is not working</p>
  <pre><code>{
      &quot;Version&quot;: &quot;2012-10-17&quot;,
      &quot;Statement&quot;: [
          {
              &quot;Effect&quot;: &quot;Allow&quot;,
              &quot;Action&quot;: &quot;*&quot;,
              &quot;Resource&quot;: &quot;*&quot;
          }
      ]
  }
  </code></pre>
  <p>So it seems the policy is not the issue. I tried to run the describes instance from CLI after ssh to the instance:</p>
  <pre><code>aws ec2 describe-instances --instance-ids i-123456789
  </code></pre>
  <p>I got AuthFailure error as well:</p>
  <pre><code>An error occurred (AuthFailure) when calling the DescribeInstances operation: AWS was not able to validate the provided access credentials
  </code></pre>
  <p>I double check and the role with the policy is attached to the instance. When I run:</p>
  <pre><code>aws sts get-caller-identity
  </code></pre>
  <p>I got</p>
  <pre><code>{
      &quot;UserId&quot;: &quot;AROQWFDSGO54123F7TD:i-00000000&quot;,
      &quot;Account&quot;: &quot;66666666666&quot;,
      &quot;Arn&quot;: &quot;arn:aws:sts::66666666666:assumed-role/MyCustomRoleStartStopWorker/i-00000000&quot;
  }
  </code></pre>
  <p>So I do have the role I want but for some reason it is not &quot;working&quot;.</p>
  <p>Any ideas ?</p>
  <p>EDIT: running the <code>aws ec2 --debug describe-instances 2&gt;&amp;1 | grep credentials</code> give more details:</p>
  <pre><code>[...]
  2021-05-03 19:46:02,352 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: iam-role
  2021-05-03 19:46:02,359 - MainThread - urllib3.connectionpool - DEBUG - http://169.254.169.254:80 &quot;GET /latest/meta-data/iam/security-credentials/ HTTP/1.1&quot; 200 27
  2021-05-03 19:46:02,362 - MainThread - urllib3.connectionpool - DEBUG - http://169.254.169.254:80 &quot;GET /latest/meta-data/iam/security-credentials/MyCustomRoleStartStopWorker HTTP/1.1&quot; 200 1310
  2021-05-03 19:46:02,362 - MainThread - botocore.credentials - DEBUG - Found credentials from IAM Role: MyCustomRoleStartStopWorker
  b'&lt;?xml version=&quot;1.0&quot; encoding=&quot;UTF-8&quot;?&gt;\n&lt;Response&gt;&lt;Errors&gt;&lt;Error&gt;&lt;Code&gt;AuthFailure&lt;/Code&gt;&lt;Message&gt;AWS was not able to validate the provided access credentials&lt;/Message&gt;&lt;/Error&gt;&lt;/Errors&gt;&lt;RequestID&gt;2454f40-6236-43ce-87fe-0ed694fessad&lt;/RequestID&gt;&lt;/Response&gt;'
  botocore.exceptions.ClientError: An error occurred (AuthFailure) when calling the DescribeInstances operation: AWS was not able to validate the provided access credentials
  An error occurred (AuthFailure) when calling the DescribeInstances operation: AWS was not able to validate the provided access credentials
  </code></pre>
  <p>It does find the IAM role but got the <code>AWS was not able to validate the provided access credentials</code> error</p>
  <p>EDIT2: I tried applying the same role on another instance and it work, no AuthFailure issue. But I need it to work on this instance, why could it be not working on this one ?
  There is no <code>~/.aws/credentials</code> file on the instance nor any <code>AWS_*</code> env variable</p>