title: GCP - Cloud Composer 2 - Create operation on this environment failed
body: <p>I am trying to create a default Composer 2 Instance on GCP and get the Errors:</p>
  <pre><code>CREATE operation on this environment failed 32 minutes ago with the following error message:
  Composer Backend timed out. Currently running tasks are [stage: CP_COMPOSER_AGENT_RUNNING
  description: &quot;No agent response published.&quot;
  ...
  </code></pre>
  <p>or</p>
  <pre><code>CREATE operation on this environment failed 32 minutes ago with the following error message:
  Environment couldn't be created, but no error was surfaced. This can be cause by a lack of 
  proper permissions. Check if this environment's service account ... .iam.gserviceaccount.com 
  has the 'roles/composer.worker' role and there is no firewall inhibiting internal 
  communications set.
  </code></pre>
  <p>I already tried to add the Composer Worker role to the service account and all other required roles (e.g. Cloud Composer v2 API Service Agent Extension) in <a href="https://cloud.google.com/composer/docs/composer-2/access-control" rel="nofollow noreferrer">https://cloud.google.com/composer/docs/composer-2/access-control</a> (for public as well as for private, eventhough instance is public).</p>
  <p>I looked into the GKE instance and found the Pod composer-agent failing with:</p>
  <pre><code>Traceback (most recent call last): File &quot;composer_agent.py&quot;, line 467, in &lt;module&gt; main() File &quot;composer_agent.py&quot;, line 292, in main responses = pubsub_subscriber.pull() (...)
  oauth2client.client.HttpAccessTokenRefreshError: Failed to retrieve
  http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/(...)
  compute@developer.gserviceaccount.com/token from the Google Compute Enginemetadata service. 
  Response:
  {'date': 'Thu, 17 Feb 2022 10:29:46 GMT', 'status': '403', 'content-length': '668', 'content- 
  type': 'text/plain; charset=utf-8', 'x-content-type-options': 'nosniff'}
  </code></pre>
  <p>So I assume there is still some permission issue but I can not figure out what, Composer 1 Instances can be created without a problem, as well for a different project Composer 2 Instances with the same permissions on the service accounts.</p>
  <p>I also tried to create different than default compute service account with the required permissions but also without success. I also checked that the service account I am adding permissions to is actually the service account sending the request in the composer-agent and is sending the environment creation request to the GKE cluster.</p>
  <p>I hope anyone can help, who faced similar issues or knows more about the error occuring in composer-agent, thank you very much!</p>