title: Unable to access Kubernetes ClusterIP services via nginx-ingress-controller
body: <p>I'm a Kubernetes amateur trying to use NGINX ingress controller on GKE. I'm following <a href="https://cloud.google.com/community/tutorials/nginx-ingress-gke" rel="nofollow noreferrer">this</a> google cloud documentation to setup NGINX Ingress for my services, but, I'm facing issues in accessing the NGINX locations.</p>
  
  <p><strong>What's working?</strong></p>
  
  <ol>
  <li>Ingress-Controller deployment using Helm (RBAC enabled)</li>
  <li>ClusterIP service deployments</li>
  </ol>
  
  <p><strong>What's not working?</strong></p>
  
  <ol>
  <li>Ingress resource to expose multiple ClusterIP services using unique paths (fanout routing)</li>
  </ol>
  
  <p><strong>K8S Services</strong></p>
  
  <pre><code>[msekar@ebs kube-base]$ kubectl get services -n payment-gateway-7682352
  NAME                            TYPE           CLUSTER-IP      EXTERNAL-IP      PORT(S)                      AGE
  nginx-ingress-controller        LoadBalancer   10.35.241.255   35.188.161.171   80:31918/TCP,443:31360/TCP   6h
  nginx-ingress-default-backend   ClusterIP      10.35.251.5     &lt;none&gt;           80/TCP                       6h
  payment-gateway-dev             ClusterIP      10.35.254.167   &lt;none&gt;           5000/TCP                     6h
  payment-gateway-qa              ClusterIP      10.35.253.94    &lt;none&gt;           5000/TCP                     6h
  </code></pre>
  
  <p><strong>K8S Ingress</strong></p>
  
  <pre><code>[msekar@ebs kube-base]$ kubectl get ing -n payment-gateway-7682352
  NAME                HOSTS     ADDRESS          PORTS     AGE
  pgw-nginx-ingress   *         104.198.78.169   80        6h
  
  [msekar@ebs kube-base]$ kubectl describe ing pgw-nginx-ingress -n payment-gateway-7682352
  Name:             pgw-nginx-ingress
  Namespace:        payment-gateway-7682352
  Address:          104.198.78.169
  Default backend:  default-http-backend:80 (10.32.1.4:8080)
  Rules:
    Host  Path  Backends
    ----  ----  --------
    *     
          /dev/   payment-gateway-dev:5000 (&lt;none&gt;)
          /qa/    payment-gateway-qa:5000 (&lt;none&gt;)
  Annotations:
    kubectl.kubernetes.io/last-applied-configuration:  {"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{"kubernetes.io/ingress.class":"nginx","nginx.ingress.kubernetes.io/ssl-redirect":"false"},"name":"pgw-nginx-ingress","namespace":"payment-gateway-7682352"},"spec":{"rules":[{"http":{"paths":[{"backend":{"serviceName":"payment-gateway-dev","servicePort":5000},"path":"/dev/"},{"backend":{"serviceName":"payment-gateway-qa","servicePort":5000},"path":"/qa/"}]}}]}}
  
    kubernetes.io/ingress.class:               nginx
    nginx.ingress.kubernetes.io/ssl-redirect:  false
  Events:                                      &lt;none&gt;
  </code></pre>
  
  <p>Last applied configuration in the annotations (ingress description output) shows the ingress resource manifest. But, I'm pasting it below for reference</p>
  
  <pre><code>---
  apiVersion: extensions/v1beta1
  kind: Ingress
  metadata:
    name: pgw-nginx-ingress
    namespace: payment-gateway-7682352
    annotations:
      kubernetes.io/ingress.class: nginx
      nginx.ingress.kubernetes.io/ssl-redirect: "false"
  spec:
    rules:
    - http:
        paths:
        - backend:
            serviceName: payment-gateway-dev
            servicePort: 5000
          path: /dev/
        - backend:
            serviceName: payment-gateway-qa
            servicePort: 5000
          path: /qa/
  </code></pre>
  
  <p><strong>Additional Info</strong></p>
  
  <p>The services I'm trying to access are springboot services that use contexts, so, the root location isn't a valid end-point.</p>
  
  <p>The container's readiness and liveliness probes are defined accordingly.</p>
  
  <p>For example, "payment-gateway-dev" service is using a context /pgw/v1 context, so, the deployment can only be accessed through the context. To access application's swagger spec you would use the URL</p>
  
  <p>http://&lt;>/pgw/v1/swagger-ui.html</p>
  
  <p><strong>Behaviour of my deployment</strong></p>
  
  <blockquote>
    <p>ingress-controller-LB-ip = 35.188.161.171</p>
  </blockquote>
  
  <ul>
  <li>Accessing ingress controller load balancer "<a href="http://35.188.161.171" rel="nofollow noreferrer">http://35.188.161.171</a>" takes me to default 404 backend</li>
  <li>Accessing ingress controller load balancer health "<a href="http://35.188.161.171/healthz" rel="nofollow noreferrer">http://35.188.161.171/healthz</a>" returns 200 HTTP response as expected</li>
  <li>Trying to access the services using the URLs below returns "404: page not found" error
  
  <ul>
  <li><a href="http://35.188.161.171/dev/pgw/v1/swagger-ui.html" rel="nofollow noreferrer">http://35.188.161.171/dev/pgw/v1/swagger-ui.html</a></li>
  <li><a href="http://35.188.161.171/qa/pgw/v1/swagger-ui.html" rel="nofollow noreferrer">http://35.188.161.171/qa/pgw/v1/swagger-ui.html</a></li>
  </ul></li>
  </ul>
  
  <p>Any suggestions about or insights into what I might be doing wrong will be much appreciated.</p>