title: K8s Ingress service returning 503, nothing in Pod logs
body: <p>I am running an <code>Ingress</code>, which supposed to connect to images inside my Pods. When I do <code>describe</code> it looks fine e.g.</p>
  
  <pre><code>$ kubectl describe svc solar-demo
  Name:              solar-demo
  Namespace:         default
  Labels:            &lt;none&gt;
  Annotations:       kubectl.kubernetes.io/last-applied-configuration={"apiVersion":"v1","kind":"Service","metadata":{"annotations":{},"name":"solar-demo","namespace":"default"},"spec":{"ports":[{"name":"city","port":3000...
  Selector:          app=solardemo
  Type:              ClusterIP
  IP:                10.97.245.248
  Port:              city  3000/TCP
  TargetPort:        3000/TCP
  Endpoints:         172.17.0.4:3000,172.17.0.6:3000
  Port:              solar  3001/TCP
  TargetPort:        3001/TCP
  Endpoints:         172.17.0.4:3001,172.17.0.6:3001
  Session Affinity:  None
  Events:            &lt;none&gt;
  </code></pre>
  
  <p>It even correctly lists IP addresses of my images. However, when I try to reach services, I get 404 when I just ask for a root, which is fine since I do not map map <code>/</code> to anything and <code>503</code> error when I try to reach routes <code>/solar</code> and/or <code>/city</code>.</p>
  
  <p>When I check the logs, it returns:</p>
  
  <pre><code>$ kubectl logs solar-demo-5845984b94-xp82l solar-svc
  npm info it worked if it ends with ok
  npm info using npm@5.6.0
  npm info using node@v9.11.2
  npm info lifecycle nf-images-test@1.0.0~prestart-solar-svc: nf-images-test@1.0.0
  npm info lifecycle nf-images-test@1.0.0~start-solar-svc: nf-images-test@1.0.0
  
  &gt; nf-images-test@1.0.0 start-solar-svc /opt/app-root/src
  &gt; node solar-svc.js
  
  {"level":30,"time":1530271233676,"msg":"Server listening at http://0.0.0.0:3001","pid":26,"hostname":"solar-demo-5845984b94-xp82l","v":1}
  server listening on 3001
  </code></pre>
  
  <p>and the same thing for the other service:</p>
  
  <pre><code>$ kubectl logs solar-demo-5845984b94-xp82l api
  npm info it worked if it ends with ok
  npm info using npm@5.6.0
  npm info using node@v9.11.2
  npm info lifecycle nf-images-test@1.0.0~prestart-api: nf-images-test@1.0.0
  npm info lifecycle nf-images-test@1.0.0~start-api: nf-images-test@1.0.0
  
  &gt; nf-images-test@1.0.0 start-api /opt/app-root/src
  &gt; node api.js
  
  {"level":30,"time":1530271244205,"msg":"Server listening at http://0.0.0.0:3000","pid":21,"hostname":"solar-demo-5845984b94-xp82l","v":1}
  server listening on 3000
  </code></pre>
  
  <p>I get 503s and images never get any requests, as if <code>Ingress</code> "thought" that every <code>Pod</code> is down or something. What could I check?</p>
  
  <pre><code>$ curl -v http://shmukler.example.com/solar
  *   Trying 192.168.99.101...
  * Connected to shmukler.example.com (192.168.99.101) port 80 (#0)
  &gt; GET /solar HTTP/1.1
  &gt; Host: shmukler.example.com
  &gt; User-Agent: curl/7.43.0
  &gt; Accept: */*
  &gt; 
  &lt; HTTP/1.1 503 Service Temporarily Unavailable
  &lt; Server: nginx/1.13.7
  &lt; Date: Sun, 01 Jul 2018 13:49:38 GMT
  &lt; Content-Type: text/html
  &lt; Content-Length: 213
  &lt; Connection: keep-alive
  &lt; 
  &lt;html&gt;
  &lt;head&gt;&lt;title&gt;503 Service Temporarily Unavailable&lt;/title&gt;&lt;/head&gt;
  &lt;body bgcolor="white"&gt;
  &lt;center&gt;&lt;h1&gt;503 Service Temporarily Unavailable&lt;/h1&gt;&lt;/center&gt;
  &lt;hr&gt;&lt;center&gt;nginx/1.13.7&lt;/center&gt;
  &lt;/body&gt;
  &lt;/html&gt;
  * Connection #0 to host shmukler.example.com left intact
  </code></pre>
  
  <p>Please advise.</p>