title: Kubernetes continuously killing and recreating last pod
body: <p>The last(3rd) container is continuously being delete and recreated by kubernetes. It goes from Running to Terminating state. The Kubernetes UI shows status as : 'Terminated: ExitCode:${state.terminated.exitCode}' </p>
  
  <p>My deployment YAML:</p>
  
  <pre><code>apiVersion: autoscaling/v1
  kind: HorizontalPodAutoscaler
  metadata:
    name: openapi
  spec:
    scaleTargetRef:
      kind: Deployment
      name: openapi
    minReplicas: 3
    maxReplicas: 10
  targetCPUUtilizationPercentage: 75
  
  ---
  kind: Service
  apiVersion: v1
  metadata:
    name: openapi
  spec:
    selector:
      app: openapi
    ports:
      - name: http
        protocol: TCP
        port: 80
        targetPort: 8080
      - name: https
        protocol: TCP
        port: 443
        targetPort: 8443
   type: LoadBalancer
  
  ---
  apiVersion: extensions/v1beta1
  kind: Deployment
  metadata:
    name: openapi
  spec:
    template:
      metadata:
        labels:
          app: openapi
      spec:
        containers:
        - name: openapi
          image: us.gcr.io/PROJECT_ID/openapi:latest
          imagePullPolicy: Always
          ports:
          - containerPort: 8080
  </code></pre>
  
  <p>Portion of Output of <code>kubectl get events -n namespace</code>: </p>
  
  <pre><code>Pod                       Normal    Created                 kubelet                     Created container
  Pod                       Normal    Started                 kubelet                     Started container
  Pod                       Normal    Killing                 kubelet                     Killing container with id docker://openapi:Need to kill Pod
  ReplicaSet                Normal    SuccessfulCreate        replicaset-controller       (combined from similar events): Created pod: openapi-7db5f8d479-p7mcl
  ReplicaSet                Normal    SuccessfulDelete        replicaset-controller       (combined from similar events): Deleted pod: openapi-7db5f8d479-pgmxf
  HorizontalPodAutoscaler   Normal    SuccessfulRescale       horizontal-pod-autoscaler    New size: 2; reason: Current number of replicas above Spec.MaxReplicas
  HorizontalPodAutoscaler   Normal    SuccessfulRescale       horizontal-pod-autoscaler    New size: 3; reason: Current number of replicas below Spec.MinReplicas
  Deployment                Normal    ScalingReplicaSet       deployment-controller        Scaled up replica set openapi-7db5f8d479 to 3
  Deployment                Normal    ScalingReplicaSet       deployment-controller        Scaled down replica set openapi-7db5f8d479 to 2
  </code></pre>
  
  <p><code>kubectl describe pod -n default openapi-7db5f8d479-2d2nm</code> for a pod that spawned and was killed:</p>
  
  <p>A different pod with a different unique id spawns each time after a pod gets killed by Kubernetes.</p>
  
  <pre><code>Name:                      openapi-7db5f8d479-2d2nm
  Namespace:                 default
  Node:                      gke-testproject-default-pool-28ce3836-t4hp/10.150.0.2
  Start Time:                Thu, 23 Nov 2017 11:50:17 +0000
  Labels:                    app=openapi
                             pod-template-hash=3861948035
  Annotations:               kubernetes.io/created-by={"kind":"SerializedReference","apiVersion":"v1","reference":{"kind":"ReplicaSet","namespace":"default","name":"openapi-7db5f8d479","uid":"b7b3e48f-ceb2-11e7-afe7-42010a960003"...
                             kubernetes.io/limit-ranger=LimitRanger plugin set: cpu request for container openapi
  Status:                    Terminating (expires Thu, 23 Nov 2017 11:51:04 +0000)
  Termination Grace Period:  30s
  IP:
  Created By:                ReplicaSet/openapi-7db5f8d479
  Controlled By:             ReplicaSet/openapi-7db5f8d479
  Containers:
    openapi:
      Container ID:   docker://93d2f1372a7ad004aaeb34b0bc9ee375b6ed48609f505b52495067dd0dcbb233
      Image:          us.gcr.io/testproject-175705/openapi:latest
      Image ID:       docker-pullable://us.gcr.io/testproject-175705/openapi@sha256:54b833548cbed32db36ba4808b33c87c15c4ecde673839c3922577f30b
      Port:           8080/TCP
      State:          Terminated
        Reason:       Error
        Exit Code:    143
        Started:      Thu, 23 Nov 2017 11:50:18 +0000
        Finished:     Thu, 23 Nov 2017 11:50:35 +0000
      Ready:          False
      Restart Count:  0
      Requests:
        cpu:        100m
      Environment:  &lt;none&gt;
      Mounts:
        /var/run/secrets/kubernetes.io/serviceaccount from default-token-61k6c (ro)
  Conditions:
    Type           Status
    Initialized    True
    Ready          False
    PodScheduled   True
  Volumes:
    default-token-61k6c:
      Type:        Secret (a volume populated by a Secret)
      SecretName:  default-token-61k6c
      Optional:    false
  QoS Class:       Burstable
  Node-Selectors:  &lt;none&gt;
  Tolerations:     node.alpha.kubernetes.io/notReady:NoExecute for 300s
                   node.alpha.kubernetes.io/unreachable:NoExecute for 300s
  Events:
    Type    Reason                 Age   From                                                Message
    ----    ------                 ----  ----                                                -------
    Normal  Scheduled              21s   default-scheduler                                   Successfully assigned openapi-7db5f8d479-2d2nm to gke-testproject-default-pool-28ce3836-t4hp
    Normal  SuccessfulMountVolume  21s   kubelet, gke-testproject-default-pool-28ce3836-t4hp  MountVolume.SetUp succeeded for volume "default-token-61k6c"
    Normal  Pulling                21s   kubelet, gke-testproject-default-pool-28ce3836-t4hp  pulling image "us.gcr.io/testproject-175705/openapi:latest"
    Normal  Pulled                 20s   kubelet, gke-testproject-default-pool-28ce3836-t4hp  Successfully pulled image "us.gcr.io/testproject-175705/openapi:latest"
    Normal  Created                20s   kubelet, gke-testproject-default-pool-28ce3836-t4hp  Created container
    Normal  Started                20s   kubelet, gke-testproject-default-pool-28ce3836-t4hp  Started container
    Normal  Killing                3s    kubelet, gke-testproject-default-pool-28ce3836-t4hp  Killing container with id docker://openapi:Need to kill Pod
  </code></pre>