{"action":"created","issue":{"url":"https://api.github.com/repos/rancher/rancher/issues/7609","repository_url":"https://api.github.com/repos/rancher/rancher","labels_url":"https://api.github.com/repos/rancher/rancher/issues/7609/labels{/name}","comments_url":"https://api.github.com/repos/rancher/rancher/issues/7609/comments","events_url":"https://api.github.com/repos/rancher/rancher/issues/7609/events","html_url":"https://github.com/rancher/rancher/issues/7609","id":204185149,"number":7609,"title":"rancher-efs: containers stuck in starting state when trying to mount a volume in different host","user":{"login":"galal-hussein","id":3309968,"avatar_url":"https://avatars1.githubusercontent.com/u/3309968?v=4","gravatar_id":"","url":"https://api.github.com/users/galal-hussein","html_url":"https://github.com/galal-hussein","followers_url":"https://api.github.com/users/galal-hussein/followers","following_url":"https://api.github.com/users/galal-hussein/following{/other_user}","gists_url":"https://api.github.com/users/galal-hussein/gists{/gist_id}","starred_url":"https://api.github.com/users/galal-hussein/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/galal-hussein/subscriptions","organizations_url":"https://api.github.com/users/galal-hussein/orgs","repos_url":"https://api.github.com/users/galal-hussein/repos","events_url":"https://api.github.com/users/galal-hussein/events{/privacy}","received_events_url":"https://api.github.com/users/galal-hussein/received_events","type":"User","site_admin":false},"labels":[{"id":294075522,"url":"https://api.github.com/repos/rancher/rancher/labels/area/storage","name":"area/storage","color":"fbca04","default":false},{"id":145695744,"url":"https://api.github.com/repos/rancher/rancher/labels/kind/bug","name":"kind/bug","color":"fc2929","default":false}],"state":"closed","locked":false,"assignee":{"login":"StrongMonkey","id":9203378,"avatar_url":"https://avatars0.githubusercontent.com/u/9203378?v=4","gravatar_id":"","url":"https://api.github.com/users/StrongMonkey","html_url":"https://github.com/StrongMonkey","followers_url":"https://api.github.com/users/StrongMonkey/followers","following_url":"https://api.github.com/users/StrongMonkey/following{/other_user}","gists_url":"https://api.github.com/users/StrongMonkey/gists{/gist_id}","starred_url":"https://api.github.com/users/StrongMonkey/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/StrongMonkey/subscriptions","organizations_url":"https://api.github.com/users/StrongMonkey/orgs","repos_url":"https://api.github.com/users/StrongMonkey/repos","events_url":"https://api.github.com/users/StrongMonkey/events{/privacy}","received_events_url":"https://api.github.com/users/StrongMonkey/received_events","type":"User","site_admin":false},"assignees":[{"login":"StrongMonkey","id":9203378,"avatar_url":"https://avatars0.githubusercontent.com/u/9203378?v=4","gravatar_id":"","url":"https://api.github.com/users/StrongMonkey","html_url":"https://github.com/StrongMonkey","followers_url":"https://api.github.com/users/StrongMonkey/followers","following_url":"https://api.github.com/users/StrongMonkey/following{/other_user}","gists_url":"https://api.github.com/users/StrongMonkey/gists{/gist_id}","starred_url":"https://api.github.com/users/StrongMonkey/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/StrongMonkey/subscriptions","organizations_url":"https://api.github.com/users/StrongMonkey/orgs","repos_url":"https://api.github.com/users/StrongMonkey/repos","events_url":"https://api.github.com/users/StrongMonkey/events{/privacy}","received_events_url":"https://api.github.com/users/StrongMonkey/received_events","type":"User","site_admin":false}],"milestone":null,"comments":0,"created_at":"2017-01-31T02:00:51Z","updated_at":"2017-10-31T11:14:53Z","closed_at":"2017-10-31T11:14:53Z","author_association":"NONE","body":"**Rancher Versions:**\r\nServer: v1.4.0-rc5\r\n\r\n**Docker Version:**\r\n1.12\r\n**OS and where are the hosts located? (cloud, bare metal, etc):**\r\nROS 0.7\r\n**Setup Details: (single node rancher vs. HA rancher, internal DB vs. external DB)**\r\nsingle node rancher\r\n**Environment Type: (Cattle/Kubernetes/Swarm/Mesos)**\r\ncattle\r\n**Steps to Reproduce:**\r\n- Register two agent hosts\r\n- Install rancher-ebs catalog vol driver\r\n- create new ebs vol called 'newvol'\r\n- start service that mounts this vol\r\n- remove the volume and service\r\n- Install rancher-efs catalog vol driver\r\n- Create a two efs volumes called 'newvol' and 'efsvol'\r\n- Start a service that mounts the 'efsvol' volume.\r\n- scale the service until new containers started started on both hosts.\r\n\r\n**Results:** \r\n\r\nAll the containers that started on the host where the first container is created are up and running, however all the containers that are scheduled on the second hosts are stuck in starting state.\r\n\r\nThe efs logs of the second hosts:\r\n\r\n```\r\n1/31/2017 2:56:59 AMtime=\"2017-01-31T00:56:59Z\" level=info msg=Running\r\n1/31/2017 2:56:59 AMtime=\"2017-01-31T00:56:59Z\" level=info msg=Starting\r\n1/31/2017 2:56:59 AMtime=\"2017-01-31T00:56:59Z\" level=info msg=\"Running on host hussein-ebs2(1h2) with driver rancher-efs(1sd2)\"\r\n1/31/2017 2:57:00 AMtime=\"2017-01-31T00:57:00Z\" level=info msg=\"Starting plugin for rancher-efs\"\r\n1/31/2017 3:02:07 AMtime=\"2017-01-31T01:02:07Z\" level=info msg=create.request name=newvol options=map[size:1]\r\n1/31/2017 3:04:49 AMtime=\"2017-01-31T01:04:49Z\" level=info msg=\"Updating volume newvol (1v63) rancher-efs:1sd2 on 1h2, state inactive => updating-inactive\" err=<nil>\r\n1/31/2017 3:04:49 AMtime=\"2017-01-31T01:04:49Z\" level=info msg=create.response\r\n1/31/2017 3:04:49 AMtime=\"2017-01-31T01:04:49Z\" level=info msg=attach.request name=newvol\r\n1/31/2017 3:04:49 AMtime=\"2017-01-31T01:04:49Z\" level=info msg=attach.response\r\n1/31/2017 3:04:49 AMtime=\"2017-01-31T01:04:49Z\" level=info msg=mount.request name=newvol\r\n1/31/2017 3:05:53 AMtime=\"2017-01-31T01:05:53Z\" level=info msg=mount.response mountpoint=\"/var/lib/rancher/volumes/rancher-efs/newvol\"\r\n1/31/2017 3:05:53 AMtime=\"2017-01-31T01:05:53Z\" level=info msg=unmount.request name=newvol\r\n1/31/2017 3:05:53 AMtime=\"2017-01-31T01:05:53Z\" level=info msg=unmount.response\r\n1/31/2017 3:05:53 AMtime=\"2017-01-31T01:05:53Z\" level=info msg=mount.request name=newvol\r\n1/31/2017 3:05:53 AMtime=\"2017-01-31T01:05:53Z\" level=info msg=\"newvol already mounted on /var/lib/rancher/volumes/rancher-efs/newvol\"\r\n1/31/2017 3:05:53 AMtime=\"2017-01-31T01:05:53Z\" level=info msg=mount.response mountpoint=\"/var/lib/rancher/volumes/rancher-efs/newvol\"\r\n1/31/2017 3:07:58 AMtime=\"2017-01-31T01:07:58Z\" level=info msg=unmount.request name=newvol\r\n1/31/2017 3:07:58 AMtime=\"2017-01-31T01:07:58Z\" level=info msg=unmount.response\r\n1/31/2017 3:07:58 AMtime=\"2017-01-31T01:07:58Z\" level=info msg=\"container bee8d17eb5a3c01040e777e4c6ab713969125f8ed8720a15a28889b4082f9462 destroyed\"\r\n1/31/2017 3:08:07 AMtime=\"2017-01-31T01:08:07Z\" level=info msg=remove.request name=newvol\r\n1/31/2017 3:08:25 AMtime=\"2017-01-31T01:08:25Z\" level=info msg=remove.response\r\n1/31/2017 3:08:40 AMtime=\"2017-01-31T01:08:40Z\" level=info msg=\"Unmounting /var/lib/rancher/volumes/rancher-efs/newvol\"\r\n```\r\n"},"comment":{"url":"https://api.github.com/repos/rancher/rancher/issues/comments/340731767","html_url":"https://github.com/rancher/rancher/issues/7609#issuecomment-340731767","issue_url":"https://api.github.com/repos/rancher/rancher/issues/7609","id":340731767,"user":{"login":"superseb","id":2620179,"avatar_url":"https://avatars2.githubusercontent.com/u/2620179?v=4","gravatar_id":"","url":"https://api.github.com/users/superseb","html_url":"https://github.com/superseb","followers_url":"https://api.github.com/users/superseb/followers","following_url":"https://api.github.com/users/superseb/following{/other_user}","gists_url":"https://api.github.com/users/superseb/gists{/gist_id}","starred_url":"https://api.github.com/users/superseb/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/superseb/subscriptions","organizations_url":"https://api.github.com/users/superseb/orgs","repos_url":"https://api.github.com/users/superseb/repos","events_url":"https://api.github.com/users/superseb/events{/privacy}","received_events_url":"https://api.github.com/users/superseb/received_events","type":"User","site_admin":false},"created_at":"2017-10-31T11:14:53Z","updated_at":"2017-10-31T11:14:53Z","author_association":"MEMBER","body":"Closing this as rancher-efs is deprecated per v1.6.6, please re-open if this is a generic issue that needs fixing."}}
{"actor":{"display_login":"superseb"}}