{"action":"created","issue":{"url":"https://api.github.com/repos/rancher/rancher/issues/8658","repository_url":"https://api.github.com/repos/rancher/rancher","labels_url":"https://api.github.com/repos/rancher/rancher/issues/8658/labels{/name}","comments_url":"https://api.github.com/repos/rancher/rancher/issues/8658/comments","events_url":"https://api.github.com/repos/rancher/rancher/issues/8658/events","html_url":"https://github.com/rancher/rancher/issues/8658","id":226314289,"number":8658,"title":"Upgrade does nothing","user":{"login":"tholcman","id":791679,"avatar_url":"https://avatars3.githubusercontent.com/u/791679?v=3","gravatar_id":"","url":"https://api.github.com/users/tholcman","html_url":"https://github.com/tholcman","followers_url":"https://api.github.com/users/tholcman/followers","following_url":"https://api.github.com/users/tholcman/following{/other_user}","gists_url":"https://api.github.com/users/tholcman/gists{/gist_id}","starred_url":"https://api.github.com/users/tholcman/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tholcman/subscriptions","organizations_url":"https://api.github.com/users/tholcman/orgs","repos_url":"https://api.github.com/users/tholcman/repos","events_url":"https://api.github.com/users/tholcman/events{/privacy}","received_events_url":"https://api.github.com/users/tholcman/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2017-05-04T15:12:20Z","updated_at":"2017-05-05T05:29:13Z","closed_at":null,"body":"**Rancher Versions:**\r\nServer: rancher/server:v1.5.3\r\nhealthcheck: 0.2.3\r\nipsec: 0.0.5\r\nnetwork-services: 0.1.2\r\nscheduler: 0.5.1\r\n\r\n**Docker Version:**\r\n17.03.x-ce\r\n\r\n**OS and where are the hosts located? (cloud, bare metal, etc):**\r\nAWS\r\n\r\n**Setup Details: (single node rancher vs. HA rancher, internal DB vs. external DB)**\r\nHA 2 nodes with public IPs (primary + failover) + nginx with upstream of both servers\r\n\r\n**Environment Type: (Cattle/Kubernetes/Swarm/Mesos)**\r\nCattle\r\n\r\n**Issue:**\r\nWe had 40 hosts, 1 stack, 2 global services with mapped ports, with 2 sidekicks.\r\n\r\nWe have tried to run `Upgrade` of one service with change of image tag. After 5 seconds Rancher told us `Upgraded` but did nothing. See attached screenshot of logs.\r\n\r\nWe have tried to run upgrade with 5, 4, 2 batch size and it happened almost all times. Even when we run upgrade one by one it is happening from time to time.\r\n\r\nSometimes it upgrades one or more batches and died telling `Upgraded` but half of instances are running with old image. In such case pressing `Finish upgrade` upgrades rest of containers by batch size.\r\n\r\nWe are facing this issue really often with almost all bigger services. There is nothing in logs of Rancher server. Where else can we look? What can we do?\r\n\r\n![rancher-bad-upgrade](https://cloud.githubusercontent.com/assets/791679/25709946/0ed84d30-30eb-11e7-8f2d-6844dc79e9e0.png)\r\n"},"comment":{"url":"https://api.github.com/repos/rancher/rancher/issues/comments/299381577","html_url":"https://github.com/rancher/rancher/issues/8658#issuecomment-299381577","issue_url":"https://api.github.com/repos/rancher/rancher/issues/8658","id":299381577,"user":{"login":"MatthewHartstonge","id":6119549,"avatar_url":"https://avatars1.githubusercontent.com/u/6119549?v=3","gravatar_id":"","url":"https://api.github.com/users/MatthewHartstonge","html_url":"https://github.com/MatthewHartstonge","followers_url":"https://api.github.com/users/MatthewHartstonge/followers","following_url":"https://api.github.com/users/MatthewHartstonge/following{/other_user}","gists_url":"https://api.github.com/users/MatthewHartstonge/gists{/gist_id}","starred_url":"https://api.github.com/users/MatthewHartstonge/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/MatthewHartstonge/subscriptions","organizations_url":"https://api.github.com/users/MatthewHartstonge/orgs","repos_url":"https://api.github.com/users/MatthewHartstonge/repos","events_url":"https://api.github.com/users/MatthewHartstonge/events{/privacy}","received_events_url":"https://api.github.com/users/MatthewHartstonge/received_events","type":"User","site_admin":false},"created_at":"2017-05-05T05:29:13Z","updated_at":"2017-05-05T05:29:13Z","body":"Hi @tholcman, did you tick the `always pull before creating` box under the 'select image' panel?\r\nBut in this case, technically, I would imagine a change of tag should automatically pull the different image..."}}
{"actor":{"display_login":"MatthewHartstonge"}}