{"action":"created","issue":{"url":"https://api.github.com/repos/moby/moby/issues/33626","repository_url":"https://api.github.com/repos/moby/moby","labels_url":"https://api.github.com/repos/moby/moby/issues/33626/labels{/name}","comments_url":"https://api.github.com/repos/moby/moby/issues/33626/comments","events_url":"https://api.github.com/repos/moby/moby/issues/33626/events","html_url":"https://github.com/moby/moby/issues/33626","id":235041926,"number":33626,"title":"Docker stack deploy cannot deploy service to different node in swarm cluster, always report that ingress network overlap","user":{"login":"lnshi","id":24580114,"avatar_url":"https://avatars3.githubusercontent.com/u/24580114?v=3","gravatar_id":"","url":"https://api.github.com/users/lnshi","html_url":"https://github.com/lnshi","followers_url":"https://api.github.com/users/lnshi/followers","following_url":"https://api.github.com/users/lnshi/following{/other_user}","gists_url":"https://api.github.com/users/lnshi/gists{/gist_id}","starred_url":"https://api.github.com/users/lnshi/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/lnshi/subscriptions","organizations_url":"https://api.github.com/users/lnshi/orgs","repos_url":"https://api.github.com/users/lnshi/repos","events_url":"https://api.github.com/users/lnshi/events{/privacy}","received_events_url":"https://api.github.com/users/lnshi/received_events","type":"User","site_admin":false},"labels":[{"id":237472034,"url":"https://api.github.com/repos/moby/moby/labels/area/networking","name":"area/networking","color":"fbca04","default":false},{"id":392678049,"url":"https://api.github.com/repos/moby/moby/labels/area/swarm","name":"area/swarm","color":"fbca04","default":false},{"id":75627479,"url":"https://api.github.com/repos/moby/moby/labels/kind/question","name":"kind/question","color":"EBEFC9","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":4,"created_at":"2017-06-11T04:03:28Z","updated_at":"2017-06-12T15:46:39Z","closed_at":null,"body":"I use `docker stack deploy` with `docker-compose.yml v3`, like below:\r\n\r\n```\r\nversion: '3'\r\nnetworks:\r\n  _0:\r\n    driver: overlay\r\n    ipam:\r\n      driver: default\r\n      config:\r\n        - subnet: 172.31.0.0/24\r\nservices:\r\n  registry:\r\n    image: registry:2\r\n    networks:\r\n      - _0\r\n    ports:\r\n      - '7777:5000'\r\n    environment:\r\n      REGISTRY_HTTP_TLS_CERTIFICATE: /certs/server-cert.pem\r\n      REGISTRY_HTTP_TLS_KEY: /certs/server-cert-private-key.pem\r\n      REGISTRY_AUTH: htpasswd\r\n      REGISTRY_AUTH_HTPASSWD_PATH: /auth/htpasswd\r\n      REGISTRY_AUTH_HTPASSWD_REALM: Registry Realm\r\n      REGISTRY_HTTP_SECRET: 94f9b4f9ddae40ed96bbb753b9ed4d14\r\n    volumes:\r\n      - /home/leonard/docker_data_only_container/tls/*.sgdev.vcube.com/server:/certs\r\n      - /home/leonard/docker_data_only_container/registry/auth:/auth\r\n    deploy:\r\n      mode: replicated\r\n      replicas: 1\r\n      placement:\r\n        constraints:\r\n          - engine.labels.name == micro02\r\n      restart_policy:\r\n        condition: any\r\n  portainer:\r\n    image: portainer/portainer\r\n    networks:\r\n      - _0\r\n    ports:\r\n      - '7778:9000'\r\n    volumes:\r\n      - /var/run/docker.sock:/var/run/docker.sock\r\n    deploy:\r\n      mode: replicated\r\n      replicas: 1\r\n      placement:\r\n        constraints:\r\n          - node.role == manager\r\n      restart_policy:\r\n        condition: any\r\n```\r\n\r\nThen do `docker stack deploy --compose-file docker-compose.yml dev` to deploy this stack to the docker swarm cluster:\r\n\r\nif I use the same `constraints` for these two services in the deploy placement, like both `engine.labels.name == micro02`, that means these two services will be located at exactly same node, then it works totally fine,\r\n\r\nbut if I use different `constraints` for these two services in the deploy placement, like one `engine.labels.name == micro02`, another one `node.role == manager`, coz the `micro02` node is not manger, that means the service will be located at different node,  then I will 100% get error:\r\n\r\n```\r\n\"Err\": \"starting container failed: subnet sandbox join failed for \\\"10.255.0.0/16\\\":\r\n    overlay subnet 10.255.0.0/16 failed check with host route table:\r\n    requested network overlaps with existing network\",\r\n```\r\n\r\nWhich the `10.255.0.0/16` network is turned to be the `ingress` network, the `NetworkAttachments` part from the `docker inspect` service command is as follow:\r\n\r\n```\r\n\r\n\"NetworksAttachments\": [\r\n        {\r\n            \"Network\": {\r\n                \"ID\": \"fz2dzdj0oceyxxoh6ndg94pfo\",\r\n                \"Version\": {\r\n                    \"Index\": 6\r\n                },\r\n                \"CreatedAt\": \"2017-06-10T02:44:27.422796031Z\",\r\n                \"UpdatedAt\": \"2017-06-10T02:44:27.441022671Z\",\r\n                \"Spec\": {\r\n                    \"Name\": \"ingress\",\r\n                    \"Labels\": {},\r\n                    \"DriverConfiguration\": {},\r\n                    \"Ingress\": true,\r\n                    \"IPAMOptions\": {\r\n                        \"Driver\": {},\r\n                        \"Configs\": [\r\n                            {\r\n                                \"Subnet\": \"10.255.0.0/16\",\r\n                                \"Gateway\": \"10.255.0.1\"\r\n                            }\r\n                        ]\r\n                    }\r\n                },\r\n                \"DriverState\": {\r\n                    \"Name\": \"overlay\",\r\n                    \"Options\": {\r\n                        \"com.docker.network.driver.overlay.vxlanid_list\": \"4096\"\r\n                    }\r\n                },\r\n                \"IPAMOptions\": {\r\n                    \"Driver\": {\r\n                        \"Name\": \"default\"\r\n                    },\r\n                    \"Configs\": [\r\n                        {\r\n                            \"Subnet\": \"10.255.0.0/16\",\r\n                            \"Gateway\": \"10.255.0.1\"\r\n                        }\r\n                    ]\r\n                }\r\n            },\r\n            \"Addresses\": [\r\n                \"10.255.0.6/16\"\r\n            ]\r\n        },\r\n        {\r\n            \"Network\": {\r\n                \"ID\": \"z7x8t42lc9rscns1u1tvmauq5\",\r\n                \"Version\": {\r\n                    \"Index\": 105031\r\n                },\r\n                \"CreatedAt\": \"2017-06-11T03:22:01.312281009Z\",\r\n                \"UpdatedAt\": \"2017-06-11T03:22:01.313259979Z\",\r\n                \"Spec\": {\r\n                    \"Name\": \"dev_stg_registry__0\",\r\n                    \"Labels\": {\r\n                        \"com.docker.stack.namespace\": \"dev_stg_registry\"\r\n                    },\r\n                    \"DriverConfiguration\": {\r\n                        \"Name\": \"overlay\"\r\n                    },\r\n                    \"IPAMOptions\": {\r\n                        \"Driver\": {\r\n                            \"Name\": \"default\"\r\n                        },\r\n                        \"Configs\": [\r\n                            {\r\n                                \"Subnet\": \"172.31.0.0/24\",\r\n                                \"Gateway\": \"172.31.0.1\"\r\n                            }\r\n                        ]\r\n                    }\r\n                },\r\n                \"DriverState\": {\r\n                    \"Name\": \"overlay\",\r\n                    \"Options\": {\r\n                        \"com.docker.network.driver.overlay.vxlanid_list\": \"4097\"\r\n                    }\r\n                },\r\n                \"IPAMOptions\": {\r\n                    \"Driver\": {\r\n                        \"Name\": \"default\"\r\n                    },\r\n                    \"Configs\": [\r\n                        {\r\n                            \"Subnet\": \"172.31.0.0/24\",\r\n                            \"Gateway\": \"172.31.0.1\"\r\n                        }\r\n                    ]\r\n                }\r\n            },\r\n            \"Addresses\": [\r\n                \"172.31.0.3/24\"\r\n            ]\r\n        }\r\n    ]\r\n```\r\n\r\nDocker swarm cluster is provisioned by using docker-machine with generic driver.\r\n\r\nOutput of `docker info` from the docker which is used in the docker swarm:\r\n\r\n```\r\nContainers: 5\r\n Running: 0\r\n Paused: 0\r\n Stopped: 5\r\nImages: 1\r\nServer Version: 17.05.0-ce\r\nStorage Driver: aufs\r\n Root Dir: /var/lib/docker/aufs\r\n Backing Filesystem: extfs\r\n Dirs: 13\r\n Dirperm1 Supported: false\r\nLogging Driver: json-file\r\nCgroup Driver: cgroupfs\r\nPlugins:\r\n Volume: local\r\n Network: bridge host macvlan null overlay\r\nSwarm: active\r\n NodeID: vq9b76vp4wuj7qwoxr87w4w6z\r\n Is Manager: true\r\n ClusterID: n6ekfek40unb602r2zphmgutd\r\n Managers: 1\r\n Nodes: 3\r\n Orchestration:\r\n  Task History Retention Limit: 5\r\n Raft:\r\n  Snapshot Interval: 10000\r\n  Number of Old Snapshots to Retain: 0\r\n  Heartbeat Tick: 1\r\n  Election Tick: 3\r\n Dispatcher:\r\n  Heartbeat Period: 5 seconds\r\n CA Configuration:\r\n  Expiry Duration: 3 months\r\n Node Address: 10.0.3.159\r\n Manager Addresses:\r\n  10.0.3.159:2377\r\nRuntimes: runc\r\nDefault Runtime: runc\r\nInit Binary: docker-init\r\ncontainerd version: 9048e5e50717ea4497b757314bad98ea3763c145\r\nrunc version: 9c2d8d184e5da67c95d601382adf14862e4f2228\r\ninit version: 949e6fa\r\nSecurity Options:\r\n apparmor\r\nKernel Version: 3.13.0-100-generic\r\nOperating System: Ubuntu 14.04.5 LTS\r\nOSType: linux\r\nArchitecture: x86_64\r\nCPUs: 1\r\nTotal Memory: 992.5 MiB\r\nName: micro01.sgdev.vcube.com\r\nID: DDWK:3I3I:GIOH:NS3C:BNUI:CX2I:6ANQ:MCBL:K4FL:2QE2:K5AH:C3VL\r\nDocker Root Dir: /var/lib/docker\r\nDebug Mode (client): false\r\nDebug Mode (server): false\r\nRegistry: https://index.docker.io/v1/\r\nWARNING: No swap limit support\r\nLabels:\r\n name=micro01\r\n provider=generic\r\nExperimental: false\r\nInsecure Registries:\r\n micro02.sgdev.vcube.com:7777\r\n 127.0.0.0/8\r\nLive Restore Enabled: false\r\n```"},"comment":{"url":"https://api.github.com/repos/moby/moby/issues/comments/307830506","html_url":"https://github.com/moby/moby/issues/33626#issuecomment-307830506","issue_url":"https://api.github.com/repos/moby/moby/issues/33626","id":307830506,"user":{"login":"lnshi","id":24580114,"avatar_url":"https://avatars3.githubusercontent.com/u/24580114?v=3","gravatar_id":"","url":"https://api.github.com/users/lnshi","html_url":"https://github.com/lnshi","followers_url":"https://api.github.com/users/lnshi/followers","following_url":"https://api.github.com/users/lnshi/following{/other_user}","gists_url":"https://api.github.com/users/lnshi/gists{/gist_id}","starred_url":"https://api.github.com/users/lnshi/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/lnshi/subscriptions","organizations_url":"https://api.github.com/users/lnshi/orgs","repos_url":"https://api.github.com/users/lnshi/repos","events_url":"https://api.github.com/users/lnshi/events{/privacy}","received_events_url":"https://api.github.com/users/lnshi/received_events","type":"User","site_admin":false},"created_at":"2017-06-12T15:46:39Z","updated_at":"2017-06-12T15:46:39Z","body":"@mavenugo Then later if I need to promote some worker node to manager, or I need to add some new manager node, what should I do?\r\n\r\nWill this be completely fixed in next release?"}}
{"actor":{"display_login":"lnshi"}}