{"action":"created","issue":{"url":"https://api.github.com/repos/kubernetes/kops/issues/2481","repository_url":"https://api.github.com/repos/kubernetes/kops","labels_url":"https://api.github.com/repos/kubernetes/kops/issues/2481/labels{/name}","comments_url":"https://api.github.com/repos/kubernetes/kops/issues/2481/comments","events_url":"https://api.github.com/repos/kubernetes/kops/issues/2481/events","html_url":"https://github.com/kubernetes/kops/issues/2481","id":225798993,"number":2481,"title":"Support private topology deployment behind http proxy","user":{"login":"DerekV","id":791592,"avatar_url":"https://avatars3.githubusercontent.com/u/791592?v=3","gravatar_id":"","url":"https://api.github.com/users/DerekV","html_url":"https://github.com/DerekV","followers_url":"https://api.github.com/users/DerekV/followers","following_url":"https://api.github.com/users/DerekV/following{/other_user}","gists_url":"https://api.github.com/users/DerekV/gists{/gist_id}","starred_url":"https://api.github.com/users/DerekV/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/DerekV/subscriptions","organizations_url":"https://api.github.com/users/DerekV/orgs","repos_url":"https://api.github.com/users/DerekV/repos","events_url":"https://api.github.com/users/DerekV/events{/privacy}","received_events_url":"https://api.github.com/users/DerekV/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":4,"created_at":"2017-05-02T19:39:02Z","updated_at":"2017-06-18T21:00:25Z","closed_at":null,"body":"\r\nCurrently, deploying to a private network without NAT, but with an available http proxy, will fail at a number of points.\r\nI have coerced it to work with some light code modifications to thread environment variables and configuration files through to all the right places.  I will get the code linked here once I get it cleaned up (get out hardcoded urls etc), for reference and further review.\r\n\r\nMeanwhile the question is, does kops want to take on the goal of supporting deployment behind http_proxies?  Even though we have run this way for a couple years now with only minor annoyance, I have realized a number of implications about this configuration and supporting it while trying to get it to work with kops.  Here's the rub: to really do it right, I believe that some discipline will need to be maintained around ensure support is propagated correctly to everywhere http calls (including, for example, AWS or other provider API calls).\r\n\r\nGenerally, the problem is that http_proxy support must be implemented by every http client, and there is no universal, standard way to inject http_proxy configuration. Go net/http will respect http_proxy, https_proxy, and no_proxy environment variables in more or less the same way that wget and curl interpret them, and this is how I was able to get it to work.  The problem is that, if you just set http_proxy, http clients will try to forward every single http request to the proxy, even requests for things located in the same private subnet as Kubernetes, on the overlay network itself, or on some other private IP address.  It does this regardless if the http request uses a hostname or an IP address.\r\n\r\nThe no_proxy environment variable allows you to exclude hostnames and hostname suffixes.  It matches on hostname _suffix_ only, eg `shoes.example.com,.mycorp.com,.mynet.local` would exclude `privateserver.mynet.local`.  So really the core issue behind the core issue is that for some reason, that never seemed weird to me until now, is that ip addresses are expressed most significant \"domain\" _first_, but hostnames express them _last_, and the no_proxy convention assumes only hostnames.  It has to provision to handle ip addresses, where the prefix is the most significant, nor does it understand ranges, globs, nor CIDR notation.   This holds for curl, wget, and by extension, go.  I'm guessing getting all three to change would not be an easy mountain to climb.  I have been able to hack around this by exuding _all ip address urls_ by including every ending ip in no_proxy `.0,.1,.2` ... `.255`, but this feels like a giant hack, is likely fragile (someone may need to address an external server by IP address at some point, which would then break this hack).\r\n\r\nSo environment variables might not be the best answer ultimately.  Perhaps maintaining a light wrapper around net/http \r\n\r\nHere is a rough list of places I modified in the process of getting it to work:\r\n* set the environment variables within cloud-init itself\r\n* added environment variables to `/etc/environment`  (can be done in cloud-init)\r\n* added environment variables to `/etc/default/docker` for docker daemon, can be done in cloud-init (can be done in cloud-init)\r\n* added proxy config to apt.conf.d, ala `echo \"Acquire::http::Proxy \\\"http://${PROXY}\\\";\" > /etc/apt/apt.conf.d/30proxy` (can be done in cloud-init)\r\n* added environment variables to systemctl \"DefaultEnvironment\" ala `echo DefaultEnvironment=http_proxy=${PROXY} https_proxy=${PROXY} ftp_proxy=${PROXY} no_proxy=${NOPROXY} >> /etc/systemd/system.conf`  (can be done in cloud-init)\r\n* modify `nodeup/pkg/model/protokube.go` to pass along environment variables (next to the s3 environment variables already being passed [here](https://github.com/kubernetes/kops/blob/master/nodeup/pkg/model/protokube.go#L248) \r\n* modify manifests for dns-controller and kube-dns addons to pass environment variables\r\n* modify manifests for kube-api and kube-controller manager to pass environment variables"},"comment":{"url":"https://api.github.com/repos/kubernetes/kops/issues/comments/309302638","html_url":"https://github.com/kubernetes/kops/issues/2481#issuecomment-309302638","issue_url":"https://api.github.com/repos/kubernetes/kops/issues/2481","id":309302638,"user":{"login":"DerekV","id":791592,"avatar_url":"https://avatars3.githubusercontent.com/u/791592?v=3","gravatar_id":"","url":"https://api.github.com/users/DerekV","html_url":"https://github.com/DerekV","followers_url":"https://api.github.com/users/DerekV/followers","following_url":"https://api.github.com/users/DerekV/following{/other_user}","gists_url":"https://api.github.com/users/DerekV/gists{/gist_id}","starred_url":"https://api.github.com/users/DerekV/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/DerekV/subscriptions","organizations_url":"https://api.github.com/users/DerekV/orgs","repos_url":"https://api.github.com/users/DerekV/repos","events_url":"https://api.github.com/users/DerekV/events{/privacy}","received_events_url":"https://api.github.com/users/DerekV/received_events","type":"User","site_admin":false},"created_at":"2017-06-18T21:00:25Z","updated_at":"2017-06-18T21:00:25Z","body":"@johnzeringue  I've got a PR that I am working on"}}
{"actor":{"display_login":"DerekV"}}