{"action":"created","issue":{"url":"https://api.github.com/repos/coreos/prometheus-operator/issues/484","repository_url":"https://api.github.com/repos/coreos/prometheus-operator","labels_url":"https://api.github.com/repos/coreos/prometheus-operator/issues/484/labels{/name}","comments_url":"https://api.github.com/repos/coreos/prometheus-operator/issues/484/comments","events_url":"https://api.github.com/repos/coreos/prometheus-operator/issues/484/events","html_url":"https://github.com/coreos/prometheus-operator/issues/484","id":243949491,"number":484,"title":"Cannot remove alert rule from prometheus","user":{"login":"cofyc","id":91730,"avatar_url":"https://avatars3.githubusercontent.com/u/91730?v=4","gravatar_id":"","url":"https://api.github.com/users/cofyc","html_url":"https://github.com/cofyc","followers_url":"https://api.github.com/users/cofyc/followers","following_url":"https://api.github.com/users/cofyc/following{/other_user}","gists_url":"https://api.github.com/users/cofyc/gists{/gist_id}","starred_url":"https://api.github.com/users/cofyc/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cofyc/subscriptions","organizations_url":"https://api.github.com/users/cofyc/orgs","repos_url":"https://api.github.com/users/cofyc/repos","events_url":"https://api.github.com/users/cofyc/events{/privacy}","received_events_url":"https://api.github.com/users/cofyc/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":5,"created_at":"2017-07-19T07:40:59Z","updated_at":"2017-07-19T10:12:19Z","closed_at":"2017-07-19T10:12:19Z","body":"**What did you do?**\r\n\r\nRemove a alert rule `K8SApiServerLatency` from config map, `<myname>-exporter-kube-api`.\r\n\r\n**What did you expect to see?**\r\n\r\nExpect this alert rule is removed from prometheus.\r\n\r\n**What did you see instead? Under which circumstances?**\r\n\r\nThis alert rule is still in prometheus configuration.\r\n\r\n**Environment**\r\n\r\nLinux.\r\n\r\n* Kubernetes version information:\r\n\r\n```\r\n$ kubectl version\r\nClient Version: version.Info{Major:\"1\", Minor:\"6\", GitVersion:\"v1.6.4\", GitCommit:\"d6f433224538d4f9ca2f7ae19b252e6fcb66a3ae\", GitTreeState:\"clean\", BuildDate:\"2017-05-19T20:41:07Z\", GoVersion:\"go1.8.1\", Compiler:\"gc\", Platform:\"darwin/amd64\"}\r\nServer Version: version.Info{Major:\"1\", Minor:\"6\", GitVersion:\"v1.6.4\", GitCommit:\"d6f433224538d4f9ca2f7ae19b252e6fcb66a3ae\", GitTreeState:\"clean\", BuildDate:\"2017-05-19T18:33:17Z\", GoVersion:\"go1.7.5\", Compiler:\"gc\", Platform:\"linux/amd64\"}\r\n```\r\n\r\n* Kubernetes cluster kind: \r\n\r\n\tkubeadm\r\n\r\n* Manifests:\r\n\r\nNONE\r\n\r\n* Prometheus Operator Logs:\r\n\r\nNONE\r\n\r\n\r\nI entered into prometheus container, and saw two `kube-api` rules:\r\n\r\n- /etc/prometheus/rules/rules-0/kube-api.rules # which is old\r\n- /etc/prometheus/rules/rules-1/kube-api.rules # which is new without `K8SApiServerLatency` alert\r\n\r\nWhy there are two versions of `kube-api` rule rule?\r\nHow can I remove alert rule from prometheus?"},"comment":{"url":"https://api.github.com/repos/coreos/prometheus-operator/issues/comments/316338991","html_url":"https://github.com/coreos/prometheus-operator/issues/484#issuecomment-316338991","issue_url":"https://api.github.com/repos/coreos/prometheus-operator/issues/484","id":316338991,"user":{"login":"cofyc","id":91730,"avatar_url":"https://avatars3.githubusercontent.com/u/91730?v=4","gravatar_id":"","url":"https://api.github.com/users/cofyc","html_url":"https://github.com/cofyc","followers_url":"https://api.github.com/users/cofyc/followers","following_url":"https://api.github.com/users/cofyc/following{/other_user}","gists_url":"https://api.github.com/users/cofyc/gists{/gist_id}","starred_url":"https://api.github.com/users/cofyc/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cofyc/subscriptions","organizations_url":"https://api.github.com/users/cofyc/orgs","repos_url":"https://api.github.com/users/cofyc/repos","events_url":"https://api.github.com/users/cofyc/events{/privacy}","received_events_url":"https://api.github.com/users/cofyc/received_events","type":"User","site_admin":false},"created_at":"2017-07-19T10:12:19Z","updated_at":"2017-07-19T10:12:19Z","body":"hi, @brancz \r\n\r\nProblem resolved, `kubectl -n kube-system get cm -lapp=prometheus,prometheus=xxx` outputs:\r\n\r\n```\r\n$ kubectl -n kube-system get cm -lapp=prometheus,prometheus=xxx\r\nNAME                                   DATA      AGE\r\nxxx-exporter-ku                        4         1d\r\nxxx-exporter-kube-api                  1         1d\r\nxxx-exporter-kube-controller-manager   1         1d\r\nxxx-exporter-kube-etcd                 1         1d\r\nxxx-exporter-kube-scheduler            1         1d\r\nxxx-exporter-kube-state                1         1d\r\nxxx-exporter-kubernetes                1         1d\r\nxxx-exporter-no                        1         23h\r\nxxx-exporter-node                      1         1d \r\n```\r\n\r\nI check `xxx-exporter-ku` and find it has `kube-api.rules`key too. I don't know why this object is created with same contents as `xxx-exporter-kube-api`...\r\n\r\nAfter deletion of this object, the prometheus rules is updated to what I expected.\r\n\r\nThank you very much! It's my bad, I only checked `xxx-exporter-kube-api` name in `configmaps`."}}
{"actor":{"display_login":"cofyc"}}