{"action":"closed","issue":{"url":"https://api.github.com/repos/lukewaite/logstash-input-cloudwatch-logs/issues/33","repository_url":"https://api.github.com/repos/lukewaite/logstash-input-cloudwatch-logs","labels_url":"https://api.github.com/repos/lukewaite/logstash-input-cloudwatch-logs/issues/33/labels{/name}","comments_url":"https://api.github.com/repos/lukewaite/logstash-input-cloudwatch-logs/issues/33/comments","events_url":"https://api.github.com/repos/lukewaite/logstash-input-cloudwatch-logs/issues/33/events","html_url":"https://github.com/lukewaite/logstash-input-cloudwatch-logs/issues/33","id":243816575,"number":33,"title":"Not getting any CloudWatch logs","user":{"login":"stuffandthings","id":5459591,"avatar_url":"https://avatars0.githubusercontent.com/u/5459591?v=4","gravatar_id":"","url":"https://api.github.com/users/stuffandthings","html_url":"https://github.com/stuffandthings","followers_url":"https://api.github.com/users/stuffandthings/followers","following_url":"https://api.github.com/users/stuffandthings/following{/other_user}","gists_url":"https://api.github.com/users/stuffandthings/gists{/gist_id}","starred_url":"https://api.github.com/users/stuffandthings/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/stuffandthings/subscriptions","organizations_url":"https://api.github.com/users/stuffandthings/orgs","repos_url":"https://api.github.com/users/stuffandthings/repos","events_url":"https://api.github.com/users/stuffandthings/events{/privacy}","received_events_url":"https://api.github.com/users/stuffandthings/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2017-07-18T18:55:28Z","updated_at":"2017-07-18T19:17:47Z","closed_at":"2017-07-18T19:17:47Z","body":"Hi, I'm unable to actually stream any logs. I'm sure my IAM permissions etc are set (I was seeing unauthorized errors before updating them)\r\n\r\nBelow is my Logstash configuration file\r\n```\r\ninput {\r\n  cloudwatch_logs {\r\n    log_group => [ \"/aws/prefix\" ]\r\n    log_group_prefix => true\r\n  }\r\n}\r\n\r\noutput {\r\n  elasticsearch {\r\n    hosts => \"http://host:9200\"\r\n    index => \"cloudwatch-%{+YYYY.MM.dd}\"\r\n  }\r\n  stdout { codec => rubydebug}\r\n}\r\n```\r\n\r\nThis is output from my logstash logs\r\n```\r\n[2017-07-18T17:55:34,221][INFO ][logstash.outputs.elasticsearch] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[http://host:9200/]}}\r\n[2017-07-18T17:55:34,224][INFO ][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck_url=>http://host:9200/, :path=>\"/\"}\r\n[2017-07-18T17:55:34,300][WARN ][logstash.outputs.elasticsearch] Restored connection to ES instance {:url=>#<Java::JavaNet::URI:0x6c7da35a>}\r\n[2017-07-18T17:55:34,302][INFO ][logstash.outputs.elasticsearch] New Elasticsearch output {:class=>\"LogStash::Outputs::ElasticSearch\", :hosts=>[#<Java::JavaNet::URI:0x490865f>]}\r\n[2017-07-18T17:55:34,304][INFO ][logstash.pipeline        ] Starting pipeline {\"id\"=>\"main\", \"pipeline.workers\"=>2, \"pipeline.batch.size\"=>125, \"pipeline.batch.delay\"=>5, \"pipeline.max_inflight\"=>250}\r\n[2017-07-18T17:55:34,332][INFO ][logstash.pipeline        ] Pipeline main started\r\n[2017-07-18T17:55:34,393][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600}\r\n```\r\n\r\nI've tried running `Logstash` with the `--verbose` flag but I'm not seeing any issues or errors. On the AWS side, I can confirm that my credentials were used to access logs. I've tried explicitly adding the `key_id` and `secret_key` in the `cloudwatch_logs` blob to no avail (currently using an IAM role associated with the instance).\r\n\r\nAny ideas on how to debug would be appreciated. I tried adding the output to `stdout` to see if there was an issue with `Elasticsearch` receiving the logs, but the issue seems to be with `Logstash` receiving the logs.\r\nFWIW, I was using `Filebeat` previously and had no issues receiving logs, creating the index, and adding the log data to the ES index.\r\n\r\nThanks!"}}
{"actor":{"display_login":"stuffandthings"}}