{"action":"created","issue":{"url":"https://api.github.com/repos/apache/incubator-openwhisk/issues/2481","repository_url":"https://api.github.com/repos/apache/incubator-openwhisk","labels_url":"https://api.github.com/repos/apache/incubator-openwhisk/issues/2481/labels{/name}","comments_url":"https://api.github.com/repos/apache/incubator-openwhisk/issues/2481/comments","events_url":"https://api.github.com/repos/apache/incubator-openwhisk/issues/2481/events","html_url":"https://github.com/apache/incubator-openwhisk/issues/2481","id":242676720,"number":2481,"title":"invoker doesn't work propertly in case of heavy load","user":{"login":"keunseob","id":5687661,"avatar_url":"https://avatars3.githubusercontent.com/u/5687661?v=3","gravatar_id":"","url":"https://api.github.com/users/keunseob","html_url":"https://github.com/keunseob","followers_url":"https://api.github.com/users/keunseob/followers","following_url":"https://api.github.com/users/keunseob/following{/other_user}","gists_url":"https://api.github.com/users/keunseob/gists{/gist_id}","starred_url":"https://api.github.com/users/keunseob/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/keunseob/subscriptions","organizations_url":"https://api.github.com/users/keunseob/orgs","repos_url":"https://api.github.com/users/keunseob/repos","events_url":"https://api.github.com/users/keunseob/events{/privacy}","received_events_url":"https://api.github.com/users/keunseob/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":1,"created_at":"2017-07-13T11:58:04Z","updated_at":"2017-07-13T13:10:12Z","closed_at":null,"body":"<!--\r\nDo NOT share passwords, credentials or other confidential information.\r\n\r\nBefore creating a new issue, please check if there is one already open that\r\nfits the defect you are reporting.\r\nIf you open an issue and realize later it is a duplicate of a pre-existing\r\nopen issue, please close yours and add a comment to the other.\r\n\r\nIssues can be created for either defects or enhancement requests. If you are a committer than please add the labels \"bug\" or \"feature\". If you are not a committer please make clear in the comments which one it is, so that committers can add these labels later.\r\n\r\nIf you are reporting a defect, please edit the issue description to include the\r\ninformation shown below.\r\n\r\nIf you are reporting an enhancement request, please include information on what you are trying to achieve and why that enhancement would help you.\r\n\r\nFor more information about reporting issues, see\r\nhttps://github.com/apache/incubator-openwhisk/blob/master/CONTRIBUTING.md#raising-issues\r\n\r\nUse the commands below to provide key information from your environment:\r\nYou do not have to include this information if this is a feature request.\r\n-->\r\n\r\n## Environment details:\r\n\r\n* Openwhisk on Kubernetes based on AWS EC2\r\n* ubuntu 14.04, Kube 1.5.6\r\n\r\n## Steps to reproduce the issue:\r\n\r\n1.   setup Environment Openwhisk on K8s\r\n2.   set 3 invokers pod\r\n2.   create a action (similiar to helloword)\r\n3.   try to requst lots of trigger action as **blocking mode**\r\n\r\n\r\n## Provide the expected results and outputs:\r\nafter lots of request consistently (about 3~4000), invoker can't return proper response. \r\nit returned only activation id, i already knew about it is normal.. but after invokers consumed all of requests, it has to be return proper response.\r\nbut It will not be .... even waiting for a long time.\r\nonce invoker can't return response propertly, invoker didn't work properly anymore\r\nso, i tried to check invoker log as below\r\n\r\n> [2017-07-11T05:54:55.453Z] [WARN] [#tid_113086] [ContainerPool] getContainer has been waiting about 10000 ms: position = 7872 completed = 7869 slack = 1 maxActive = 4 activeCount = 3 startingCounter = 0\r\n> [2017-07-11T05:54:55.453Z] [WARN] [#tid_113086] [ContainerPool] getContainer has been waiting about 10000 ms: position = 7872 completed = 7869 slack = 1 maxActive = 4 activeCount = 3 startingCounter = 0\r\n> [2017-07-11T05:54:55.453Z] [WARN] [#tid_113086] [ContainerPool] getContainer has been waiting about 10000 ms: position = 7872 completed = 7869 slack = 1 maxActive = 4 activeCount = 3 startingCounter = 0\r\n\r\n\r\nin this case, invoker not able to handle requests.\r\nhowever controller log shows it is healthy...\r\n\r\nit need to be fixed... current healthy checker mechanism is used only for checking invoker creation..\r\n\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/apache/incubator-openwhisk/issues/comments/315073064","html_url":"https://github.com/apache/incubator-openwhisk/issues/2481#issuecomment-315073064","issue_url":"https://api.github.com/repos/apache/incubator-openwhisk/issues/2481","id":315073064,"user":{"login":"keunseob","id":5687661,"avatar_url":"https://avatars3.githubusercontent.com/u/5687661?v=3","gravatar_id":"","url":"https://api.github.com/users/keunseob","html_url":"https://github.com/keunseob","followers_url":"https://api.github.com/users/keunseob/followers","following_url":"https://api.github.com/users/keunseob/following{/other_user}","gists_url":"https://api.github.com/users/keunseob/gists{/gist_id}","starred_url":"https://api.github.com/users/keunseob/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/keunseob/subscriptions","organizations_url":"https://api.github.com/users/keunseob/orgs","repos_url":"https://api.github.com/users/keunseob/repos","events_url":"https://api.github.com/users/keunseob/events{/privacy}","received_events_url":"https://api.github.com/users/keunseob/received_events","type":"User","site_admin":false},"created_at":"2017-07-13T13:10:12Z","updated_at":"2017-07-13T13:10:12Z","body":"Thanks for your information. I already know about the poll when i received the activationId from OW.\r\nhowever i'm afraid that it would be not right result user want.\r\nand @rabbah 's PR also looks great. however regarding health checking is not little different approach of me. I give it a try to suggest another PR that i tried and tested recently."}}
{"actor":{"display_login":"keunseob"}}