{"action":"created","issue":{"url":"https://api.github.com/repos/soheilhy/cmux/issues/55","repository_url":"https://api.github.com/repos/soheilhy/cmux","labels_url":"https://api.github.com/repos/soheilhy/cmux/issues/55/labels{/name}","comments_url":"https://api.github.com/repos/soheilhy/cmux/issues/55/comments","events_url":"https://api.github.com/repos/soheilhy/cmux/issues/55/events","html_url":"https://github.com/soheilhy/cmux/issues/55","id":269116415,"number":55,"title":"cmux leaks goroutines","user":{"login":"runcom","id":849915,"avatar_url":"https://avatars2.githubusercontent.com/u/849915?v=4","gravatar_id":"","url":"https://api.github.com/users/runcom","html_url":"https://github.com/runcom","followers_url":"https://api.github.com/users/runcom/followers","following_url":"https://api.github.com/users/runcom/following{/other_user}","gists_url":"https://api.github.com/users/runcom/gists{/gist_id}","starred_url":"https://api.github.com/users/runcom/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/runcom/subscriptions","organizations_url":"https://api.github.com/users/runcom/orgs","repos_url":"https://api.github.com/users/runcom/repos","events_url":"https://api.github.com/users/runcom/events{/privacy}","received_events_url":"https://api.github.com/users/runcom/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":9,"created_at":"2017-10-27T13:58:02Z","updated_at":"2017-10-29T16:45:59Z","closed_at":null,"author_association":"NONE","body":"Hey there, we're using cmux to serve HTTP and gRPC together but our application is receiving a very large amount of requests (both gRPC and HTTP). We see tons of goroutines leaking around like:\r\n```\r\n   813 goroutine 754 [IO wait, 215 minutes]:\r\n   814 internal/poll.runtime_pollWait(0x7eff86b282b0, 0x72, 0x0)\r\n   815         /usr/lib/golang/src/runtime/netpoll.go:173 +0x57\r\n   816 internal/poll.(*pollDesc).wait(0xc420329718, 0x72, 0xffffffffffffff00, 0x341d980, 0x340d       888)\r\n   817         /usr/lib/golang/src/internal/poll/fd_poll_runtime.go:85 +0xae\r\n   818 internal/poll.(*pollDesc).waitRead(0xc420329718, 0xc42172a000, 0x1000, 0x1000)\r\n   819         /usr/lib/golang/src/internal/poll/fd_poll_runtime.go:90 +0x3d\r\n   820 internal/poll.(*FD).Read(0xc420329700, 0xc42172a000, 0x1000, 0x1000, 0x0, 0x0, 0x0)\r\n   821         /usr/lib/golang/src/internal/poll/fd_unix.go:125 +0x18a\r\n   822 net.(*netFD).Read(0xc420329700, 0xc42172a000, 0x1000, 0x1000, 0x13ce, 0xc420956000, 0x13       ce)\r\n   823         /usr/lib/golang/src/net/fd_unix.go:202 +0x52\r\n   824 net.(*conn).Read(0xc4207b2158, 0xc42172a000, 0x1000, 0x1000, 0x0, 0x0, 0x0)\r\n   825         /usr/lib/golang/src/net/net.go:176 +0x6d\r\n   826 github.com/kubernetes-incubator/cri-o/vendor/github.com/soheilhy/cmux.(*bufferedReader).       Read(0xc4207be380, 0xc42172a000, 0x1000, 0x1000, 0xc42098f8c8, 0x74db11, 0xc4202e9688)\r\n   827         /root/go/src/github.com/kubernetes-incubator/cri-o/_output/src/github.com/kubern       etes-incubator/cri-o/vendor/github.com/soheilhy/cmux/buffer.go:53 +0x144\r\n   828 github.com/kubernetes-incubator/cri-o/vendor/github.com/soheilhy/cmux.(*MuxConn).Read(0x       c4207be370, 0xc42172a000, 0x1000, 0x1000, 0x0, 0x13ce, 0x1c6e)\r\n   829         /root/go/src/github.com/kubernetes-incubator/cri-o/_output/src/github.com/kubern       etes-incubator/cri-o/vendor/github.com/soheilhy/cmux/cmux.go:259 +0x4f\r\n   830 net/http.(*connReader).Read(0xc4202e9680, 0xc42172a000, 0x1000, 0x1000, 0xc420460380, 0x       1cee4a0, 0xc420460380)\r\n   831         /usr/lib/golang/src/net/http/server.go:753 +0x105\r\n   832 bufio.(*Reader).fill(0xc4200599e0)\r\n   833         /usr/lib/golang/src/bufio/bufio.go:97 +0x11a\r\n   834 bufio.(*Reader).ReadSlice(0xc4200599e0, 0x46860a, 0x48d7dd, 0xc42098fa20, 0xfffffffe1104       c0ba, 0xc42098fa60, 0x48da77)\r\n   835         /usr/lib/golang/src/bufio/bufio.go:338 +0x2c\r\n   836 bufio.(*Reader).ReadLine(0xc4200599e0, 0x100, 0xf8, 0x2157540, 0x468676, 0x22002098faa8,        0xf8)\r\n   837         /usr/lib/golang/src/bufio/bufio.go:367 +0x34\r\n   838 net/textproto.(*Reader).readLineSlice(0xc4202e96b0, 0xc42098fb20, 0xc42098fb20, 0x41b6d8       , 0x100, 0x2157540)\r\n   839         /usr/lib/golang/src/net/textproto/reader.go:55 +0x70\r\n   840 net/textproto.(*Reader).ReadLine(0xc4202e96b0, 0xc420750800, 0x0, 0x0, 0xc42098fba8)\r\n   841         /usr/lib/golang/src/net/textproto/reader.go:36 +0x2b\r\n   842 net/http.readRequest(0xc4200599e0, 0x0, 0xc420750800, 0x0, 0x0)\r\n   843         /usr/lib/golang/src/net/http/request.go:925 +0x99\r\n   844 net/http.(*conn).readRequest(0xc42096c280, 0x3435900, 0xc4207e4880, 0x0, 0x0, 0x0)\r\n   845         /usr/lib/golang/src/net/http/server.go:933 +0x17c\r\n   846 net/http.(*conn).serve(0xc42096c280, 0x3435900, 0xc4207e4880)\r\n   847         /usr/lib/golang/src/net/http/server.go:1739 +0x50e\r\n   848 created by net/http.(*Server).Serve\r\n   849         /usr/lib/golang/src/net/http/server.go:2720 +0x288\r\n```\r\nDo you guys know what's happening?"},"comment":{"url":"https://api.github.com/repos/soheilhy/cmux/issues/comments/340276115","html_url":"https://github.com/soheilhy/cmux/issues/55#issuecomment-340276115","issue_url":"https://api.github.com/repos/soheilhy/cmux/issues/55","id":340276115,"user":{"login":"runcom","id":849915,"avatar_url":"https://avatars2.githubusercontent.com/u/849915?v=4","gravatar_id":"","url":"https://api.github.com/users/runcom","html_url":"https://github.com/runcom","followers_url":"https://api.github.com/users/runcom/followers","following_url":"https://api.github.com/users/runcom/following{/other_user}","gists_url":"https://api.github.com/users/runcom/gists{/gist_id}","starred_url":"https://api.github.com/users/runcom/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/runcom/subscriptions","organizations_url":"https://api.github.com/users/runcom/orgs","repos_url":"https://api.github.com/users/runcom/repos","events_url":"https://api.github.com/users/runcom/events{/privacy}","received_events_url":"https://api.github.com/users/runcom/received_events","type":"User","site_admin":false},"created_at":"2017-10-29T16:45:59Z","updated_at":"2017-10-29T16:45:59Z","author_association":"NONE","body":"> All of these go routines are stuck in IO wait? can you send me the stack trace of a couple of them? I'm not sure how to reproduce.\r\n\r\nI can send you a stack trace but they look exactly the same as the one I pasted in the first comment"}}
{"actor":{"display_login":"runcom"}}