{"action":"created","issue":{"url":"https://api.github.com/repos/apache/kafka/issues/2967","repository_url":"https://api.github.com/repos/apache/kafka","labels_url":"https://api.github.com/repos/apache/kafka/issues/2967/labels{/name}","comments_url":"https://api.github.com/repos/apache/kafka/issues/2967/comments","events_url":"https://api.github.com/repos/apache/kafka/issues/2967/events","html_url":"https://github.com/apache/kafka/pull/2967","id":226113355,"number":2967,"title":"KAFKA-5150 reduce lz4 decompression overhead","user":{"login":"xvrl","id":815147,"avatar_url":"https://avatars1.githubusercontent.com/u/815147?v=3","gravatar_id":"","url":"https://api.github.com/users/xvrl","html_url":"https://github.com/xvrl","followers_url":"https://api.github.com/users/xvrl/followers","following_url":"https://api.github.com/users/xvrl/following{/other_user}","gists_url":"https://api.github.com/users/xvrl/gists{/gist_id}","starred_url":"https://api.github.com/users/xvrl/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/xvrl/subscriptions","organizations_url":"https://api.github.com/users/xvrl/orgs","repos_url":"https://api.github.com/users/xvrl/repos","events_url":"https://api.github.com/users/xvrl/events{/privacy}","received_events_url":"https://api.github.com/users/xvrl/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":21,"created_at":"2017-05-03T21:00:48Z","updated_at":"2017-05-04T17:30:31Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/kafka/pulls/2967","html_url":"https://github.com/apache/kafka/pull/2967","diff_url":"https://github.com/apache/kafka/pull/2967.diff","patch_url":"https://github.com/apache/kafka/pull/2967.patch"},"body":"- reuse decompression buffers, keeping one per thread\r\n- switch lz4 input stream to operate directly on ByteBuffers\r\n- more tests with both compressible / incompressible data, multiple\r\n  blocks, and various other combinations to increase code coverage\r\n- fixes bug that would cause EOFException instead of invalid block size\r\n  for invalid incompressible blocks\r\n\r\nOverall this improves LZ4 decompression performance by up to 23x for small batches.\r\nMost improvements are seen for batches of size 1 with messages on the order of ~100B.\r\nAt least 10x improvements for for batch sizes of < 10 messages, with messages of < 10kB\r\n\r\nSee benchmark code and results here\r\nhttps://gist.github.com/xvrl/05132e0643513df4adf842288be86efd"},"comment":{"url":"https://api.github.com/repos/apache/kafka/issues/comments/299254452","html_url":"https://github.com/apache/kafka/pull/2967#issuecomment-299254452","issue_url":"https://api.github.com/repos/apache/kafka/issues/2967","id":299254452,"user":{"login":"asfbot","id":5243949,"avatar_url":"https://avatars0.githubusercontent.com/u/5243949?v=3","gravatar_id":"","url":"https://api.github.com/users/asfbot","html_url":"https://github.com/asfbot","followers_url":"https://api.github.com/users/asfbot/followers","following_url":"https://api.github.com/users/asfbot/following{/other_user}","gists_url":"https://api.github.com/users/asfbot/gists{/gist_id}","starred_url":"https://api.github.com/users/asfbot/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/asfbot/subscriptions","organizations_url":"https://api.github.com/users/asfbot/orgs","repos_url":"https://api.github.com/users/asfbot/repos","events_url":"https://api.github.com/users/asfbot/events{/privacy}","received_events_url":"https://api.github.com/users/asfbot/received_events","type":"User","site_admin":false},"created_at":"2017-05-04T17:30:31Z","updated_at":"2017-05-04T17:30:31Z","body":"\nRefer to this link for build results (access rights to CI server needed): \nhttps://builds.apache.org/job/kafka-pr-jdk8-scala2.11/3496/\nTest FAILed (JDK 8 and Scala 2.11).\n"}}
{"actor":{"display_login":"asfbot"}}