{"action":"created","issue":{"url":"https://api.github.com/repos/logstash-plugins/logstash-output-stomp/issues/16","repository_url":"https://api.github.com/repos/logstash-plugins/logstash-output-stomp","labels_url":"https://api.github.com/repos/logstash-plugins/logstash-output-stomp/issues/16/labels{/name}","comments_url":"https://api.github.com/repos/logstash-plugins/logstash-output-stomp/issues/16/comments","events_url":"https://api.github.com/repos/logstash-plugins/logstash-output-stomp/issues/16/events","html_url":"https://github.com/logstash-plugins/logstash-output-stomp/issues/16","id":206824322,"number":16,"title":"10-20% message loss when using Stomp Output","user":{"login":"SuperFunks","id":20201219,"avatar_url":"https://avatars.githubusercontent.com/u/20201219?v=3","gravatar_id":"","url":"https://api.github.com/users/SuperFunks","html_url":"https://github.com/SuperFunks","followers_url":"https://api.github.com/users/SuperFunks/followers","following_url":"https://api.github.com/users/SuperFunks/following{/other_user}","gists_url":"https://api.github.com/users/SuperFunks/gists{/gist_id}","starred_url":"https://api.github.com/users/SuperFunks/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/SuperFunks/subscriptions","organizations_url":"https://api.github.com/users/SuperFunks/orgs","repos_url":"https://api.github.com/users/SuperFunks/repos","events_url":"https://api.github.com/users/SuperFunks/events{/privacy}","received_events_url":"https://api.github.com/users/SuperFunks/received_events","type":"User","site_admin":false},"labels":[{"id":140355323,"url":"https://api.github.com/repos/logstash-plugins/logstash-output-stomp/labels/bug","name":"bug","color":"e11d21","default":true}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":48,"created_at":"2017-02-10T15:25:19Z","updated_at":"2017-02-22T14:19:00Z","closed_at":null,"body":"Hi there,\r\n\r\nWe are experiencing 10-20% message drop when sending messages from Logstash 5.2 to a Universal Messaging Queue with the STOMP protocol. (Also verified with Logstash 2.4)\r\n\r\n**Config to reproduce:**\r\n\r\n```\r\ninput {\r\n        tcp {\r\n                port => 5001\r\n                type => \"umrelay\"\r\n        }\r\n}\r\n\r\noutput {\r\n        file {\r\n            path => \"/tmp/stash.out\"\r\n        }\r\n        if [type] == \"umrelay\" {\r\n            stomp {\r\n                 destination => \"/test/testqueue\"\r\n                 host => \"x.x.x.x\"\r\n                port => 17092\r\n            }\r\n        }\r\n        stdout{}\r\n}\r\n```\r\n\r\nWe are using python to send 100.000 messages to the Logstash TCP input - and outs the messages to a file, stomp and console.\r\n\r\nThe file contains all 100.000 messages\r\nThe console writes all 100.000 messages\r\nThe Stomp output produces 83.560 messages\r\n\r\nThe loss happens at rates above 100 pr. second.\r\n\r\nWe have verified the STOMP functionality on UM by running 100.000 messages to it directly with Python - and all messages are persisted.\r\nWe have also tested multiple input path to logstash but we consistantly get everything written to file-out and console, but 10-20% is lost on the STOMP output.\r\n\r\n**Operating System:**\r\nSource: Debian\r\nTarget: OpenSuse\r\n\r\n**Sample Data:**\r\nThe message sent from Python in all experiements is \"i\\n\" where i is the current message number in an iteration of 100.000\r\n\r\n**Steps to reproduce:**\r\nSend messages to any logstash input at a message rate above 100 pr. second, and output on Stomp Output.\r\nCount your input and output and assess the message drop. We see about 10-20% drop.\r\n\r\nThis is a problem - And would like to see a fix to this :wink:\r\n\r\nBest regards,\r\n\r\nSuperFunks"},"comment":{"url":"https://api.github.com/repos/logstash-plugins/logstash-output-stomp/issues/comments/281681430","html_url":"https://github.com/logstash-plugins/logstash-output-stomp/issues/16#issuecomment-281681430","issue_url":"https://api.github.com/repos/logstash-plugins/logstash-output-stomp/issues/16","id":281681430,"user":{"login":"consulthys","id":1280019,"avatar_url":"https://avatars.githubusercontent.com/u/1280019?v=3","gravatar_id":"","url":"https://api.github.com/users/consulthys","html_url":"https://github.com/consulthys","followers_url":"https://api.github.com/users/consulthys/followers","following_url":"https://api.github.com/users/consulthys/following{/other_user}","gists_url":"https://api.github.com/users/consulthys/gists{/gist_id}","starred_url":"https://api.github.com/users/consulthys/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/consulthys/subscriptions","organizations_url":"https://api.github.com/users/consulthys/orgs","repos_url":"https://api.github.com/users/consulthys/repos","events_url":"https://api.github.com/users/consulthys/events{/privacy}","received_events_url":"https://api.github.com/users/consulthys/received_events","type":"User","site_admin":false},"created_at":"2017-02-22T14:19:00Z","updated_at":"2017-02-22T14:19:00Z","body":"This happens [here](https://github.com/stompgem/stomp/blob/42cda45a6e3e90e4677a152e7353bb36017d7a60/lib/stomp/connection.rb#L481-L485), I'll investigate how to avoid this."}}
{"actor":{"display_login":"consulthys"}}