{"action":"created","issue":{"url":"https://api.github.com/repos/Parsely/pykafka/issues/602","repository_url":"https://api.github.com/repos/Parsely/pykafka","labels_url":"https://api.github.com/repos/Parsely/pykafka/issues/602/labels{/name}","comments_url":"https://api.github.com/repos/Parsely/pykafka/issues/602/comments","events_url":"https://api.github.com/repos/Parsely/pykafka/issues/602/events","html_url":"https://github.com/Parsely/pykafka/issues/602","id":177751572,"number":602,"title":"Eventlet support with celery . ","user":{"login":"madhur230491","id":15320163,"avatar_url":"https://avatars2.githubusercontent.com/u/15320163?v=3","gravatar_id":"","url":"https://api.github.com/users/madhur230491","html_url":"https://github.com/madhur230491","followers_url":"https://api.github.com/users/madhur230491/followers","following_url":"https://api.github.com/users/madhur230491/following{/other_user}","gists_url":"https://api.github.com/users/madhur230491/gists{/gist_id}","starred_url":"https://api.github.com/users/madhur230491/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/madhur230491/subscriptions","organizations_url":"https://api.github.com/users/madhur230491/orgs","repos_url":"https://api.github.com/users/madhur230491/repos","events_url":"https://api.github.com/users/madhur230491/events{/privacy}","received_events_url":"https://api.github.com/users/madhur230491/received_events","type":"User","site_admin":false},"labels":[{"id":603289911,"url":"https://api.github.com/repos/Parsely/pykafka/labels/gevent","name":"gevent","color":"e99695","default":false},{"id":227537714,"url":"https://api.github.com/repos/Parsely/pykafka/labels/hazy","name":"hazy","color":"fbca04","default":false},{"id":8945347,"url":"https://api.github.com/repos/Parsely/pykafka/labels/question","name":"question","color":"cc317c","default":true}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":3,"created_at":"2016-09-19T10:33:52Z","updated_at":"2017-05-11T21:57:00Z","closed_at":null,"body":"Hi , \r\n\r\nAm looking for some help here . Am using celery workers to send some data to kafka brokers. Am starting celery with evenlet and pykafka client initialization is not working . \r\n\r\nBelow are the option with which I start celery - \r\n`CELERYD_OPTS=\"-Q prppush,productemails -P eventlet\"`\r\n\r\nI read that recenlty supprt for greenlets and gevent was added but not working for me . \r\nThis is how am initialising my client - \r\n`self._kafka_connection = KafkaClient(hosts=\"kafkabroker0.moengage.com:10252,kafkabroker1.moengage.com:10252,kafkabroker2.moengage.com:10252\" ,use_greenlets=True)`\r\n\r\nMy python version - Python 2.7.6\r\nMy pykafka version - latest [_maintainer's note: on 11/19/16, this is 2.5.0_]\r\nkafka brokers - 0.8.2.1\r\n\r\n**My implementation is below** - \r\n\r\n``` python\r\nclass PyKafkaProducerConsumer(object):\r\n    # Put the bellow in all classes which needs singleton!\r\n    __metaclass__ = SingletonMetaClass\r\n\r\n    _kafka_connection = None\r\n    # Moved this to uwsgi init section, check delight/__init__.py\r\n\r\n    def __init__(self,topic=\"test\"):\r\n\r\n        print \"*********** entered pykafka init\"\r\n        self._kafka_connection = KafkaClient(hosts=\"kafkabroker0.moengage.com:10252,kafkabroker1.moengage.com:10252,kafkabroker2.moengage.com:10252\",use_greenlets=True)\r\n        print \"*********** exit pykafka init\"\r\n        #self._topic_connection = self._kafka_connection.topics[topic]\r\n\r\n\r\n\r\n    def sendToKafka(self, topic=\"test\" ,msg={}):\r\n        print \"******* sending thru pykafka\"\r\n        topic_connection = self._kafka_connection.topics[\"test\"]\r\n        json_msg=json.dumps(msg)\r\n        # Asynchronous by default\r\n        producer=topic_connection.get_sync_producer(min_queued_messages=1,delivery_reports=True)\r\n        print producer\r\n        producer.produce(json_msg)\r\n```\r\n\r\nI only see \"**********\\* entered pykafka init\" when I try to initialize the client , and it doesnt return the client object form there . Any help would be very much appreciated as am stuck on this from a long time . \r\n\r\nAlso if celery is started without eventlet , this works fine . \r\n"},"comment":{"url":"https://api.github.com/repos/Parsely/pykafka/issues/comments/300928287","html_url":"https://github.com/Parsely/pykafka/issues/602#issuecomment-300928287","issue_url":"https://api.github.com/repos/Parsely/pykafka/issues/602","id":300928287,"user":{"login":"emmett9001","id":723615,"avatar_url":"https://avatars3.githubusercontent.com/u/723615?v=3","gravatar_id":"","url":"https://api.github.com/users/emmett9001","html_url":"https://github.com/emmett9001","followers_url":"https://api.github.com/users/emmett9001/followers","following_url":"https://api.github.com/users/emmett9001/following{/other_user}","gists_url":"https://api.github.com/users/emmett9001/gists{/gist_id}","starred_url":"https://api.github.com/users/emmett9001/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/emmett9001/subscriptions","organizations_url":"https://api.github.com/users/emmett9001/orgs","repos_url":"https://api.github.com/users/emmett9001/repos","events_url":"https://api.github.com/users/emmett9001/events{/privacy}","received_events_url":"https://api.github.com/users/emmett9001/received_events","type":"User","site_admin":false},"created_at":"2017-05-11T21:57:00Z","updated_at":"2017-05-11T21:57:00Z","body":"A reasonable way forward toward closing this ticket is to verify that the sync producer works when `use_greenlets=True`. "}}
{"actor":{"display_login":"emmett9001"}}