{"action":"created","issue":{"url":"https://api.github.com/repos/citusdata/citus/issues/1617","repository_url":"https://api.github.com/repos/citusdata/citus","labels_url":"https://api.github.com/repos/citusdata/citus/issues/1617/labels{/name}","comments_url":"https://api.github.com/repos/citusdata/citus/issues/1617/comments","events_url":"https://api.github.com/repos/citusdata/citus/issues/1617/events","html_url":"https://github.com/citusdata/citus/issues/1617","id":252357307,"number":1617,"title":"Out of shared memory error during JDBC testing","user":{"login":"furkansahin","id":6233557,"avatar_url":"https://avatars3.githubusercontent.com/u/6233557?v=4","gravatar_id":"","url":"https://api.github.com/users/furkansahin","html_url":"https://github.com/furkansahin","followers_url":"https://api.github.com/users/furkansahin/followers","following_url":"https://api.github.com/users/furkansahin/following{/other_user}","gists_url":"https://api.github.com/users/furkansahin/gists{/gist_id}","starred_url":"https://api.github.com/users/furkansahin/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/furkansahin/subscriptions","organizations_url":"https://api.github.com/users/furkansahin/orgs","repos_url":"https://api.github.com/users/furkansahin/repos","events_url":"https://api.github.com/users/furkansahin/events{/privacy}","received_events_url":"https://api.github.com/users/furkansahin/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":4,"created_at":"2017-08-23T17:23:13Z","updated_at":"2017-09-13T16:53:18Z","closed_at":null,"author_association":"MEMBER","body":"I used JDBCReleaseTest.java file under our test-automation/jdbc.\r\nFirstly I ran the tpch_create_hash_partitioned_tables.ddl which is in the same repo, under tpch_2_13_0 folder. After creating and distributing the table in a single machine Citus cluster (1C, 2W) I ran the JDBCReleaseTest.java just like test automation does;\r\n```\r\njavac JDBCReleaseTest.java\r\njava -classpath postgresql-9.4.1212.jre6.jar:. JDBCReleaseTest\r\n```\r\nOf course you have to have a valid jdbc file. \r\nAfter starting the execution, just monitor the memory usage.\r\nEven if the tables are empty, it's using massive amount of memory for a while and then it stops the execution with the following message \r\n![screen shot 2017-08-23 at 20 25 35](https://user-images.githubusercontent.com/6233557/29629388-46cfc724-8841-11e7-88c3-67dca96ba319.png)\r\n\r\nIf I run the query from the method which casued the error manually in psql, the error message becomes\r\n![screen shot 2017-08-23 at 19 40 38](https://user-images.githubusercontent.com/6233557/29629184-943a4f4e-8840-11e7-8c06-827d95363b0c.png)\r\n\r\nExecution ends in a point depending on your memory tho. In my local machine, it stoped at the test_no_4, but in a larger machine in EC2, it went up to simplePreparedTest1()."},"comment":{"url":"https://api.github.com/repos/citusdata/citus/issues/comments/329229938","html_url":"https://github.com/citusdata/citus/issues/1617#issuecomment-329229938","issue_url":"https://api.github.com/repos/citusdata/citus/issues/1617","id":329229938,"user":{"login":"ozgune","id":2545443,"avatar_url":"https://avatars3.githubusercontent.com/u/2545443?v=4","gravatar_id":"","url":"https://api.github.com/users/ozgune","html_url":"https://github.com/ozgune","followers_url":"https://api.github.com/users/ozgune/followers","following_url":"https://api.github.com/users/ozgune/following{/other_user}","gists_url":"https://api.github.com/users/ozgune/gists{/gist_id}","starred_url":"https://api.github.com/users/ozgune/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ozgune/subscriptions","organizations_url":"https://api.github.com/users/ozgune/orgs","repos_url":"https://api.github.com/users/ozgune/repos","events_url":"https://api.github.com/users/ozgune/events{/privacy}","received_events_url":"https://api.github.com/users/ozgune/received_events","type":"User","site_admin":false},"created_at":"2017-09-13T16:53:18Z","updated_at":"2017-09-13T16:53:18Z","author_association":"OWNER","body":"@furkansahin -- We may have three separate issues in this thread. Let's break them down and focus on one of them.\r\n\r\n1. @mtuncer commented on an issue related to `CheckNodeCopyAndSerialization()`. We were also able to reproduce this issue in #542. This issue should be fixed in Citus 7.0 - unless assertions are enabled. This is a memory issue, not a \"shared memory\" issue.\r\n\r\n2. Task tracker works like the following. When you issue a query, the coordinator breaks that query into query fragments and sends them worker nodes. Those query fragments then take shared memory on worker nodes. Once the query completes, the coordinator sends a cleanup task to worker nodes. This cleans up all shared memory allocated by query fragments.\r\n\r\nIf you're running on a dev box, your machine may be overloaded by work. In that case, the cleanup task may never get assigned to worker nodes. In that case, the shared memory could keep growing.\r\n\r\n3. JDBC related issues. I haven't run tests from JDBC in a while. Could you try running the same test suite on your developer box without JDBC and see if you can reproduce the problem? \r\n\r\nIf you can, it means the issue is with (2). If not, this issue could be related to JDBC."}}
{"actor":{"display_login":"ozgune"}}