{"action":"created","issue":{"url":"https://api.github.com/repos/alibaba/DataX/issues/200","repository_url":"https://api.github.com/repos/alibaba/DataX","labels_url":"https://api.github.com/repos/alibaba/DataX/issues/200/labels{/name}","comments_url":"https://api.github.com/repos/alibaba/DataX/issues/200/comments","events_url":"https://api.github.com/repos/alibaba/DataX/issues/200/events","html_url":"https://github.com/alibaba/DataX/issues/200","id":231788282,"number":200,"title":"mongodbreader越来越慢","user":{"login":"ted-baiguan","id":22489431,"avatar_url":"https://avatars1.githubusercontent.com/u/22489431?v=3","gravatar_id":"","url":"https://api.github.com/users/ted-baiguan","html_url":"https://github.com/ted-baiguan","followers_url":"https://api.github.com/users/ted-baiguan/followers","following_url":"https://api.github.com/users/ted-baiguan/following{/other_user}","gists_url":"https://api.github.com/users/ted-baiguan/gists{/gist_id}","starred_url":"https://api.github.com/users/ted-baiguan/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ted-baiguan/subscriptions","organizations_url":"https://api.github.com/users/ted-baiguan/orgs","repos_url":"https://api.github.com/users/ted-baiguan/repos","events_url":"https://api.github.com/users/ted-baiguan/events{/privacy}","received_events_url":"https://api.github.com/users/ted-baiguan/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":1,"created_at":"2017-05-27T09:06:40Z","updated_at":"2017-07-11T06:55:36Z","closed_at":null,"body":"一开始速度很快，然后会越来越慢。我怀疑是遍历mongo表的时候用到了skip+limit，mongodb的skip不会用index，所以是会越来越慢的。\r\n如果改成按照上一次skip的最后一条的objectid为起始点，find+limit下一次，应该会快很多。thanks!"},"comment":{"url":"https://api.github.com/repos/alibaba/DataX/issues/comments/314352661","html_url":"https://github.com/alibaba/DataX/issues/200#issuecomment-314352661","issue_url":"https://api.github.com/repos/alibaba/DataX/issues/200","id":314352661,"user":{"login":"ted-baiguan","id":22489431,"avatar_url":"https://avatars1.githubusercontent.com/u/22489431?v=3","gravatar_id":"","url":"https://api.github.com/users/ted-baiguan","html_url":"https://github.com/ted-baiguan","followers_url":"https://api.github.com/users/ted-baiguan/followers","following_url":"https://api.github.com/users/ted-baiguan/following{/other_user}","gists_url":"https://api.github.com/users/ted-baiguan/gists{/gist_id}","starred_url":"https://api.github.com/users/ted-baiguan/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ted-baiguan/subscriptions","organizations_url":"https://api.github.com/users/ted-baiguan/orgs","repos_url":"https://api.github.com/users/ted-baiguan/repos","events_url":"https://api.github.com/users/ted-baiguan/events{/privacy}","received_events_url":"https://api.github.com/users/ted-baiguan/received_events","type":"User","site_admin":false},"created_at":"2017-07-11T06:55:36Z","updated_at":"2017-07-11T06:55:36Z","body":"here you go:\r\n\r\n{\r\n\"job\": {\r\n    \"setting\": {\r\n        \"speed\": {\r\n            \"channel\": 8\r\n        }\r\n    },\r\n    \"content\": [\r\n        {\r\n            \"reader\": {\r\n                \"name\": \"mongodbreader\",\r\n                \"parameter\": {\r\n                    \"address\": [\"2.2.2.2:3717\"],\r\n                    \"userName\": \"momo_reader\",\r\n                    \"userPassword\": \"xxxxxx\",\r\n                    \"dbName\": \"aaaa\",\r\n                    \"collectionName\": \"bbbb\",\r\n                    \"column\": [\r\n                        {\r\n                            \"name\": \"city\",\r\n                            \"type\": \"string\",\r\n                        },\r\n                        {\r\n                            \"name\": \"uid\",\r\n                            \"type\": \"float\"\r\n                        },\r\n                        {\r\n                            \"name\": \"isodate\",\r\n                            \"type\": \"date\"\r\n                        },\r\n                        {\r\n                            \"name\": \"lvl\",\r\n                            \"type\": \"float\"\r\n                        }\r\n                    ]\r\n                }\r\n            },\r\n            \"writer\": {\r\n                \"name\": \"mysqlwriter\",\r\n                \"parameter\": {\r\n                    \"writeMode\": \"insert\",\r\n                    \"username\": \"root\",\r\n                    \"password\": \"yyyyyy\",\r\n                    \"column\": [\r\n                        \"city\",\r\n                        \"uid\",\r\n                        \"ts\",\r\n                        \"lvl\"\r\n                    ],\r\n                    \"connection\": [\r\n                        {\r\n                            \"jdbcUrl\": \"jdbc:mysql://1.1.1.1:1111/intime?useUnicode=true&characterEncoding=utf-8\",\r\n                            \"table\": [\r\n                                \"user_from_mongo\"\r\n                                ]\r\n                            }\r\n                        ]\r\n                    }\r\n                }\r\n            }\r\n        ]\r\n    }\r\n}"}}
{"actor":{"display_login":"ted-baiguan"}}