{"action":"created","issue":{"url":"https://api.github.com/repos/magento/magento2/issues/4531","repository_url":"https://api.github.com/repos/magento/magento2","labels_url":"https://api.github.com/repos/magento/magento2/issues/4531/labels{/name}","comments_url":"https://api.github.com/repos/magento/magento2/issues/4531/comments","events_url":"https://api.github.com/repos/magento/magento2/issues/4531/events","html_url":"https://github.com/magento/magento2/issues/4531","id":155114595,"number":4531,"title":"Duplicated & missing product rows in system product export","user":{"login":"wbyrnetx","id":4029452,"avatar_url":"https://avatars0.githubusercontent.com/u/4029452?v=4","gravatar_id":"","url":"https://api.github.com/users/wbyrnetx","html_url":"https://github.com/wbyrnetx","followers_url":"https://api.github.com/users/wbyrnetx/followers","following_url":"https://api.github.com/users/wbyrnetx/following{/other_user}","gists_url":"https://api.github.com/users/wbyrnetx/gists{/gist_id}","starred_url":"https://api.github.com/users/wbyrnetx/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/wbyrnetx/subscriptions","organizations_url":"https://api.github.com/users/wbyrnetx/orgs","repos_url":"https://api.github.com/users/wbyrnetx/repos","events_url":"https://api.github.com/users/wbyrnetx/events{/privacy}","received_events_url":"https://api.github.com/users/wbyrnetx/received_events","type":"User","site_admin":false},"labels":[{"id":429635121,"url":"https://api.github.com/repos/magento/magento2/labels/2.0.x","name":"2.0.x","color":"fbca04","default":false},{"id":429635502,"url":"https://api.github.com/repos/magento/magento2/labels/2.1.x","name":"2.1.x","color":"fbca04","default":false},{"id":21716835,"url":"https://api.github.com/repos/magento/magento2/labels/acknowledged","name":"acknowledged","color":"02d7e1","default":false},{"id":147131376,"url":"https://api.github.com/repos/magento/magento2/labels/bug%20report","name":"bug report","color":"d93f0b","default":false},{"id":429635650,"url":"https://api.github.com/repos/magento/magento2/labels/develop","name":"develop","color":"fbca04","default":false},{"id":689434599,"url":"https://api.github.com/repos/magento/magento2/labels/G1%20Failed","name":"G1 Failed","color":"d93f0b","default":false},{"id":375320504,"url":"https://api.github.com/repos/magento/magento2/labels/Import/Export","name":"Import/Export","color":"1d76db","default":false}],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":11,"created_at":"2016-05-16T20:29:19Z","updated_at":"2017-09-20T21:42:40Z","closed_at":"2017-09-20T21:42:40Z","author_association":"CONTRIBUTOR","body":"## Steps to reproduce\n\n1) Install Magento from `2.0.2` tag (the one we tested on, but likely affects all versions).\n2) Create/import enough products so that when running an export, the main product query will have to use offsets to batch the results\n3) Look at the CSV and you will find duplicated product entries and some products that are missing altogether. From my testing, the total count is always correct and matches a SELECT COUN(*) FROM catalog_product_entity.\n\nI spent some time looking into what may be causing this, and I believe it has to do with the ORDER BY has_options clause in the MySQL query. What I'm not sure about, is why the Magento export orders by that field in the first place.\n\nWhen combining that ORDER BY with an OFFSET, the MySQL result order is inconsistent.\n\nThis can be observed by running the following queries against the Magento2 sample database.\n\n1) Query the first 10 products\n`SELECT entity_id, sku FROM catalog_product_entity AS e ORDER BY e.has_options ASC LIMIT 10;`\n\n 2) Query the first 20 products\n`SELECT entity_id, sku FROM catalog_product_entity AS e ORDER BY e.has_options ASC LIMIT 20;`\n\nIf you compare the result set... you'll see what I'm talking about. For me, the first row in query 1 has entity_id 9347. In query 2, the first row has entity_id 90.  Once you introduce offsetting the results, you get the same problem.\n\nMy proposed solution would be to order by entity_id in the export query, as that will always produce consistent result ordering, even with limits and offsets. This would require a change in Magento\\CatalogImportExport\\Model\\Export\\Product:: export\n\nHowever, without knowing why it's sorted by has_options in the first place, I can't be sure.\n"},"comment":{"url":"https://api.github.com/repos/magento/magento2/issues/comments/330989573","html_url":"https://github.com/magento/magento2/issues/4531#issuecomment-330989573","issue_url":"https://api.github.com/repos/magento/magento2/issues/4531","id":330989573,"user":{"login":"magento-engcom-team","id":31669971,"avatar_url":"https://avatars1.githubusercontent.com/u/31669971?v=4","gravatar_id":"","url":"https://api.github.com/users/magento-engcom-team","html_url":"https://github.com/magento-engcom-team","followers_url":"https://api.github.com/users/magento-engcom-team/followers","following_url":"https://api.github.com/users/magento-engcom-team/following{/other_user}","gists_url":"https://api.github.com/users/magento-engcom-team/gists{/gist_id}","starred_url":"https://api.github.com/users/magento-engcom-team/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/magento-engcom-team/subscriptions","organizations_url":"https://api.github.com/users/magento-engcom-team/orgs","repos_url":"https://api.github.com/users/magento-engcom-team/repos","events_url":"https://api.github.com/users/magento-engcom-team/events{/privacy}","received_events_url":"https://api.github.com/users/magento-engcom-team/received_events","type":"User","site_admin":false},"created_at":"2017-09-20T21:42:39Z","updated_at":"2017-09-20T21:42:39Z","author_association":"NONE","body":"@wbyrnetx thank you for your report.\r\nThe fix for this issue is already fixed in 2.2.0 release"}}
{"actor":{"display_login":"magento-engcom-team"}}