{"action":"created","issue":{"url":"https://api.github.com/repos/owncloud/core/issues/28253","repository_url":"https://api.github.com/repos/owncloud/core","labels_url":"https://api.github.com/repos/owncloud/core/issues/28253/labels{/name}","comments_url":"https://api.github.com/repos/owncloud/core/issues/28253/comments","events_url":"https://api.github.com/repos/owncloud/core/issues/28253/events","html_url":"https://github.com/owncloud/core/pull/28253","id":239230434,"number":28253,"title":"Add repair step to repair mismatch filecache paths","user":{"login":"PVince81","id":277525,"avatar_url":"https://avatars1.githubusercontent.com/u/277525?v=3","gravatar_id":"","url":"https://api.github.com/users/PVince81","html_url":"https://github.com/PVince81","followers_url":"https://api.github.com/users/PVince81/followers","following_url":"https://api.github.com/users/PVince81/following{/other_user}","gists_url":"https://api.github.com/users/PVince81/gists{/gist_id}","starred_url":"https://api.github.com/users/PVince81/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/PVince81/subscriptions","organizations_url":"https://api.github.com/users/PVince81/orgs","repos_url":"https://api.github.com/users/PVince81/repos","events_url":"https://api.github.com/users/PVince81/events{/privacy}","received_events_url":"https://api.github.com/users/PVince81/received_events","type":"User","site_admin":false},"labels":[{"id":17709833,"url":"https://api.github.com/repos/owncloud/core/labels/2%20-%20Developing","name":"2 - Developing","color":"ededed","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":{"url":"https://api.github.com/repos/owncloud/core/milestones/114","html_url":"https://github.com/owncloud/core/milestone/114","labels_url":"https://api.github.com/repos/owncloud/core/milestones/114/labels","id":2544592,"number":114,"title":"development","description":"10.0.3","creator":{"login":"PVince81","id":277525,"avatar_url":"https://avatars1.githubusercontent.com/u/277525?v=3","gravatar_id":"","url":"https://api.github.com/users/PVince81","html_url":"https://github.com/PVince81","followers_url":"https://api.github.com/users/PVince81/followers","following_url":"https://api.github.com/users/PVince81/following{/other_user}","gists_url":"https://api.github.com/users/PVince81/gists{/gist_id}","starred_url":"https://api.github.com/users/PVince81/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/PVince81/subscriptions","organizations_url":"https://api.github.com/users/PVince81/orgs","repos_url":"https://api.github.com/users/PVince81/repos","events_url":"https://api.github.com/users/PVince81/events{/privacy}","received_events_url":"https://api.github.com/users/PVince81/received_events","type":"User","site_admin":false},"open_issues":66,"closed_issues":100,"state":"open","created_at":"2017-05-26T08:45:21Z","updated_at":"2017-07-11T15:10:53Z","due_on":null,"closed_at":null},"comments":89,"created_at":"2017-06-28T16:59:59Z","updated_at":"2017-07-11T19:34:55Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/owncloud/core/pulls/28253","html_url":"https://github.com/owncloud/core/pull/28253","diff_url":"https://github.com/owncloud/core/pull/28253.diff","patch_url":"https://github.com/owncloud/core/pull/28253.patch"},"body":"## Description\r\nWhenever a parent-child relationship describes a specific path but the\r\nentry's actual path is different, this new repair step will adjust it.\r\n\r\nIn the event where another entry already exists under that name, the\r\nlatter is deleted and replaced by the above entry. This should help\r\npreserve the metadata associated with the original entry.\r\n\r\nThis kind of situation can be create through the bug from https://github.com/owncloud/core/issues/28018\r\n\r\nThis is a better fix than https://github.com/owncloud/core/pull/28217\r\n\r\nThe cases that this repair step fixes:\r\n- current path doesn't match the one from the parent on same storage: reproducible before https://github.com/owncloud/core/pull/28022 by moving a file between two received shares from the same user or by moving a file out of a received share.\r\n- current path doesn't match the one from the parent on a different storage: reproducible before https://github.com/owncloud/core/pull/28022 by moving a file between two received shares where each share comes from a different user (two storages)\r\n- parentid pointing to a non-existing entry: not reproducible in the wild, however the first phase of the repair step will create such entries which the second phase fixes. I assume that there might be such cases existing out there so this would fix them too.\r\n- `parent = fileid`: not reproducible at all, was reported by @butonic as seen in the wild (clustered env). I added it because such entries would have caused an infinite loop in the repair step which is bad.\r\n\r\nThis repair step **does not**:\r\n- repair mismatch path_hashes\r\n- mime types\r\n\r\nHere are the queries that are run to find broken entries, could be used to pre-check an instance:\r\n\r\n```sql\r\nselect fc.storage,fc.fileid,fc.parent as \"wrongparent\",fc3.fileid \"correctparent\",fc.path,fc.etag from oc_filecache fc, oc_filecache fc3 where fc.parent <> -1 and fc.parent != fc3.fileid and fc3.storage=fc.storage and fc3.path = substring(fc.path, 1, length(fc.path) - length(substring_index(fc.path, '/', -1)) - 1) and not exists (select 1 from oc_filecache fc2 where fc2.fileid = fc.parent) order by path;\r\nSELECT fc.`storage`, fc.`fileid`, fc.`path` as `path`, fc.`name`, fcp.`storage` as `parentstorage`, fcp.`path` as `parentpath`     FROM `oc_filecache` fc, `oc_filecache` fcp     WHERE (fc.`parent` = fcp.`fileid`)     AND (         (CONCAT(fcp.`path`, '/', fc.`name`) <> fc.`path`)         OR (fc.`storage` <> fcp.`storage`)     )     AND (fcp.`path` <> '')     AND (fc.`fileid` <> fcp.`fileid`);\r\nselect fc.storage,fc.fileid,fc.parent as \"wrongparent\",fc.path,fc.etag from oc_filecache fc where fc.parent <> -1 and not exists (select 1 from oc_filecache fc2 where fc2.fileid = fc.parent) order by path;\r\nselect * from oc_filecache where parent=fileid;\r\n```\r\n\r\n## Related Issue\r\nFixes the fallout created by https://github.com/owncloud/core/issues/28018\r\n\r\n## Motivation and Context\r\nBecause we hate filecache inconsistencies.\r\n\r\n## How Has This Been Tested?\r\n- [x] TEST: Unit test (some pending)\r\n- [x] TEST: manual test\r\n\r\n⚠️ Use the test steps from here: https://github.com/owncloud/core/pull/28253#issuecomment-311781465\r\n\r\n## Screenshots (if appropriate):\r\n\r\n## Types of changes\r\n<!--- What types of changes does your code introduce? Put an `x` in all the boxes that apply: -->\r\n- [x] Bug fix (non-breaking change which fixes an issue)\r\n- [ ] New feature (non-breaking change which adds functionality)\r\n- [ ] Breaking change (fix or feature that would cause existing functionality to change)\r\n\r\n## Checklist:\r\n<!--- Go over all the following points, and put an `x` in all the boxes that apply. -->\r\n<!--- If you're unsure about any of these, don't hesitate to ask. We're here to help! -->\r\n- [x] My code follows the code style of this project.\r\n- [ ] My change requires a change to the documentation.\r\n- [ ] I have updated the documentation accordingly.\r\n- [ ] I have read the **CONTRIBUTING** document.\r\n- [x] I have added tests to cover my changes.\r\n- [x] All new and existing tests passed.\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/owncloud/core/issues/comments/314548783","html_url":"https://github.com/owncloud/core/pull/28253#issuecomment-314548783","issue_url":"https://api.github.com/repos/owncloud/core/issues/28253","id":314548783,"user":{"login":"PVince81","id":277525,"avatar_url":"https://avatars1.githubusercontent.com/u/277525?v=3","gravatar_id":"","url":"https://api.github.com/users/PVince81","html_url":"https://github.com/PVince81","followers_url":"https://api.github.com/users/PVince81/followers","following_url":"https://api.github.com/users/PVince81/following{/other_user}","gists_url":"https://api.github.com/users/PVince81/gists{/gist_id}","starred_url":"https://api.github.com/users/PVince81/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/PVince81/subscriptions","organizations_url":"https://api.github.com/users/PVince81/orgs","repos_url":"https://api.github.com/users/PVince81/repos","events_url":"https://api.github.com/users/PVince81/events{/privacy}","received_events_url":"https://api.github.com/users/PVince81/received_events","type":"User","site_admin":false},"created_at":"2017-07-11T19:34:55Z","updated_at":"2017-07-11T19:34:55Z","body":"Anoher local test, on my laptop with SSD while on battery:\r\n\r\n- 746980 file cache entries, from which 186744 were purposefully broken.\r\n- first phase of repair took almost 11 minutes (652 seconds)\r\n- second phase of repair took 13 seconds\r\n\r\nDue to a bug in my setup I couldn't run `occ files:scan` to complete it.\r\nNeeds to retest.\r\n\r\nIn general I have the feeling that we might not need to split the command. However adding a list of storages to rescan at the end could be useful.\r\n"}}
{"actor":{"display_login":"PVince81"}}