{"action":"opened","issue":{"url":"https://api.github.com/repos/DevExpress/testcafe/issues/1207","repository_url":"https://api.github.com/repos/DevExpress/testcafe","labels_url":"https://api.github.com/repos/DevExpress/testcafe/issues/1207/labels{/name}","comments_url":"https://api.github.com/repos/DevExpress/testcafe/issues/1207/comments","events_url":"https://api.github.com/repos/DevExpress/testcafe/issues/1207/events","html_url":"https://github.com/DevExpress/testcafe/issues/1207","id":204856346,"number":1207,"title":"Layout testing","user":{"login":"kirovboris","id":3633477,"avatar_url":"https://avatars.githubusercontent.com/u/3633477?v=3","gravatar_id":"","url":"https://api.github.com/users/kirovboris","html_url":"https://github.com/kirovboris","followers_url":"https://api.github.com/users/kirovboris/followers","following_url":"https://api.github.com/users/kirovboris/following{/other_user}","gists_url":"https://api.github.com/users/kirovboris/gists{/gist_id}","starred_url":"https://api.github.com/users/kirovboris/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/kirovboris/subscriptions","organizations_url":"https://api.github.com/users/kirovboris/orgs","repos_url":"https://api.github.com/users/kirovboris/repos","events_url":"https://api.github.com/users/kirovboris/events{/privacy}","received_events_url":"https://api.github.com/users/kirovboris/received_events","type":"User","site_admin":false},"labels":[{"id":266676019,"url":"https://api.github.com/repos/DevExpress/testcafe/labels/API%20MODIFICATION%20LEVEL:%20non-breaking%20changes","name":"API MODIFICATION LEVEL: non-breaking changes","color":"fbca04","default":false},{"id":267549313,"url":"https://api.github.com/repos/DevExpress/testcafe/labels/DOCUMENTATION:%20required","name":"DOCUMENTATION: required","color":"006b75","default":false},{"id":243973778,"url":"https://api.github.com/repos/DevExpress/testcafe/labels/SYSTEM:%20API","name":"SYSTEM: API","color":"bfe5bf","default":false},{"id":246571256,"url":"https://api.github.com/repos/DevExpress/testcafe/labels/TYPE:%20proposal","name":"TYPE: proposal","color":"f7c6c7","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2017-02-02T12:12:15Z","updated_at":"2017-02-02T12:12:15Z","closed_at":null,"body":"@DevExpress/testcafe  Let's discuss functionality of layout testing \r\n\r\nI suggest to use the following approach. Perhaps it looks too over engineered, but it is pretty flexible at my sight.\r\n\r\n### Screenshot provider\r\n\r\nI suggest to provide possibility to use any comparing screenshots library.\r\nWe do two options out-of-the-box - per-pixel comparison and perceptual hash comparison.\r\n\r\nTo avoid working with huge binary images we can calculate perceptual hash. \r\nIn this case we don't create screenshot file and compare only hash value.\r\nTo build the diff images use should run the local tests in per-pixel mode.\r\n\r\nAlso we can provide possibility of taking screenshot in black and white mode or ultra contrast mode. I think it can be passed to the chosen library as `layoutTestOptions`.\r\n\r\nSo we create:\r\n1. Pixel perfect provider with options\r\n```js\r\n{\r\n    mode: 'color' | 'black_and_white',\r\n    threshold: Number\t\r\n}\r\n```\r\n\r\n2. Perceptual hash with options\r\n```js\r\n{\r\n    threshold: Number \r\n    // or\r\n    exactMatching: true | false \r\n    //it will be depend of implementation\r\n}\r\n```\r\n\r\n### Run options\r\nFor the `run` function we add options for screenshot comparing:\r\n\r\n```js\r\n{\r\n    etalonsPath: <relative_or_absolute_path>,\r\n    layoutTestProvider: <provider_name>,\r\n    layoutTestOptions: {...},\r\n    updateEtalons: true | false\r\n}\r\n```\r\n\r\nAccordingly, we add the similar options to the CLI interface:\r\n```--etalons-path```, ```--layout-provider```, ```--update-etalons```.\r\n\r\n### Test API\r\nWe provide two different ways to testing layout.\r\n1. We look for an etalon automatically and user should just call ```t.checkLayout()``` method.\r\n\r\n`checkLayout()` is searching for a screenshot for test in according to index of using in the test code body, starting with `0`.\r\n\r\n```js\r\ntest('testName', async t => {\r\n    await t\r\n        .click('#el')\r\n        .checkLayout() //looking for .../<testName>/<workerName>/1.png        \r\n        .type('.inputClass', 'value')\r\n        .checkLayout() //looking for .../<testName>/<workerName>/2.png\r\n        ......\r\n        .click(...)\r\n        .checkLayout() //looking for .../<testName>/<workerName>/<N>.png\r\n});\r\n```\r\n\r\n2. We provide `testControler.Image` constructor and `.equalLayout()` assertion.\r\nThat means user should decide on their own how to store artifacts and etalons, we just use comparing logic from provider.\r\n\r\nE.g.:\r\n```js\r\n test('testName', async t => {\r\n     await t.expect(new t.Image(<image_path>)).equalLayout();\r\n });\r\n```\r\n\r\nAlso we should resize browser window up to screenshot size.\r\n\r\n### Screenshots storage\r\nEvery provider implements mechanism of storing artifacts.\r\nFor per-pixel provider we store screenshots similarly with current screenshots directory. Also we should create difference files in that directory.\r\n\r\nEtalons will be got from path that specified in `etalonsPath` for programmatically API or from `--etalons-path` parameter for CLI.\r\nFor hash comparison we write key-value pairs to the `.json` file.\r\n```js\r\n{\r\n    '<testName1><workerName1><etalonId1>': '3c3e0e1a3a1e1e2e',\r\n    ...\r\n    '<testNameN><workerNameN><etalonIdN>': 'ac3e0e1a3a1e1e2F'\r\n}\r\n```\r\n\r\n### Updating of etalons\r\nAs soon as the first screenshot will be different from etalon we run web application with GUI interface for managing etalons.\r\nService will run only if was passed `--update-etalons` to CLI or `updateEtalons` to the `runOptions` in programmatically API.\r\nIn this case we output in report only path to the difference file.\r\n\r\nAs alternative we can just output paths to the artifacts, etalons and difference files, but it does not look convenient.\r\n\r\n\r\n\r\n\r\n"}}
{"actor":{"display_login":"kirovboris"}}