{"action":"created","issue":{"url":"https://api.github.com/repos/opencontainers/image-spec/issues/637","repository_url":"https://api.github.com/repos/opencontainers/image-spec","labels_url":"https://api.github.com/repos/opencontainers/image-spec/issues/637/labels{/name}","comments_url":"https://api.github.com/repos/opencontainers/image-spec/issues/637/comments","events_url":"https://api.github.com/repos/opencontainers/image-spec/issues/637/events","html_url":"https://github.com/opencontainers/image-spec/issues/637","id":219377368,"number":637,"title":"Golden master testsuite","user":{"login":"stevvooe","id":120601,"avatar_url":"https://avatars2.githubusercontent.com/u/120601?v=3","gravatar_id":"","url":"https://api.github.com/users/stevvooe","html_url":"https://github.com/stevvooe","followers_url":"https://api.github.com/users/stevvooe/followers","following_url":"https://api.github.com/users/stevvooe/following{/other_user}","gists_url":"https://api.github.com/users/stevvooe/gists{/gist_id}","starred_url":"https://api.github.com/users/stevvooe/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/stevvooe/subscriptions","organizations_url":"https://api.github.com/users/stevvooe/orgs","repos_url":"https://api.github.com/users/stevvooe/repos","events_url":"https://api.github.com/users/stevvooe/events{/privacy}","received_events_url":"https://api.github.com/users/stevvooe/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":{"url":"https://api.github.com/repos/opencontainers/image-spec/milestones/3","html_url":"https://github.com/opencontainers/image-spec/milestone/3","labels_url":"https://api.github.com/repos/opencontainers/image-spec/milestones/3/labels","id":1705177,"number":3,"title":"v1.0.0","description":"Release initial version of spec with two independent non-experimental implementations from OCI members\r\n","creator":{"login":"philips","id":33544,"avatar_url":"https://avatars2.githubusercontent.com/u/33544?v=3","gravatar_id":"","url":"https://api.github.com/users/philips","html_url":"https://github.com/philips","followers_url":"https://api.github.com/users/philips/followers","following_url":"https://api.github.com/users/philips/following{/other_user}","gists_url":"https://api.github.com/users/philips/gists{/gist_id}","starred_url":"https://api.github.com/users/philips/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/philips/subscriptions","organizations_url":"https://api.github.com/users/philips/orgs","repos_url":"https://api.github.com/users/philips/repos","events_url":"https://api.github.com/users/philips/events{/privacy}","received_events_url":"https://api.github.com/users/philips/received_events","type":"User","site_admin":false},"open_issues":3,"closed_issues":15,"state":"open","created_at":"2016-04-14T04:42:50Z","updated_at":"2017-04-26T15:44:01Z","due_on":null,"closed_at":null},"comments":8,"created_at":"2017-04-04T20:45:07Z","updated_at":"2017-04-28T20:44:44Z","closed_at":null,"body":"After seeing issues like https://github.com/opencontainers/image-spec/issues/626, it seems wise to create a golden master testsuite that contains known valid data structures and ensures that we parse and generate them correctly. Such a test suite should be done outside of the scope tools like `gojsonschema` to ensure they are operating correctly."},"comment":{"url":"https://api.github.com/repos/opencontainers/image-spec/issues/comments/298102189","html_url":"https://github.com/opencontainers/image-spec/issues/637#issuecomment-298102189","issue_url":"https://api.github.com/repos/opencontainers/image-spec/issues/637","id":298102189,"user":{"login":"stevvooe","id":120601,"avatar_url":"https://avatars2.githubusercontent.com/u/120601?v=3","gravatar_id":"","url":"https://api.github.com/users/stevvooe","html_url":"https://github.com/stevvooe","followers_url":"https://api.github.com/users/stevvooe/followers","following_url":"https://api.github.com/users/stevvooe/following{/other_user}","gists_url":"https://api.github.com/users/stevvooe/gists{/gist_id}","starred_url":"https://api.github.com/users/stevvooe/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/stevvooe/subscriptions","organizations_url":"https://api.github.com/users/stevvooe/orgs","repos_url":"https://api.github.com/users/stevvooe/repos","events_url":"https://api.github.com/users/stevvooe/events{/privacy}","received_events_url":"https://api.github.com/users/stevvooe/received_events","type":"User","site_admin":false},"created_at":"2017-04-28T20:44:44Z","updated_at":"2017-04-28T20:44:44Z","body":"For the most part, we don't really need to validate fully baked image layouts. This effort should focus on getting masters together for the individual components. At this point, we've seen several reports of various problems that are internal to each data type, so we can keep the scope here relatively small.\r\n\r\n> the field generation/processing could be provided either with direct code or text/template generation, not entirely sure how that would work yet, but the text/template idea was basically to allow restricted generation (e.g., with a fixed name so you always get your result with it in there).\r\n\r\nThe idea of a golden master avoids any generation.\r\n\r\n> Validation can be performed trivially by gojsonschema I think\r\n\r\nAgain, we need to go around gojsonschema: we can see that it is not actually working or performing validation in all cases. The idea of a golden master verifies gojsonschema, as well.\r\n\r\nYou basically have a test like this:\r\n\r\n```go\r\nfunc TestGolden(t *testing.T) {\r\n  for _, testcase := range []struct{\r\n    name string\r\n    input interface{}\r\n    expected string // can be a literal or a path\r\n  }{\r\n  name: \"MyCase\",\r\n  input: ocispec.Manifest{...},\r\n  expected: \r\n}{\r\n    t.Run(testcase.name, func(t *testing.T) {\r\n      p, err := json.Marshal(testcase.input)\r\n      if err != nil { t.Fatal(err) }\r\n\r\n      // note that it is okay to reindent the expected for consistency\r\n      if !bytes.Equal(p, testcase.expected) { // check marshaling\r\n        t.Fatal(...) // show a diff\r\n      }\r\n\r\n      // check we got back what was expected\r\n      roundtrip := reflect.Zero(reflect.TypeOf(testcase.input)).Interface() // i think this is right\r\n      if err := json.Unmarshal(testcase.expected, roundtrip); err != nil { t.Fatal(err) }\r\n      if !reflect.DeepEqual(roundtrip, testcase.input) { t.Fatal(...) }\r\n  }})\r\n}\r\n```"}}
{"actor":{"display_login":"stevvooe"}}