{"action":"created","issue":{"url":"https://api.github.com/repos/weaveworks/flux/issues/438","repository_url":"https://api.github.com/repos/weaveworks/flux","labels_url":"https://api.github.com/repos/weaveworks/flux/issues/438/labels{/name}","comments_url":"https://api.github.com/repos/weaveworks/flux/issues/438/comments","events_url":"https://api.github.com/repos/weaveworks/flux/issues/438/events","html_url":"https://github.com/weaveworks/flux/issues/438","id":206835049,"number":438,"title":"Separate Release records and Jobs","user":{"login":"squaremo","id":62033,"avatar_url":"https://avatars.githubusercontent.com/u/62033?v=3","gravatar_id":"","url":"https://api.github.com/users/squaremo","html_url":"https://github.com/squaremo","followers_url":"https://api.github.com/users/squaremo/followers","following_url":"https://api.github.com/users/squaremo/following{/other_user}","gists_url":"https://api.github.com/users/squaremo/gists{/gist_id}","starred_url":"https://api.github.com/users/squaremo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/squaremo/subscriptions","organizations_url":"https://api.github.com/users/squaremo/orgs","repos_url":"https://api.github.com/users/squaremo/repos","events_url":"https://api.github.com/users/squaremo/events{/privacy}","received_events_url":"https://api.github.com/users/squaremo/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":1,"created_at":"2017-02-10T16:06:02Z","updated_at":"2017-02-10T16:23:23Z","closed_at":null,"body":"The `Job` table holds ephemeral records of different kinds of job to be run, and currently includes (in release jobs) information about releases.\r\n\r\nWe want the records of releases to be more persistent, though; and, to be able to store information about the result of a release, for instance, which is not represented in the jobs table.\r\n\r\nThe Job table needs all of the things necessary for selecting, dispatching, and (potentially) recovering jobs:\r\n\r\n```\r\nJob\r\n  ID -- PK\r\n  Instance -- multitenancy\r\n\r\n  Queue  -- )\r\n  Method -- ) dispatch\r\n  Params -- )\r\n\r\n  ScheduledAt -- queuing (delayed jobs)\r\n  Priority -- queueing\r\n  Key -- queueing (deduplication)\r\n\r\n  Claimed --   ) healthcheck / recovery\r\n  Heartbeat -- ) \r\n\r\n  Finished -- ) GC and reporting\r\n  Error --    )\r\n```\r\n\r\nThe Release table records everything about a release:\r\n```\r\nRelease\r\n  ID -- PK\r\n  Instance -- multitenancy\r\n  Spec -- ReleaseSpec\r\n  Log -- possibly incomplete log\r\n  Result -- possibly incomplete result\r\n  StartedAt --  ) for audit\r\n  FinishedAt -- )\r\n```\r\n\r\nThere is a chicken and egg problem: does the job just update the release, or does the job create the release? (Alternatively put, if I want to check the status of a release, where do I check?)\r\n\r\nI am inclined to make it the job of the .. er, job .. to create the release. So, if you're checking its progress, you\r\n\r\n 1. look for the release record and look for the job record;\r\n 2. if a release record and no job record, it has been completed\r\n 3. if a job record and no release record, it has not been started yet\r\n 4. if both, it's in progress\r\n 5. if neither, there's no such release\r\n"},"comment":{"url":"https://api.github.com/repos/weaveworks/flux/issues/comments/278989503","html_url":"https://github.com/weaveworks/flux/issues/438#issuecomment-278989503","issue_url":"https://api.github.com/repos/weaveworks/flux/issues/438","id":278989503,"user":{"login":"squaremo","id":62033,"avatar_url":"https://avatars.githubusercontent.com/u/62033?v=3","gravatar_id":"","url":"https://api.github.com/users/squaremo","html_url":"https://github.com/squaremo","followers_url":"https://api.github.com/users/squaremo/followers","following_url":"https://api.github.com/users/squaremo/following{/other_user}","gists_url":"https://api.github.com/users/squaremo/gists{/gist_id}","starred_url":"https://api.github.com/users/squaremo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/squaremo/subscriptions","organizations_url":"https://api.github.com/users/squaremo/orgs","repos_url":"https://api.github.com/users/squaremo/repos","events_url":"https://api.github.com/users/squaremo/events{/privacy}","received_events_url":"https://api.github.com/users/squaremo/received_events","type":"User","site_admin":false},"created_at":"2017-02-10T16:23:22Z","updated_at":"2017-02-10T16:23:22Z","body":"Yeah, that's the other way around. That alternative has its own trickinesses:\r\n\r\n - if the job doesn't run for whatever reason, you end up with orphaned, never-to-be-completed release records. I guess these can be excluded in queries, and GCed if never started.\r\n - we won't be able to rely on deduplication (of jobs) to avoid repeating work\r\n\r\nNeither of these are insurmountable of course."}}
{"actor":{"display_login":"squaremo"}}