{"action":"created","comment":{"url":"https://api.github.com/repos/NUbots/NUsight2/pulls/comments/128496276","pull_request_review_id":51179762,"id":128496276,"diff_hunk":"@@ -0,0 +1,60 @@\n+import { ReadStream } from 'fs'\n+import * as stream from 'stream'\n+import { NUClearNetClient } from '../../shared/nuclearnet/nuclearnet_client'\n+import { Clock } from '../time/clock'\n+import { NodeSystemClock } from '../time/node_clock'\n+import { NbsFrameChunker } from './nbs_frame_chunker'\n+import { NbsFrame } from './nbs_frame_codecs'\n+import { NbsFrameDecoder } from './nbs_frame_streams'\n+import * as fs from 'fs'\n+import { createGunzip } from 'zlib'\n+import { PassThrough } from 'stream'\n+\n+export class NbsNUClearPlayback extends stream.Writable {\n+  private firstFrameTimestamp?: number\n+  private firstLocalTimestamp?: number\n+\n+  public constructor(private nuclearnetClient: NUClearNetClient,\n+                     private clock: Clock) {\n+    super({\n+      objectMode: true,\n+    })\n+  }\n+\n+  public static of(nuclearnetClient: NUClearNetClient) {\n+    return new NbsNUClearPlayback(nuclearnetClient, NodeSystemClock)\n+  }\n+\n+  public static fromFile(filename: string, nuclearnetClient: NUClearNetClient) {\n+    const playback = NbsNUClearPlayback.of(nuclearnetClient)\n+    let rawStream = fs.createReadStream(filename)\n+    const decompress = filename.endsWith('.nbz') ? createGunzip() : new PassThrough()","path":"src/server/nbs/nbs_nuclear_playback.ts","position":31,"original_position":31,"commit_id":"d2fe72aec172bc11efd1db224d633dd4797d7dd4","original_commit_id":"cfc090967e423d2b741b393643ffe0355bdac591","user":{"login":"TrentHouliston","id":957962,"avatar_url":"https://avatars1.githubusercontent.com/u/957962?v=4","gravatar_id":"","url":"https://api.github.com/users/TrentHouliston","html_url":"https://github.com/TrentHouliston","followers_url":"https://api.github.com/users/TrentHouliston/followers","following_url":"https://api.github.com/users/TrentHouliston/following{/other_user}","gists_url":"https://api.github.com/users/TrentHouliston/gists{/gist_id}","starred_url":"https://api.github.com/users/TrentHouliston/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/TrentHouliston/subscriptions","organizations_url":"https://api.github.com/users/TrentHouliston/orgs","repos_url":"https://api.github.com/users/TrentHouliston/repos","events_url":"https://api.github.com/users/TrentHouliston/events{/privacy}","received_events_url":"https://api.github.com/users/TrentHouliston/received_events","type":"User","site_admin":false},"body":"This should be ends with .nbz or .nbs.gz both are valid extensions","created_at":"2017-07-20T12:07:25Z","updated_at":"2017-07-20T12:10:14Z","html_url":"https://github.com/NUbots/NUsight2/pull/97#discussion_r128496276","pull_request_url":"https://api.github.com/repos/NUbots/NUsight2/pulls/97","_links":{"self":{"href":"https://api.github.com/repos/NUbots/NUsight2/pulls/comments/128496276"},"html":{"href":"https://github.com/NUbots/NUsight2/pull/97#discussion_r128496276"},"pull_request":{"href":"https://api.github.com/repos/NUbots/NUsight2/pulls/97"}}},"pull_request":{"url":"https://api.github.com/repos/NUbots/NUsight2/pulls/97","id":131535709,"html_url":"https://github.com/NUbots/NUsight2/pull/97","diff_url":"https://github.com/NUbots/NUsight2/pull/97.diff","patch_url":"https://github.com/NUbots/NUsight2/pull/97.patch","issue_url":"https://api.github.com/repos/NUbots/NUsight2/issues/97","number":97,"state":"open","locked":false,"title":"Add NBS streaming primitives","user":{"login":"BrendanAnnable","id":132620,"avatar_url":"https://avatars0.githubusercontent.com/u/132620?v=4","gravatar_id":"","url":"https://api.github.com/users/BrendanAnnable","html_url":"https://github.com/BrendanAnnable","followers_url":"https://api.github.com/users/BrendanAnnable/followers","following_url":"https://api.github.com/users/BrendanAnnable/following{/other_user}","gists_url":"https://api.github.com/users/BrendanAnnable/gists{/gist_id}","starred_url":"https://api.github.com/users/BrendanAnnable/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/BrendanAnnable/subscriptions","organizations_url":"https://api.github.com/users/BrendanAnnable/orgs","repos_url":"https://api.github.com/users/BrendanAnnable/repos","events_url":"https://api.github.com/users/BrendanAnnable/events{/privacy}","received_events_url":"https://api.github.com/users/BrendanAnnable/received_events","type":"User","site_admin":false},"body":"This PR adds the building blocks for adding recording and playback. The frontend UI and integration with these classes will come later.\r\n\r\nFirstly there is the core `encodeFrame` and `decodeFrame` functions. These encode and decode NBS frames/buffers. They are raw simply raw functions which pack/unpack the data to/from a binary format.\r\n\r\nThe rest of the classes are sugar designed to make it easy to use these functions in practical ways. They are written using node streams, which allow us to easily pipe data around, such as piping a gzipped `.nbz` file straight onto the NUClear network.\r\n\r\nOverall this is how it works:\r\n\r\n`FileStream` -> `NbsFrameChunker` -> `NbsFrameDecoder` -> `NbsNUClearPlayback` -> `NUClearNetClient` -> `Browser and/or Robot`.\r\n\r\n`NbsFrameChunker` — The entire job of this stream tranformer is to find the nbs header (0xe298a2) and read just enough to know its size, and then form nbs-packet sized buffer chunks. In other words it decodes the framing protocol and emits those frames in the form of buffers.\r\n\r\n`NbsFrameDecoder` — This is the next step, actually decoding that framed buffer into a JavaScript object with readable properties. There is also its partner `NbsFrameDecoder`, which should let us stream those same JavaScript objects to a file.\r\n\r\n`NbsNUClearPlayback` — The job of this class is to take the JavaScript frames and put them on the network. It also streams them at the same rate they were recorded. i.e. it reads the `timestampInMicroseconds` property and emits packets at the appropriate time to simulate playback.\r\n\r\nNow just look at those node streams in action 😍 :\r\n\r\n```\r\n  public static fromFile(filename: string, nuclearnetClient: NUClearNetClient) {\r\n    const playback = NbsNUClearPlayback.of(nuclearnetClient)\r\n    let rawStream = fs.createReadStream(filename)\r\n    const decompress = filename.endsWith('.nbz') ? createGunzip() : new PassThrough()\r\n    rawStream.pipe(decompress).pipe(new NbsFrameChunker()).pipe(new NbsFrameDecoder()).pipe(playback)\r\n    return playback\r\n  }\r\n```","created_at":"2017-07-20T11:55:37Z","updated_at":"2017-07-20T12:10:14Z","closed_at":null,"merged_at":null,"merge_commit_sha":"a382cedec2ea280ae9b165ab24f7f978e2c99968","assignee":null,"assignees":[],"requested_reviewers":[],"milestone":null,"commits_url":"https://api.github.com/repos/NUbots/NUsight2/pulls/97/commits","review_comments_url":"https://api.github.com/repos/NUbots/NUsight2/pulls/97/comments","review_comment_url":"https://api.github.com/repos/NUbots/NUsight2/pulls/comments{/number}","comments_url":"https://api.github.com/repos/NUbots/NUsight2/issues/97/comments","statuses_url":"https://api.github.com/repos/NUbots/NUsight2/statuses/d2fe72aec172bc11efd1db224d633dd4797d7dd4","head":{"label":"NUbots:annable/nbs-streams","ref":"annable/nbs-streams","sha":"d2fe72aec172bc11efd1db224d633dd4797d7dd4","user":{"login":"NUbots","id":150660,"avatar_url":"https://avatars1.githubusercontent.com/u/150660?v=4","gravatar_id":"","url":"https://api.github.com/users/NUbots","html_url":"https://github.com/NUbots","followers_url":"https://api.github.com/users/NUbots/followers","following_url":"https://api.github.com/users/NUbots/following{/other_user}","gists_url":"https://api.github.com/users/NUbots/gists{/gist_id}","starred_url":"https://api.github.com/users/NUbots/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/NUbots/subscriptions","organizations_url":"https://api.github.com/users/NUbots/orgs","repos_url":"https://api.github.com/users/NUbots/repos","events_url":"https://api.github.com/users/NUbots/events{/privacy}","received_events_url":"https://api.github.com/users/NUbots/received_events","type":"Organization","site_admin":false},"repo":{"id":91078846,"name":"NUsight2","full_name":"NUbots/NUsight2","owner":{"login":"NUbots","id":150660,"avatar_url":"https://avatars1.githubusercontent.com/u/150660?v=4","gravatar_id":"","url":"https://api.github.com/users/NUbots","html_url":"https://github.com/NUbots","followers_url":"https://api.github.com/users/NUbots/followers","following_url":"https://api.github.com/users/NUbots/following{/other_user}","gists_url":"https://api.github.com/users/NUbots/gists{/gist_id}","starred_url":"https://api.github.com/users/NUbots/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/NUbots/subscriptions","organizations_url":"https://api.github.com/users/NUbots/orgs","repos_url":"https://api.github.com/users/NUbots/repos","events_url":"https://api.github.com/users/NUbots/events{/privacy}","received_events_url":"https://api.github.com/users/NUbots/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/NUbots/NUsight2","description":"A visual real-time web-based debugging environment used to understand and improve robotics systems.","fork":false,"url":"https://api.github.com/repos/NUbots/NUsight2","forks_url":"https://api.github.com/repos/NUbots/NUsight2/forks","keys_url":"https://api.github.com/repos/NUbots/NUsight2/keys{/key_id}","collaborators_url":"https://api.github.com/repos/NUbots/NUsight2/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/NUbots/NUsight2/teams","hooks_url":"https://api.github.com/repos/NUbots/NUsight2/hooks","issue_events_url":"https://api.github.com/repos/NUbots/NUsight2/issues/events{/number}","events_url":"https://api.github.com/repos/NUbots/NUsight2/events","assignees_url":"https://api.github.com/repos/NUbots/NUsight2/assignees{/user}","branches_url":"https://api.github.com/repos/NUbots/NUsight2/branches{/branch}","tags_url":"https://api.github.com/repos/NUbots/NUsight2/tags","blobs_url":"https://api.github.com/repos/NUbots/NUsight2/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/NUbots/NUsight2/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/NUbots/NUsight2/git/refs{/sha}","trees_url":"https://api.github.com/repos/NUbots/NUsight2/git/trees{/sha}","statuses_url":"https://api.github.com/repos/NUbots/NUsight2/statuses/{sha}","languages_url":"https://api.github.com/repos/NUbots/NUsight2/languages","stargazers_url":"https://api.github.com/repos/NUbots/NUsight2/stargazers","contributors_url":"https://api.github.com/repos/NUbots/NUsight2/contributors","subscribers_url":"https://api.github.com/repos/NUbots/NUsight2/subscribers","subscription_url":"https://api.github.com/repos/NUbots/NUsight2/subscription","commits_url":"https://api.github.com/repos/NUbots/NUsight2/commits{/sha}","git_commits_url":"https://api.github.com/repos/NUbots/NUsight2/git/commits{/sha}","comments_url":"https://api.github.com/repos/NUbots/NUsight2/comments{/number}","issue_comment_url":"https://api.github.com/repos/NUbots/NUsight2/issues/comments{/number}","contents_url":"https://api.github.com/repos/NUbots/NUsight2/contents/{+path}","compare_url":"https://api.github.com/repos/NUbots/NUsight2/compare/{base}...{head}","merges_url":"https://api.github.com/repos/NUbots/NUsight2/merges","archive_url":"https://api.github.com/repos/NUbots/NUsight2/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/NUbots/NUsight2/downloads","issues_url":"https://api.github.com/repos/NUbots/NUsight2/issues{/number}","pulls_url":"https://api.github.com/repos/NUbots/NUsight2/pulls{/number}","milestones_url":"https://api.github.com/repos/NUbots/NUsight2/milestones{/number}","notifications_url":"https://api.github.com/repos/NUbots/NUsight2/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/NUbots/NUsight2/labels{/name}","releases_url":"https://api.github.com/repos/NUbots/NUsight2/releases{/id}","deployments_url":"https://api.github.com/repos/NUbots/NUsight2/deployments","created_at":"2017-05-12T10:10:59Z","updated_at":"2017-07-20T03:33:12Z","pushed_at":"2017-07-20T12:08:06Z","git_url":"git://github.com/NUbots/NUsight2.git","ssh_url":"git@github.com:NUbots/NUsight2.git","clone_url":"https://github.com/NUbots/NUsight2.git","svn_url":"https://github.com/NUbots/NUsight2","homepage":"https://nusight.herokuapp.com","size":1347,"stargazers_count":6,"watchers_count":6,"language":"JavaScript","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":13,"forks":0,"open_issues":13,"watchers":6,"default_branch":"master"}},"base":{"label":"NUbots:master","ref":"master","sha":"e50d7ca81ed6ad62b3a5892417f43cd239fffd95","user":{"login":"NUbots","id":150660,"avatar_url":"https://avatars1.githubusercontent.com/u/150660?v=4","gravatar_id":"","url":"https://api.github.com/users/NUbots","html_url":"https://github.com/NUbots","followers_url":"https://api.github.com/users/NUbots/followers","following_url":"https://api.github.com/users/NUbots/following{/other_user}","gists_url":"https://api.github.com/users/NUbots/gists{/gist_id}","starred_url":"https://api.github.com/users/NUbots/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/NUbots/subscriptions","organizations_url":"https://api.github.com/users/NUbots/orgs","repos_url":"https://api.github.com/users/NUbots/repos","events_url":"https://api.github.com/users/NUbots/events{/privacy}","received_events_url":"https://api.github.com/users/NUbots/received_events","type":"Organization","site_admin":false},"repo":{"id":91078846,"name":"NUsight2","full_name":"NUbots/NUsight2","owner":{"login":"NUbots","id":150660,"avatar_url":"https://avatars1.githubusercontent.com/u/150660?v=4","gravatar_id":"","url":"https://api.github.com/users/NUbots","html_url":"https://github.com/NUbots","followers_url":"https://api.github.com/users/NUbots/followers","following_url":"https://api.github.com/users/NUbots/following{/other_user}","gists_url":"https://api.github.com/users/NUbots/gists{/gist_id}","starred_url":"https://api.github.com/users/NUbots/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/NUbots/subscriptions","organizations_url":"https://api.github.com/users/NUbots/orgs","repos_url":"https://api.github.com/users/NUbots/repos","events_url":"https://api.github.com/users/NUbots/events{/privacy}","received_events_url":"https://api.github.com/users/NUbots/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/NUbots/NUsight2","description":"A visual real-time web-based debugging environment used to understand and improve robotics systems.","fork":false,"url":"https://api.github.com/repos/NUbots/NUsight2","forks_url":"https://api.github.com/repos/NUbots/NUsight2/forks","keys_url":"https://api.github.com/repos/NUbots/NUsight2/keys{/key_id}","collaborators_url":"https://api.github.com/repos/NUbots/NUsight2/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/NUbots/NUsight2/teams","hooks_url":"https://api.github.com/repos/NUbots/NUsight2/hooks","issue_events_url":"https://api.github.com/repos/NUbots/NUsight2/issues/events{/number}","events_url":"https://api.github.com/repos/NUbots/NUsight2/events","assignees_url":"https://api.github.com/repos/NUbots/NUsight2/assignees{/user}","branches_url":"https://api.github.com/repos/NUbots/NUsight2/branches{/branch}","tags_url":"https://api.github.com/repos/NUbots/NUsight2/tags","blobs_url":"https://api.github.com/repos/NUbots/NUsight2/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/NUbots/NUsight2/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/NUbots/NUsight2/git/refs{/sha}","trees_url":"https://api.github.com/repos/NUbots/NUsight2/git/trees{/sha}","statuses_url":"https://api.github.com/repos/NUbots/NUsight2/statuses/{sha}","languages_url":"https://api.github.com/repos/NUbots/NUsight2/languages","stargazers_url":"https://api.github.com/repos/NUbots/NUsight2/stargazers","contributors_url":"https://api.github.com/repos/NUbots/NUsight2/contributors","subscribers_url":"https://api.github.com/repos/NUbots/NUsight2/subscribers","subscription_url":"https://api.github.com/repos/NUbots/NUsight2/subscription","commits_url":"https://api.github.com/repos/NUbots/NUsight2/commits{/sha}","git_commits_url":"https://api.github.com/repos/NUbots/NUsight2/git/commits{/sha}","comments_url":"https://api.github.com/repos/NUbots/NUsight2/comments{/number}","issue_comment_url":"https://api.github.com/repos/NUbots/NUsight2/issues/comments{/number}","contents_url":"https://api.github.com/repos/NUbots/NUsight2/contents/{+path}","compare_url":"https://api.github.com/repos/NUbots/NUsight2/compare/{base}...{head}","merges_url":"https://api.github.com/repos/NUbots/NUsight2/merges","archive_url":"https://api.github.com/repos/NUbots/NUsight2/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/NUbots/NUsight2/downloads","issues_url":"https://api.github.com/repos/NUbots/NUsight2/issues{/number}","pulls_url":"https://api.github.com/repos/NUbots/NUsight2/pulls{/number}","milestones_url":"https://api.github.com/repos/NUbots/NUsight2/milestones{/number}","notifications_url":"https://api.github.com/repos/NUbots/NUsight2/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/NUbots/NUsight2/labels{/name}","releases_url":"https://api.github.com/repos/NUbots/NUsight2/releases{/id}","deployments_url":"https://api.github.com/repos/NUbots/NUsight2/deployments","created_at":"2017-05-12T10:10:59Z","updated_at":"2017-07-20T03:33:12Z","pushed_at":"2017-07-20T12:08:06Z","git_url":"git://github.com/NUbots/NUsight2.git","ssh_url":"git@github.com:NUbots/NUsight2.git","clone_url":"https://github.com/NUbots/NUsight2.git","svn_url":"https://github.com/NUbots/NUsight2","homepage":"https://nusight.herokuapp.com","size":1347,"stargazers_count":6,"watchers_count":6,"language":"JavaScript","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":13,"forks":0,"open_issues":13,"watchers":6,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/NUbots/NUsight2/pulls/97"},"html":{"href":"https://github.com/NUbots/NUsight2/pull/97"},"issue":{"href":"https://api.github.com/repos/NUbots/NUsight2/issues/97"},"comments":{"href":"https://api.github.com/repos/NUbots/NUsight2/issues/97/comments"},"review_comments":{"href":"https://api.github.com/repos/NUbots/NUsight2/pulls/97/comments"},"review_comment":{"href":"https://api.github.com/repos/NUbots/NUsight2/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/NUbots/NUsight2/pulls/97/commits"},"statuses":{"href":"https://api.github.com/repos/NUbots/NUsight2/statuses/d2fe72aec172bc11efd1db224d633dd4797d7dd4"}}}}
{"actor":{"display_login":"TrentHouliston"}}