{"action":"created","issue":{"url":"https://api.github.com/repos/kaitai-io/kaitai_struct/issues/229","repository_url":"https://api.github.com/repos/kaitai-io/kaitai_struct","labels_url":"https://api.github.com/repos/kaitai-io/kaitai_struct/issues/229/labels{/name}","comments_url":"https://api.github.com/repos/kaitai-io/kaitai_struct/issues/229/comments","events_url":"https://api.github.com/repos/kaitai-io/kaitai_struct/issues/229/events","html_url":"https://github.com/kaitai-io/kaitai_struct/issues/229","id":252929833,"number":229,"title":"Better (our own?) YAML parser","user":{"login":"GreyCat","id":157154,"avatar_url":"https://avatars0.githubusercontent.com/u/157154?v=4","gravatar_id":"","url":"https://api.github.com/users/GreyCat","html_url":"https://github.com/GreyCat","followers_url":"https://api.github.com/users/GreyCat/followers","following_url":"https://api.github.com/users/GreyCat/following{/other_user}","gists_url":"https://api.github.com/users/GreyCat/gists{/gist_id}","starred_url":"https://api.github.com/users/GreyCat/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/GreyCat/subscriptions","organizations_url":"https://api.github.com/users/GreyCat/orgs","repos_url":"https://api.github.com/users/GreyCat/repos","events_url":"https://api.github.com/users/GreyCat/events{/privacy}","received_events_url":"https://api.github.com/users/GreyCat/received_events","type":"User","site_admin":false},"labels":[{"id":329236478,"url":"https://api.github.com/repos/kaitai-io/kaitai_struct/labels/enhancement","name":"enhancement","color":"84b6eb","default":true}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2017-08-25T15:04:16Z","updated_at":"2017-08-25T15:35:40Z","closed_at":null,"body":"This is the question that was discussed a zillion of times, but I'd want to create a separate issue for this one.\r\n\r\n## Current state of things\r\n\r\nRight now we're using external YAML parsers to parse .ksy files, that is:\r\n\r\n* for JVM build of ksc — [SnakeYAML](https://bitbucket.org/asomov/snakeyaml), written in Java\r\n* for JS build of ksc — yamljs / jsyaml / ..., written in JavaScript; effectively they just convert YAML into JS objects/arrays structure\r\n\r\n## What we're generally ok to drop\r\n\r\nSome YAML compatibility, i.e. we're generally ok to implement a smaller YAML subset. For example, we don't need:\r\n\r\n* multiple documents in a file\r\n* directives\r\n* tags\r\n* node anchors and references\r\n* block chomping controls\r\n* explicit typing with `!!`\r\n* timestamps\r\n* actually anything beyond [failsafe schema](http://www.yaml.org/spec/1.2/spec.html#id2802346)\r\n\r\nHowever, we shouldn't *add* and *modify* YAML semantics, i.e. all our .ksy files should still stay valid YAML documents, available for parsing in other YAML parsers.\r\n\r\n## What we'd want to have\r\n\r\nA parser that:\r\n\r\n* is written in Scala with no major extra dependencies — this way it would compile smoothly for all 3 major Scala targets: JVM, JS and native binaries\r\n* allows semi-complete lines that appear while one's in progress of typing a line (useful for IDE); it should report an error here, but try to resume parsing, probably by throwing erroneous line and trying to resume from the next line\r\n* reports exact positions of the problems — i.e. line/column, not only YAML path\r\n* reports many (ideally, all) problems from the run, not only the very first one\r\n* allow distinction (and proper error reporting) between stuff like:\r\n  * having `null` specified vs having no value at all\r\n* allows to check [our style guide](http://doc.kaitai.io/ksy_style_guide.html) stuff and issue warnings on them (and maybe offer autocorrection?)\r\n\r\nAnything else?\r\n\r\n## Current efforts\r\n\r\n* @GreyCat tried to port SnakeYAML into Scala, but that project kind of stalled.\r\n* @koczkatamas wrote [proof-of-concept AST parser of YAML subset in TypeScript](https://github.com/kaitai-io/kaitai_struct_webide/blob/master/playground/KsyAst.ts), which is used as addendum for code completion purposes only"},"comment":{"url":"https://api.github.com/repos/kaitai-io/kaitai_struct/issues/comments/324956326","html_url":"https://github.com/kaitai-io/kaitai_struct/issues/229#issuecomment-324956326","issue_url":"https://api.github.com/repos/kaitai-io/kaitai_struct/issues/229","id":324956326,"user":{"login":"koczkatamas","id":2608082,"avatar_url":"https://avatars1.githubusercontent.com/u/2608082?v=4","gravatar_id":"","url":"https://api.github.com/users/koczkatamas","html_url":"https://github.com/koczkatamas","followers_url":"https://api.github.com/users/koczkatamas/followers","following_url":"https://api.github.com/users/koczkatamas/following{/other_user}","gists_url":"https://api.github.com/users/koczkatamas/gists{/gist_id}","starred_url":"https://api.github.com/users/koczkatamas/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/koczkatamas/subscriptions","organizations_url":"https://api.github.com/users/koczkatamas/orgs","repos_url":"https://api.github.com/users/koczkatamas/repos","events_url":"https://api.github.com/users/koczkatamas/events{/privacy}","received_events_url":"https://api.github.com/users/koczkatamas/received_events","type":"User","site_admin":false},"created_at":"2017-08-25T15:35:40Z","updated_at":"2017-08-25T15:35:40Z","body":"> block chomping controls\r\n\r\nIf this is about string parsing then the formats repo currently use the `|-` operator. I don't know if it's a mistake or a required feature.\r\n\r\n> Anything else?\r\n\r\nI forgot to mention but I would need a mode where the parser also stores every node's original position (file byte offset, row and column number in an optimal situation), so if I want to select the `/types/something/seq/2/encoding` node then I could set the cursor position to this node in the text editor.\r\n\r\nI can also revert this position-node map and if ex. the user is currently at the 153 byte offset, I'll know he is at the `/types/something/seq/2/encoding` node and I can show him the auto complete at that position or jump to the parent node if needed, etc.\r\n\r\nOr if we ever create source maps then those will store that \"this JS code was generated by the `/types/something/seq/2` field\" and I can - again - select that node if needed.\r\n\r\nCurrently my AST parser stores every node's exact start and end position (separately even for a map's key, etc). Maybe it's enough to store only the start position, but I wanted to be future-proof."}}
{"actor":{"display_login":"koczkatamas"}}