{"action":"created","issue":{"url":"https://api.github.com/repos/tgingold/ghdl/issues/111","repository_url":"https://api.github.com/repos/tgingold/ghdl","labels_url":"https://api.github.com/repos/tgingold/ghdl/issues/111/labels{/name}","comments_url":"https://api.github.com/repos/tgingold/ghdl/issues/111/comments","events_url":"https://api.github.com/repos/tgingold/ghdl/issues/111/events","html_url":"https://github.com/tgingold/ghdl/issues/111","id":163503865,"number":111,"title":"GHDL: a complete VHDL parser","user":{"login":"1138-4EB","id":6628437,"avatar_url":"https://avatars1.githubusercontent.com/u/6628437?v=4","gravatar_id":"","url":"https://api.github.com/users/1138-4EB","html_url":"https://github.com/1138-4EB","followers_url":"https://api.github.com/users/1138-4EB/followers","following_url":"https://api.github.com/users/1138-4EB/following{/other_user}","gists_url":"https://api.github.com/users/1138-4EB/gists{/gist_id}","starred_url":"https://api.github.com/users/1138-4EB/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/1138-4EB/subscriptions","organizations_url":"https://api.github.com/users/1138-4EB/orgs","repos_url":"https://api.github.com/users/1138-4EB/repos","events_url":"https://api.github.com/users/1138-4EB/events{/privacy}","received_events_url":"https://api.github.com/users/1138-4EB/received_events","type":"User","site_admin":false},"labels":[{"id":288528274,"url":"https://api.github.com/repos/tgingold/ghdl/labels/Question","name":"Question","color":"cc317c","default":false}],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":23,"created_at":"2016-07-02T05:52:29Z","updated_at":"2017-12-09T08:12:58Z","closed_at":"2016-07-06T04:30:21Z","author_association":"CONTRIBUTOR","body":"I've been thinking long about having some multiplatform (windows and gnu/linux), free (libre), lightweight and standalone tool to analize large bodies of VHDL 2008 code at block/RTL level. That tool would provide a hierarchy browser of the compiled/synthetized design, so that all the parameterized types and components are resolved. Ideally, the info in the hierarchy would be enough to build a command line tool to draw diagrams with different levels of abstraction (entities, processes and concurrent black boxes) for documentation purposes. As a dream, a GUI could be built with linked references between diagrams and sources.\r\n\r\nAmong those features, hierarchy browsers are already available in all the big (and heavy) EDA tools and simulators; diagrams and linked references are also supported by most of the big editors. However, most of those tools require several GB to be installed, since they provide lower levels of abstraction and many other utilities. On top of that, the heavy GUIs must be launched even to just analize the hierarchy.\r\n\r\nThe closest alternative I am aware of is [RTLVision](http://www.concept.de/nlview.html#FAMILY). It [seems](http://www.concept.de/flash/RTL_basic.swf) to be very complete technically, it is supported in multiple platforms (windows/linux/solaris), and accepts VHDL, Verilog and System Verilog. Moreover, the [engine](http://www.concept.de/nlview.html#FAMILY) can be used in many GUI environments. Indeed, I believe it is used by some of the big players in the industry. It is not specified which the supported versions of the standard are, but I believe it should be quite complete for 2008, since they [use](http://www.concept.de/nlview.html#FAMILY) [Verific](http://www.verific.com/)'s parser. However, I don't know how fast it is, given that it accepts multiple languages, and that it includes a waveform viewer to allow signal tracing. In any case, it is neither libre nor free or open source.\r\n\r\nThere are some projects from academia, such as [VHDLVisualizer](http://www2.fiit.stuba.sk/~macko/VHDLVisualizer/) and [HDLVisualizer](http://labss2.fiit.stuba.sk/TeamProject/2011/team05pkss/docs/); and even web solutions: [vhdltolatex.mamikon.net](http://vhdltolatex.mamikon.net/). But all of them lack proper language support. Is worth referring to [VisualHDL](http://visualhdl.sysprogs.org/) even though its input is THDL++ and outputs VHDL.\r\n\r\n---\r\n\r\nA a result, I realized that the starting point should be a reliable VHDL parser. So I searched for free alternatives:\r\n- Many grammars and parsers for VHDL 1993 (or version not specified) [[tams.informatik.uni-hamburg.de/vhdl](https://tams.informatik.uni-hamburg.de/vhdl/index.php?content=07-tools#grammar)]\r\n- Not complete VHDL 2008 grammar for Coco/R [[github.com/wltr/coco-vhdl-2008](https://github.com/wltr/coco-vhdl-2008)]\r\n- Handmade prototype of parser based on regexps [[bitbucket.org/AngelEzquerra/vhdl_analyzer](https://bitbucket.org/AngelEzquerra/vhdl_analyzer)].\r\n- Parser for Perl last updated in 2002 [[cpan.org/~gslondon/Hardware-Vhdl-Parser-0.12](http://search.cpan.org/~gslondon/Hardware-Vhdl-Parser-0.12/)]\r\n- Grammar for Lex-Yacc (version not specified) [[vhdlverif.sourceforge.net](http://vhdlverif.sourceforge.net/index.html)]\r\n- Not complete VHDL 2008 AST [[github.com/grwlf/vsim/blob/master/grammar/vhdl-grammar](https://github.com/grwlf/vsim/blob/master/grammar/vhdl-grammar)]\r\n- ANTLR4 grammars [[github.com/antlr/grammars-v4](https://github.com/antlr/grammars-v4)]\r\n\r\nSince I found no valid candidate, I had to accept that parsing VHDL is not straightforward. Some others reached the same conclusion before: [Parsing VHDL is [very] hard ](http://eli.thegreenplace.net/2009/05/19/parsing-vhdl-is-very-hard), [A Note on the Parsing of Complete VHDL-2002](http://ceur-ws.org/Vol-255/paper11.pdf).\r\n\r\nThen, even though ANTLR4 could be the least bad starting point, maybe parsing VHDL with a general purpose engine is not the way to go.\r\n\r\n**EDIT 2017-12-07 >**\r\nSince this first message was posted, I have been informed about new projects:\r\n\r\n- [Nic30/hdlConvertor](https://github.com/Nic30/hdlConvertor) is an implementation using ANTLR4 for VHDL2002, which uses [this](https://guest.iis.ee.ethz.ch/%7Ezimmi/download/vhdl02_syntax.html) grammar, along with the one at antlr/grammars-v4.\r\n- [Paebbels/pyVHDLParser](https://github.com/Paebbels/pyVHDLParser) is a \"different kind\" of parser. There are several characteristics that make this project specially interesting:\r\n  - It is explicitly meant to support VHDL-2008 and (hopefully) VHDL-2018.\r\n  - The author and maintainer (@Paebbels):\r\n    - Is activaly participating in the evolution of the language and pushing towards the integration of new features. I.e., his knowledge about corner cases and how hard parsing VHDL can be is far beyond any average or even advanced user/developer.\r\n    - Is also involved in the development of GHDL (he is one of the 3 most active developers) and is currently in the maintainers group.\r\n    - Has recently stated that his objective is to use GHDL professionally for training: https://github.com/tgingold/ghdl/issues/469#issuecomment-347028389 . Therfore, I expect him to push pyVHDLParser to be as strict at conforming the standard as GHDL is.\r\n  - Some python integration is being added to GHDL, which will expectedly allow better integration between both projects.\r\n\r\n---\r\n\r\nI think that pyVHDLParser is the best bet. Indeed, if I were to start this thread right now, I would focus on understanding what the differences between GHDL's built-in features and pyVHDLParser are:\r\n\r\n- Will they always be split projects?\r\n- Can part/all of pyVHDLParser be embedded in GHDL?\r\n- Can some of the features in pyVHDLParser be externalized to GHDL?\r\n- Does it make sense to use GHDL as a preprocessor (i.e. get the hierarchy) \r\n\r\n> [@tgindold 2016-07-03](https://github.com/tgingold/ghdl/issues/111#issuecomment-230159008)\r\n> In fact you want more than a vhdl parser: you also want a vhdl\r\nelaborator to get access to all instantiated units, signals and connections.\r\n>\r\n> GHDL has all these data, at least internally. The options --disp-tree\r\ncan be used to dump some of them and vpi/vhpi could also be used to get\r\nmore details. However, this won't be as complete as RTL view, as ghdl\r\nis not a synthetizer and doesn't create a netlist.\r\n> \r\n> Generating a dot/graphviz connection net shouldn't be very difficult.\r\n\r\nNOTE: the rest of this comment are old thoughts, comments and questions that lead to the conclusion above. They are kept because technical details are discussed, which are useful on the path to understand the questions above.\r\n**EDIT 2017-12-07 <**\r\n\r\n---\r\n\r\nSince GHDL is a compiler, it actually parses at least part of the code. It's fast, lightweight, multiplatform, standalone and free. Moreover, it is known for being strict at conforming the standard. I think this is the most valuable feature, given that there already exist multiple incomplete projects which are not expected to be ever improved to this extent.\r\n\r\nHowever, I know nothing about Ada and less about how compilers work. I don't know how much of the structure of the design has to be explicitly known by the GHDL core in order to generate machine code; i.e., whether blocks of VHDL code can be fed directly into an Ada compiler. Anyway, I have analized the content of a library file:\r\n\r\n```\r\n  package pkg at 1( 0) + 0 on 11;\r\n  package body pkg at 6( 93) + 0 on 12;\r\n  package mygpkg at 13( 232) + 0 on 13;\r\n  entity ent at 17( 293) + 0 on 14;\r\n  architecture arch of ent at 23( 400) + 0 on 15;\r\n```\r\n\r\nIt seems that I could build a hierarchy view of the packages, entities and architectures defined in each file. On top of that, the [pretty print command](http://ghdl.readthedocs.io/en/latest/Invoking_GHDL.html#pretty-print-command) and the [cross-reference command](http://ghdl.readthedocs.io/en/latest/Invoking_GHDL.html#cross-reference-command) syntax highlight every word; the second one also links every identifier (packages, entities, architectures, types, signals...) to it's declaration. Then, although I have not found explicit references in the user guide about how the html is generated, I would say that GHDL could be a good candidate. Nevertheless, the [interfacing to other languages](http://ghdl.readthedocs.io/en/latest/GHDL_implementation_of_VHDL.html#interfacing-to-other-languages) section of the user guide explains how to import subprograms, but not how to access to information about the design. I would say that the only choice at now is `AVHPI`, which is _a very light and slightly adapted version of VHPI_. I suppose that it is a very first step forward to something similar to [Aldec's VHPI Applications](https://www.aldec.com/en/support/resources/documentation/articles/1457).\r\n\r\nIn the end, the questions:\r\n- @tgingold, is GHDL suitable for this purposes, or is it focused on generating executable machine code?\r\n> [@tgindold 2016-07-02](https://github.com/tgingold/ghdl/issues/111#issuecomment-230090108)\r\n> Yes, it is suitable. GHDL is able to parse and analyze vhdl sources and uses an AST as an internal representation of vhdl source. This part is clearly separated from the code generation and simulation part.\r\nFor example, the cross-reference only uses the parser and analyzer, but not at all the code generation part.\r\n\r\n- Is Ada the only language which can be used to extract information of the elaborated design from GHDL?\r\n> [@tgindold 2016-07-02](https://github.com/tgingold/ghdl/issues/111#issuecomment-230090108)\r\n> No. The AST representation (documented in [iirs.ads](https://github.com/tgingold/ghdl/blob/master/src/vhdl/iirs.ads)) could be read from any language. It shouldn't be very hard to have a python interface for example.\r\n\r\n- Which are the supported VHPI functions and objects?\r\n> [@tgindold 2016-07-02](https://github.com/tgingold/ghdl/issues/111#issuecomment-230090108)\r\n> Far from being complete. VHPI is only supported for the simulation aspect, and just complete enough to support cocotb.\r\n\r\n- Given the actual status, how much effort would be needed to print the complete hierarchy of a design (top entity, ports, signals, processes, generate blocks, instantiated modules...) in any format (ascii tree to stdout, json, xml...).\r\n> [@tgindold 2016-07-02](https://github.com/tgingold/ghdl/issues/111#issuecomment-230090108)\r\n> It is already possible to display the hierarchy using the --disp-tree option. The output is an ascii tree, but it wouldn't be that difficult to generate a json file.\r\n\r\nAlthough I don't feel confident enough to dive into the sources of GHDL, I wouldn't mind to give Ada a try someweek in the next months. I'd like to have a big picture, to have a better reference of how much time it would require."},"comment":{"url":"https://api.github.com/repos/tgingold/ghdl/issues/comments/350431865","html_url":"https://github.com/tgingold/ghdl/issues/111#issuecomment-350431865","issue_url":"https://api.github.com/repos/tgingold/ghdl/issues/111","id":350431865,"user":{"login":"Paebbels","id":956109,"avatar_url":"https://avatars0.githubusercontent.com/u/956109?v=4","gravatar_id":"","url":"https://api.github.com/users/Paebbels","html_url":"https://github.com/Paebbels","followers_url":"https://api.github.com/users/Paebbels/followers","following_url":"https://api.github.com/users/Paebbels/following{/other_user}","gists_url":"https://api.github.com/users/Paebbels/gists{/gist_id}","starred_url":"https://api.github.com/users/Paebbels/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Paebbels/subscriptions","organizations_url":"https://api.github.com/users/Paebbels/orgs","repos_url":"https://api.github.com/users/Paebbels/repos","events_url":"https://api.github.com/users/Paebbels/events{/privacy}","received_events_url":"https://api.github.com/users/Paebbels/received_events","type":"User","site_admin":false},"created_at":"2017-12-09T08:12:58Z","updated_at":"2017-12-09T08:12:58Z","author_association":"COLLABORATOR","body":"@1138-4EB \r\n\r\nThe main difference or disadvantage of GHDL is that it is not Python. I don't mean this in a case of how it is written, but more in a case that several free cloud services allow us to install Python packages and addons, but no binary code. I'm using a Python based chain of cloud services to do post processing of VHDL source files. I see no way to handle this with GHDL.\r\n\r\nThe second difference is that GHDL (as any compiler) removed comments. As a tool that extracts e.g. documentation from sources, GHDL can not be used.\r\n\r\npyVHDLParser e.g. has an generic and abstract description of language constructs. We can move this part to a separate repository so both (or more) projects can use this description.\r\n\r\npyVHDLParser does not create a classic AST (Abstract Syntax Tree) as an intermediate representation. It creates a CodeDOM (Code Document Object Model). So modelling does not start at tokens as classical use; it starts at the language model and goes backward to the characters in a file.\r\n\r\nI think that the later processing steps in pyVHDLParser could use GHDL and pyVHDLParser \"frontend\" as an input.\r\n"}}
{"actor":{"display_login":"Paebbels"}}