{'fraction_non_alphanumeric': 0.0342031029619182, 'fraction_numerical': 0.009520451339915375, 'mean_word_length': 4.476833976833976, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
The New R Help System the Parser Rd Syntax

Version 2.10.0 of R includes new code for processing ‘.Rd’ help files. There are some changes to what is allowed, and some new capabilities and opportunities. One of the reasons for the success of R is that package authors are required to write documentation in a structured format for all of the functions and datasets that they make visible in their packages. This means users can count on asking for help on a topic "foo" using either the function call help("foo"), or one of the shortcuts: entering ?foo in the console, or finding help through the menu system in one of the graphical user interfaces. The quality of the help is improved by the structured format: the quality assurance tools such as R CMD check can report inconsistencies between the documentation and the code, undocumented objects, and other errors, and it is possible to build indices automatically and do other computations on the text. The original help system was motivated by the help system for S (Becker et al., 1988), and the look of the input files was loosely based on LATEX (Lamport, 1986). Perl (Wall et al., 2000) scripts transformed the input files into formatted text to display in the R console, LATEX input files to be processed into Postscript or PDF documents, and HTML files to be viewed in a web browser. These Perl scripts were hard to maintain, and inconsistencies crept into the system: different output formats could inadvertently contain different content. Moreover, since the scripts could only look for fairly simple patterns, the quality control software had trouble detecting many errors which would slip through and result in rendering errors in the help pages, unknown to the author. At the useR! 2008 meeting in Dortmund, one of us (Murdoch) was convinced to write a full-fledged parser for ‘.Rd’ files. This made it into the 2.9.0 release in April, 2009, but only as part of the quality control system: and many package authors started receiving warning and error messages. Over the summer since then several members of the R Core team (including especially Brian Ripley and Kurt Hornik, as well as the authors of this article) have refined that parser, and written renderers to replace the Perl scripts, so that now all help processing is done in R code. We have also added an HTTP server to R to construct and deliver the help pages to a web browser on demand, rather than relying on static copies of the pages. This article describes the components of the new help system. The Parser In the new help system, the transformation from an ‘.Rd’ file to a ‘.tex’, ‘.html’ or ‘.txt’ file is a two-step process. The parse_Rd() function in the tools package converts the ‘.Rd’ file to an R object with class "Rd" representing its structure, and the renderers convert that to the target form.