ASpecD: A Modular Framework for the Analysis of Spectroscopic Data Focussing on Reproducibility and Good Scientic Practice
Reproducibility is at the heart of science. However, most published results usually lack the information necessary to be independently reproduced. Even more, most authors will not be able to reproduce the results from a few years ago due to lacking a gap-less record of every processing and analysis step including all parameters involved. There is only one way to overcome this problem: developing robust tools for data analysis that, while maintaining a maximum of exibility in their application, allow the user to perform advanced processing steps in a scientically sound way. At the same time, the only viable approach for reproducible and traceable analysis is to relieve the user of the responsibility for logging all processing steps and their parameters. This can only be achieved by using a system that takes care of these crucial though often neglected tasks. Here, we present a solution to this problem: a framework for the analysis of spectroscopic data (ASpecD) written in the Python programming language that can be used without any actual programming needed. This framework is made available open-source and free of charge and focusses on usability, small footprint and modularity while ensuring reproducibility and good scientic practice. Furthermore, we present a set of best practices and design rules for scientic software development and data analysis. Together, this empowers scientists to focus on their research minimising the need to implement complex software tools while ensuring full reproducibility. We anticipate this to have a major impact on reproducibility and good scientic practice, as we raise the awareness of their importance, summarise proven best practices and present a working user-friendly software solution.
Introduction<!>Scientic software development<!>Aims and prerequisites<!>Reproducible<!>Reusable<!>Self-documenting<!>User-friendly<!>Extensible<!>Further aspects<!>Reproducible data analysis:<!>Basic concepts and components<!>Dataset<!>Metadata<!>History<!>Annotations<!>Representations<!>Reports<!>Models<!>No programming skills needed:<!>Easy handling of multiple datasets<!>Reproducible and automated<!>Fully unattended<!>History<!>Human-writable<!>Comparison with programmatic approach<!>Implementation details<!>Working with the ASpecD framework<!>Examples of concrete analysis packages<!>Outlook<!>Additional features<!>Support for further spectroscopic methods<!>ASpecD in context of a laboratory information management system<!>Conicts of interest<!>Supporting Information<!>Software availability
<p>Newton's famous sentence `If I have seen further it is by standing on y e shoulders of giants' [1,2] reveals a basic assumption about science probably every scientist will share: Progress in science is always based on the results obtained by others prior to us. Therefore, these results should better be correct, and to ensure this, they need to be independently reproducible and reproduced. Hence, reproducibility is a key aspect of science. [3,4,5,6,7] This emphasis on independent testability can be traced back at least to Karl Popper and his critical rationalism, particularly his notion that scientic hypotheses can never be proven, but only falsied. [8] Dening the term `science' has been notoriously dicult, [9] although it has been characterised as a systematic endeavour early on. [10] However, key to empirical science and to what is known as the `scientic method' [11] is a structured sequence of events summarised in Fig. 1. Everything starts o with an observation that catches our curiosity, resulting in an approach to quantify the eect, i.e., a measurement. The data thus acquired need to be displayed rst, [12] before we can think of any model to describe them. Finally, after having modelled the data, we can present the results, leading to new observations that start again the whole cycle. Without properly documenting each of the steps, no reliable scientic knowledge can be built upon it.</p><p>With the advent of computers in the experimental research, the amount of data has grown exponentially, and today, data analysis is mostly computer-based. Therefore, the approach of a hand-written lab book to document each processing step is at best impracticable. The result: Most scientists will have hard time to trace a gure in their own publications back to the original dataset and give a detailed and gap-less account of each processing step that eventually led to the representation. [13,14,15] This poses a severe problem to science, [16] rendering the results obtained this way eventually unscientic. [17] Furthermore, requiring scientists to provide a complete record of their data processing would also cut down on cases of scientic misconduct.</p><p>Recently, developments towards implementing and using open-source laboratory information management system (LIMS) [18,19] and electronic lab notebooks (ELNs) [20] have gained traction that aim at increasing reproducibility of academic research. Some even include editors for spectra frequently encountered in analytical chemistry. [21] However, these systems usually cannot incorporate complicated computational analyses of recorded data that are regularly performed by spectroscopic groups. Furthermore, these analyses would again lack a gap-less record of each step and hence reproducibility.</p><p>Another problem arises from scientists being more and more forced to write their own software, whereas they have never been trained how to do so in a reliable fashion. [22] The results obtained by such software are often irreproducible and sometimes even inadvertently wrong, depending on the complexity of the matter. [23] Teaching essential aspects of software development, besides some very introductory courses in programming, basically has no part in the curricula of most science courses. [24,25,6,26] However, knowing and following a few basic rules both, in writing software for scientic data analysis as well as in analysing data using such software, can greatly enhance the reproducibility and in turn lead to better and more reliable research. [27,28,29] There are some investigations into how far scientic results are reproducible and correct, [23,30,31,16] as well as a poll on the topic. [32] Another aspect often discussed is the possible contributions of journals and publishers towards a greater availability of research by encouraging authors to provide both, data and source code, together with the publication of the results. [33,34,35] However, just publishing the source code of analysis software along with the raw data and results is not sucient. Rather, independent reproducibility needs to be a design goal early on in data handling and analysis. [36] Much has been written about the underlying general problem. [24,6,26,22,27,37,28,15,38,29,39]. For dierent scientic disciplines, the specic problems arising from computational science for reproducible research have been described, [40,41,42] and some areas have developed tools that are reasonably widespread. [43] Furthermore, there exist short contributions summing up `best practices' [44,45,46,47,28,48,49,50]. However, many of these articles have appeared either in engineering journals or in PLoS Computational Biology. The authors are, however, not aware of any publication in a journal regularly con-sulted by spectroscopists. Furthermore, no software package seems available addressing the aforementioned problems in their area of research.</p><p>To this end, we present here a modular framework for the analysis of spectroscopic data, termed ASpecD, built upon more than a decade of personal practical experience. [51,52,53] The result is a number of best practices and design rules for both, obtaining data and developing computer programs for their analysis. The ASpecD framework focusses on usability, small footprint and modularity, providing the user with a maximum of personal freedom while ensuring reproducibility and good scientic practice [54] and tremendously reducing the eort necessary to achieve these goals. Few such systems have been described before, perhaps most prominent is the work of Claerbout and his group in geology. [55,56] However, to the best of our knowledge, no such system exists for spectroscopic data.</p><p>Often spectroscopists seem even not to be aware of the importance of systematic and reliable tools for data processing and analysis, or they (understandably) lack ideas how to practically achieve the required reproducibility.</p><p>Two key concepts of the ASpecD framework are the unied representation of (numerical) data and accompanying metadata in a dataset independent of the original raw format and the possibility to reproducibly analyse data without having to program a single line using `recipedriven data analysis'. Here, the user only provides a list of datasets and tasks that should be performed on these datasets, including processing, analysis, and representation of the data.</p><p>After shortly discussing the aims and requirements of scientic software development in general, we introduce the basic concepts and components of a framework for reliable and reproducible analysis of spectroscopic data, followed by highlighting a few essential aspects of its general implementation. Using optical absorption (UV/Vis) data, a short example demonstrates what is possible even with the ASpecD framework itself, not using any derived package. Afterwards, details for packages based on the ASpecD framework are given and nally the relation to a complete workow including aspects such as data storage and unied access is shortly discussed.</p><!><p>As a matter of fact, programming skills are a fundamental requirement for many scientists nowadays, particularly in but not restricted to the eld of spectroscopy. Furthermore, the role of scientic thought in software development has been highlighted early on. [57] Therefore, scientists should be well equipped to develop reliable software for their own use if they would care as much about it as they do about their experiments and theories. [6]</p><!><p>Aspects central to scientic data analysis aiming at reproducibility and good scientic practice are summarised in Fig. 2 and will be detailed below. While originating from the context of scientic software development, the criteria listed here are true for all kinds of data analysis, regardless whether computers are involved. The principles listed below can be related to the FAIR principles published recently, [58] although the latter have a much broader scope and are usually connected to long-term availability and reuse of (processed) data.</p><!><p>The eventual goal of a framework for spectroscopic data analysis is the full reproducibility from the raw data to the nal publication and vice versa. The latter means to start from a published gure (or table) and trace it back to the original data, including the complete provenance and history of the processing steps involved. [55,56] Putting aside such aspects as long-term archival, unied access to data, and availability of the required software including the underlying operating system and compatible hardware, [17] we focus here on design rules of a modular framework taking care of the provenance of eventually published data, i.e. the history of processing steps from the raw data up to the nal representation and publication. Hence, the strategies discussed are entirely within the realm of the individual scientist's work and personal responsibility.</p><p>We note that there is some inconsistency in the literature regarding the usage of the two terms `reproducible' and `replicable', even between dierent disciplines, that have been addressed recently. [59,60,61] Here, we focus on the workow from the data acquisition to the nal publication of the analyses of the results and the traceability in both directions.</p><p>From a software development perspective, two aspects are crucial for obtaining reproducible results: a version control system (VCS) in conjunction with a scheme for unique version numbers and making the source code of the programs available (open source). Using a VCS is of utmost importance, as frequently highlighted in descriptions of best practices for scientic software development. [44] In the world of professional software development, there is no dispute about the importance of using a VCS, [62,63,64,65] such as git. [66] Providing the source code of software underlying scientic data analysis open source and free of charge is, as far as possible, a crucial step towards reproducible research and intellectual honesty. [67,68,69,70,35] From a scientic point of view, there is no good reason not to publish your source code, and the worst of these reasons is that it is not `good' enough. [71,72] However, source code can not always be published entirely, particularly if commercial programs are involved in data analysis, both as programming languages as well as tools. [73] Furthermore, while open source is a necessary requirement, it is not sucient on its own to allow for reproducibility. [36] In particular, appropriate metadata accompanying the raw data and the processing and analysis steps performed on them are of outstanding importance.</p><!><p>Every question addressed by scientists is new in a certain sense. However, often a series of routine steps is carried out that should be performed automatically as much as possible. Furthermore, code reuse is quite important, as the main focus of (most) scientists is not on developing software, but on performing research using software. Reusing existing components and combining them in new ways is a very general theme not only in science and a matter of eciency. To achieve this goal, software needs to be as modular as possible and sensible. Therefore, it needs to consist of cohesive and loosely coupled components. [74,62] To design a modular system, regardless of the actual context, requires abstraction. [75] Abstraction in turn requires a solid understanding of the `problem domain'. Software development should be driven by solving problems of the problem domain, as highlighted by Evans coining the term `domain-driven design'. [76] All this is well-known and at the heart of software engineering, as presented in numerous books throughout the last decades. [77,78,79,62,80,81,63,82] Many dierent tools have been developed to aid with nding appropriate abstractions and to implement them in software, starting with the now fundamental paradigms of structured, [83,84] objectoriented [85] and functional programming [86] all the way up to complicated frameworks for developing mobile apps, hiding most of the underlying complexity from the developers.</p><p>From another angle, reusing software components requires a solid understanding of the individual modules. This is even more true in science, where it is usually not sucient to use tools that give just about sensible results, but to know their exact specications and range of validity. Therefore, scientic software needs to be as readable and understandable as possible, as discussed in the next section.</p><!><p>The term `self-documenting' has two meanings in the given context. From a scientic point of view, all parameters of the data processing and analysis should be (automatically) logged. This is at the heart of reproducibility and traceability. However, with the advent of computeraided data processing, this aspect seems to have been gone lost. As a result, reproducibility of results published in the scientic literature is currently probably worse than it was previously, with tremendous impact on the reliability of science as such. [17] From a programming perspective, self-documenting implies the code to be readable on its own, ideally without the help of comments within the source code. Many dierent metaphors for programming have been brought forward, and the one seeing the programmer as an author might be most useful here. [87] This culminates in the advice for the software developer: `Write programs for people, not computers'. [88,44] Other scientists are the audience, and they should be able to read the code and make sense of it, even without too much prior knowledge of the programming language used. However, this is not restricted to science, but is a general theme highlighted over the last decades of professional software development [78,62,87] and recently summed up and advocated for under the term `clean code'. [87,82] 2.1.4. Reliable A basic requirement for software, particularly in science, should be correct and functioning code that is exible and comprehensible. [17] This requires rigorous tests of the software during its development. The idea of tests is, however, not to prove the program to work correctly, as this is mostly impossible based on logical grounds. Tests aim at nding errors in existing software. [17] While the importance of tests is not disputed and often highlighted, [62,89,90,80,63,91,65] testing software is a dicult matter. [92] This led some prominent professional programmers with decades of experience to inverting the process of writing functional code and corresponding tests. The idea of `test-driven development' is to rst write a (failing) test and afterwards the piece of software necessary to make it pass. [93,94] Used properly, this leads to software that is much more reliable and of higher quality. Important is to test already the smallest building blocks of software using automated unit tests and a testing framework to help with this. [95,94,96]. On aside, software that has been developed with testability in mind is intrinsically more modular, thus aiding with reuse and other of the aspects highlighted here. Furthermore, a high test coverage allows to enhance code readability without aecting the functionality, as advocated under the term `refactoring'. [90] 2.1.5. Veriable Strictly speaking, scientic hypotheses cannot be veried in sense of a formal proof, but only be falsied. [8] The same is true for most software. Both, the the general impossibility to formally prove most software, [97] as well as the general lines of Popper's thinking on science, [8] is rooted in Gödel's discovery of the fundamental limits of proofs in formal logic and mathematics. [98] However, data analysis as well as the programs used for it should be veriable in a sense that each step is fully transparently documented and all necessary information available required for an independent audit. This includes all parameters as well as the access to the very version of a software used.</p><p>From a programming perspective, a few more aspects are important. First of all, the source code should be readable and understandable on its own (`clean code'). [87] Furthermore, underlying concepts should be documented. While ideally, source code can be read and understood without additional comments, the underlying concepts and the interplay between dierent parts of the code need to be independently documented, e.g., in form of a user manual. This is particularly true for the implementation of complicated algorithms. Eventually, (automated) tests are a good way to ensure a routine to work reasonably correct, at least as far as the behaviour is covered by the tests.</p><!><p>Aspects that might be less obvious on rst thought but eventually decide whether a system will be adopted in practice can be summarised under the term `usability'. Only a system that behaves intuitively and is easy to use will be used on a daily basis. As any system aiming at providing a gap-less record of data processing requires additional work and discipline of its users, the obvious benets from using the system need to clearly outperform the additional eort in order to not undermine the very reason it has been implemented in the rst place.</p><p>Aspects from a programming perspective include intuitive and stable interfaces, robust code that gracefully handles errors as well as wrong user input, a proper user manual, and the habit of the developers to listen to the users and their needs.</p><!><p>Any framework for scientic data analysis needs to be highly modular and extensible, as frequent changes in the requirements and questions adressed to the data are at the heart of research. Many aspects of data analysis are routine tasks. However, a key aspect of scientic research is to creatively combine the available tools in new and unforeseen ways. This is why software as a separate component of computers was developed originally and where its name stems from. To be able to adapt a system to new requirements in a highly exible manner by (only) changing software while using general-purpose standard hardware was a necessary requirement for the unprecedented trail of success of computers that has revolutionised human life ever since. [75] From a programming perspective, this translates to highly modular code where every routine performs only one task. Furthermore, code needs to be easy to read and understand, as it will be read far more often than it is written. Automated tests are a necessary prerequisite for extending code and adding functionality, as they allow the programmer to make changes with reasonable condence to not break existing functionality. Similarly, a VCS greatly facilitates expanding software. Finally, extensible software requires the underlying concepts as well as the implemented interfaces to be properly documented, most probably in terms of an up-to-date user and developer documentation.</p><!><p>Besides the criteria mentioned above, additional prerequisites are immediately obvious from a software implementer's perspective: Due to the heterogeneity of the IT infrastructure in science, platform independence is imperative. Furthermore, such a system needs to be easy to integrate into existing workows of data processing. Here, modularity is a great benet, making it possible to start small using (parts of) the system without risk. Scientists are not software engineers, and software development will never receive the attention it might deserve in science. Hence, adopting the system `on the go' while still focussing on the actual science will be the likely usage scenario the system has to account for and adopt to. Additionally, using well-developed and freely available programming languages and making the source code available help to maintain and further develop the system even if the original developers are no longer available.</p><!><p>The ASpecD framework</p><p>To achieve the aforementioned goals, a framework for the Analysis of Spectroscopic Data (ASpecD) has been developed. The framework itself is implemented in Python and available open-source and free of charge. [99] Therefore, it can be used out of the box to derive processing and analysis routines for the specic data at hand. At the same time, it can serve as starting point for own developments aiming at better reproducibility. Therefore, we will rst give an overview of its basic concepts and components. Afterwards, its user interface that is radically dierent to all approaches currently employed in scientic data processing will be introduced: `recipe-driven data analysis'.</p><!><p>A graphical overview of the basic concepts and components of the ASpecD framework is given in Fig. 3. Each of the components will be discussed hereafter in some detail. One key concept is the dataset containing both, (numerical) data as well as information about these data (termed metadata). At the same time, a dataset contains a history of each processing step performed on its data together with the full set of parameters necessary to reproduce it. Representations are stored as abstract descriptions (another form of metadata), and annotations are used to store information that cannot be obtained by any automatic analysis, but requires human intervention. Reports are nally a way to obtain a well-formatted overview of certain or all aspects stored within a dataset. Tasks are at the heart of recipe-driven data analysis and can be thought of as abstract summary of any action taken on a single dataset or a dened list of datasets. Models are parameterised mathematical descriptions of the numerical data that can be tted to the latter. For details see the text.</p><p>Their interplay is graphically represented in Fig. 4. All parts have been implemented in a highly modular fashion (for implementation details see below) and can readily be used to develop packages for processing and analysis of spectroscopic data. Developers can entirely focus on implementing the science rather than ensuring reproducibility that is taken care of by the framework. Users, on the other hand, do not require any programming skills at all thanks to recipe-driven data analysis.</p><!><p>The dataset is one key concept of the ASpecD framework, storing (numerical) data, metadata and the history of processing and analysis steps in one place. One can think of it as an entity containing a eld for the numerical data, a structured array (called associative array, dictionary, map, hash, or struct, depending on the programming language, but essentially providing a keyvalue store) for the metadata, and a list of processing and analysis steps, i.e. their parameters and all information necessary to reproduce such a step, forming the history. If stored as a single unit, a dataset becomes the independent smallest component of the ASpecD framework, aiding much to reproducibility while maintaining modularity and a small footprint, i.e. a minimum of external dependencies.</p><p>Furthermore, the dataset serves as universal exchange format within the ASpecD framework: While each measurement software will come with its own le format and metadata organisation, the dataset abstracts from this di- Key is the dataset as a unit consisting of (numerical) data, metadata, and a full history of each processing and analysis step performed on the (numerical) data. Metadata of both, representations and annotations are stored as well within the dataset. The dierence between processing and analysis steps is that the former modify the data of a dataset, whereas the latter return an independent result. `info le' refers to a specic textual format for storing metadata accompanying a measurement. Its le format was developed to be easily (human) writable while retaining machine readability. For details see the text.</p><p>versity and represents a unied interface for all routines of the framework. This even allows to mix datasets of different origin, e.g. plotting the data together in one gure.</p><p>Starting from the initial import of the data and metadata, every routine operates exclusively on datasets, thus maintaining full reproducibility by automatically logging processing and analysis steps in the history and representations and annotations in their respective lists. At the same time, using the dataset as universal exchange format for all the dierent routines of the framework and its derived programs dramatically simplies the interfaces of these routines. Additionally, this provides all functionality necessary to full the requirements for a reproducible, modular and extensible data analysis.</p><!><p>Numerical data on their own are rather useless if they are not accompanied by additional information, here termed metadata. This insight is central to the development of the ASpecD framework. For a start, metadata of a measurement are all the information available at the very moment of the data acquisition, such as spectrometer type and software version, sample details and temperature of the setup. Hence, we need a way to record this information in an organised fashion, using a format that is both, human and machine readable. In its simplest form, this may just be a bare text le. Details will be discussed later on. Furthermore, it is the responsibility of the individual scientist to reduce the vast amount of information available during data acquisition to the relevant aspects. However, important for now is the intimate connection between data and associated metadata, as well as recording the metadata during data acquisition. The latter is crucial, as we cannot rely on our memory, not to mention that all too often, we will never provide the required information later on.</p><!><p>The key to reproducibility is to automatically log all tasks that are performed on a given set of data. Not only the processing and analysis steps as such, but as well a complete set of parameters and their values, including implicit or default parameters, need to be recorded. The version number of the routines, information about the underlying operating system, a timestamp, and the name of the operator are added as well, resulting in a gap-less history of processing steps on a given set of raw data. Additionally, this history gets stored such that it can be used directly to replay every step automatically. This opens the way for repeating (and systematically varying) the processing and analysis on the original dataset as well as applying it to other sets of data in a similar fashion, greatly assisting with comparing results and analyses.</p><p>At the same time, this allows for a straight-forward implementation of an undo/redo mechanism. An additional read-only eld contains the original raw data of the dataset. This prevents the application from having to reacquire the raw data from wherever they were loaded originally. Thus, reverting a processing step can be translated into starting o from the raw data and performing each processing step stored in the history except for the last one. Similarly, not immediately throwing away the metadata of the reverted processing step allows for its easy reapplication, termed `redo' in most software applications.</p><!><p>Whereas many routine processing steps on data can be carried out in a fully automated fashion, the actual data analysis always involves human intervention, at least with interpreting the results of preceding data processing and analysis steps. Often, there is the need to store additional information about a dataset, be it a comment applying to the entire data or highlighting an area or a single point. This is what annotations are meant for. Whereas the term is borrowed from bioinformatics and here from annotating DNA and protein sequences, it can be easily applied to spectroscopic data as well. In the context of the ASpecD framework, an annotation is characterised by its intrinsic human origin: The information contained cannot be (re)created in an automated fashion. Hence, excessively annotating datasets imposes severe constraints to the way the information is stored, regarding data safety (against accidential losses) as well as security (restricting access to the information).</p><!><p>A picture is worth a thousand words. Mostly, data are represented in a graphical or tabular manner in a publication. Whereas creating good gures is a topic on its own, [100,12,101] ideally, representations should be fully reproducible by replaying a `recipe' stored in form of an instructional text le containing all required parameters and information. This abstract description of a graphical or tabular display of (characteristics of) a dataset is termed representation in the ASpecD framework. If a representation depends solely on the data contained in a single dataset, its metadata will usually be stored within the dataset.</p><!><p>Finally, all the information collected by using the framework and contained in a dataset needs to be accessed in simple yet powerful ways. This is the realm of reports. Automatically generated from a dataset as source of structured information, they can be fully customised, resulting e.g. in a L A T E X le that gets automatically rendered to a PDF document. A key aspect is to separate formatting and content using template engines. [81] This allows for using a large list of dierent output formats fully independent of both, the underlying data source, in the given context a dataset, and the programming language used for data analysis as well as report generation. Similarly, reports of identical content but in dierent (human) languages can easily be created. Additionally, writing and modifying templates needs next to no programming experience and is fully independent of the other aspects of the data analysis framework. Hence, template engines are a very powerful application of the concept known as `separation of concerns'. [75,77] 3.1.7. Tasks Processing data consists of lots of dierent single tasks that can mostly be automated. This is the idea behind recipe-driven data analysis: lists of datasets and tasks that can easily be created by a user and processed fully automated. Here, `tasks' has a broad meaning, covering every aspect of data analysis that can be automated, such as processing and analysis steps, creating representations and annotations, and nally reports. For details, see the next section.</p><!><p>To make sense of and to interpret the physical reality reected in numerical data, usually mathematical models are used. These models typically depend on a number of parameters that may or may not be adjustable to t the actual data. Models can therefore be seen as abstraction to simulations. In this respect, they play a central role in conjunction with tting models to data by adjusting their respective parameters, a quite general approach in science and particularly in spectroscopy. Therefore, models provide the interface towards a tting framework [102] that is currently being developed in the authors' group.</p><!><p>Recipe-driven data analysis</p><p>The components previously described can all readily be used, adopting the perspective of a programmer. Therefore, they can form the basis for own scripts and programs as well as whole graphical user interfaces, while providing all the prerequisites for fully reproducible data analysis. Still, the user would need to be familiar with the underlying programming language to interconnect the dierent functions in a short program. In contrast, recipe-driven data analysis provides a wholly dierent level of abstraction, relieving the user from needing any programming skills. For a rst impression, see Listing 1.</p><p>Basically, recipe-driven data analysis can be thought of a special type of user interface to the ASpecD framework and derived packages. The normal user of such packages has a clear idea how to process and analyse data, as this belongs to the realm of science. At the same time, however, the user may not necessarily be interested in actually programming a lot. Furthermore, reproducible science requires the history of each and every processing and analysis step to be recorded and stored in a way that can be used and understood long after the steps have been carried out (think of many years rather than weeks or months).</p><p>From the user's perspective, all that is required is a human-writable le format and a list of datasets followed by a list of tasks to be performed on these datasets. For each task, the user will want to provide all necessary parameters, i.e., the metadata of the data analysis. Introducing the metaphor of recipe and cook circumvents using the term `metadata' in dierent meanings and the confusion this might cause. A recipe is a list of datasets and tasks to be performed on them. Such recipe is processed by a cook invoking the respective routines for each task. This is the origin of the term `recipe-driven data analysis'. An actual example of how such a recipe may look like is given in Listing 1. Of course, this is only a very simple example, and further details can be found in the online documentation [99] and the Supporting Information as well as in the online documentation of the cwepr package [103,104] based on the ASpecD framework. `Serving' the results of this recipe is as simple as issuing a single command on the terminal.</p><p>Essential aspects of the recipe-driven data analysis are detailed below and a graphical representation, contrasting the dataset-centric view, is shown in Fig. 5. Eventually, this abstraction renders scientic data analysis programing language agnostic and provides a complete new level of reproducible research. From own experience, the dramatically simplied user interface has the potential to revolutionise the way spectroscopic data are analysed, paving the way for truly reproducible research.</p><p>Listing 1: Example of a recipe used for recipe-driven data analysis within the ASpecD framework. Here, a list of datasets is followed by a list of tasks. The user needs no programming skills, but can fully focus on the tasks to be performed. `Cooking' this recipe is a matter of issuing a single command on the terminal. For comparison, Listing 2 shows the Python code of the same analysis carried out fully programmatically instantiating objects of the respective classes.</p><!><p>Besides providing an entirely new level of abstraction and a user-friendly interface to data analysis, recipe-driven data analysis overcomes an essential limit of the datasetcentric usage scenario shown in Fig. 4. namely its intrinsic limitation to operating on single datasets. However, data analysis often requires comparing dierent datasets. Therefore, history, representations and reports need to be abstracted from the single dataset and connected to a higher level, in this case the recipe allowing to span arbitrary numbers of datasets that should be processed and analysed together and compared in some way or other. Nevertheless, recipe-driven data analysis can at the same time be used in exactly the same way as the dataset-centric view, processing and analysing only one dataset at a time, but providing a much more user-friendly interface. Perhaps most crucial here is the lack of any programming skills needed, as the tasks are dened in textual, though structured way. This may be most obvious when comparing the elegance of the recipe given in Listing 1 with the corresponding Python code in Listing 2.</p><!><p>Processing data consists of lots of dierent single tasks that can mostly be automated. This is the idea behind recipe-driven data analysis: lists of datasets and tasks that can easily be created by a user and processed fully automated. `Tasks' has a broad meaning here, covering basically every automatable aspect of data analysis, including Figure 5: Recipe-driven data analysis as implemented within the ASpecD framework. Recipes provide a level of abstraction that empower users not familiar with programming to describe the details of arbitrarily complex data analysis step-by-step in a formalised way. They can be thought of as a special kind of user interface. A recipe consists of a list of datasets to operate on and a list of tasks that shall be performed. Tasks can be everything implemented within a package based on the ASpecD framework, from simple processing steps to complex reports. For details see the text.</p><p>processing and analysis steps, creating representations and annotations, and nally reports.</p><p>Storing the (somewhat abstract) recipes rather than scripts consisting of code depending too much on implementation details helps with reproducibility. On the one hand, problems with other versions of the underlying programs should be less frequent, and on the other hand, a human-readable list of datasets and tasks with their respective parameters is much easier to understand than actual code. For a direct comparison of the two approaches, see Listings 1 and 2 as well as the accompanying discussion.</p><p>Automatisation comes with several advantages. Generally, everything that can be automated can be delegated to computers. That does not mean that the tasks carried out are necessarily correct, but they are usually consistent. And if mistakes in the input are detected, this can be easily xed, resulting in a consistent (hopefully more correct) result. Additionally, every automated step saves the users from performing boring and error-prone routine tasks and allows them to use their brains for goodthinking of ways how to process and analyse the data and to make sense of the results, i.e. things computers cannot really do for us.</p><p>The power of automatisation is nicely reected in a quote by Whitehead in his `Introduction to Mathematics': `Civilization advances by extending the number of important operations which we can perform without thinking about them.' [105] The real important aspect of data analysis in science is to think about the data and the results obtained from automatically processing the (raw) data and to interpret these results. Everything else can (and should) be delegated to the computer as much as possible.</p><!><p>Recipe-driven data analysis is carried out fully unattended (i.e., non-interactive). This allows for using it in context of separate hardware and a scheduling system. Situations particularly beneting from this approach are either many datasets that need to be processed all in the same way, or few datasets requiring expensive processing such as simulation and tting [106,107]. The latter is even more true in context of global tting and/or sampling of dierent starting parameters, such as Monte-Carlo [108] or Latin-Hypercube [109] sampling approaches.</p><p>Furthermore, this approach allows for decoupling the place the actual data processing will be carried out from the input. This leads naturally to containerisation (e.g., docker [110]) where the actual packages derived from AS-pecD are located in a container that is self-contained and could basically even be shared with others. 4 Particularly in case of the above-mentioned long-running tting tasks, having the data analysis run not on the personal computer, but on dedicated hardware can be a great advantage.</p><!><p>Processing a recipe always results in documenting all tasks performed. This includes the complete set of available information necessary to reproduce and replay the tasks, both parameters and version information of the actual routines and all package dependencies used. For ease of use, these protocols can be used again as recipes. Therefore, the history connected now with the recipe rather than a single dataset represents a generalisation of the history concept described above. Besides that, all tasks operating on datasets will still write a history on each of the datasets.</p><!><p>As the recipe is used as kind of a user interface, it should be as simple to write and read as possible. Besides that, the format should be platform-independent and longtermstable. Therefore, a simple text format is the most natural choice. [64] The actual le format of the recipes is an unimportant detail. However, at least in a rst implementation of the ASpecD framework, the YAML le format [111] is used. Being very easy to write and read by humans due to the minimum of formatting required is perhaps its biggest advantage. Furthermore, due to its hierarchical structure, it allows for formulating even complex tasks such as a graphical representation with all the details that can be controlled and explicitly specied. For more complex examples see the Supporting Information. This format has been proven useful in similar settings, e.g., automatic provisioning of software using Ansible [112].</p><!><p>Both usage scenarios described above, the datasetcentric approach and the recipe-driven data analysis, have their particular strengths. The dataset-centric approach to the ASpecD framework is user-interface agnostic and hence compatible to application in context of an interactive graphical user interface (GUI). Recipe-driven data analysis, in contrast, provides a particular type of user interface aiming at an abstract description of the dierent tasks to be performed on a list of datasets. For a direct comparison of the two approaches, cf. Listings 1 and 2.</p><p>Generally, with reproducibility in mind, every kind of interactive graphical user interface usually poses a certain risk of lacking the complete history of events. This is leveraged using the dataset-centric approach and the dataset as universal exchange format between every routine. For a rst overview of the characteristics of a single dataset, a graphical user interface clearly has its advantages, while for routine processing and comparative analysis of multiple datasets, the recipe-driven data analysis approach shows its full potential. Besides full reproduciblity spanning multiple datasets, recipe-driven data analysis is at its most impressive when coming to fully unattended and automated routine data processing and analysis and when comparing a larger list of datasets in various ways. Furthermore, from the user's point of view, recipe-driven data analysis is much simpler, as it does not require programming skills and allows to focus entirely on the scientic side of data processing and analysis.</p><!><p>In contrast to engineering with a clear distinction between the design and the real object, in software development, the source code of the actual program is the design. [113] Furthermore, software is intrinsically subject to frequent changes, even more during active development. Therefore, the implementation details of the AS-pecD framework laid out in the following are necessarily abstract to a certain extent. However, due to it being open-source and fully documented, the interested reader is invited to have a look at the source code itself.</p><p>Software development in general and scientic software development in particular is a topic lling entire volumes. [63,79,80,81,76,114,65] Hence we will only provide a very brief overview here of the important aspects that shaped the implementation of the ASpecD framework. Further details can be found in the Supporting Information. Generally, three aspects can be distinguished: the necessary infrastructure for development and usage, the general implementation, and the actual code providing the functionality.</p><p>Listing 2: Example of a dataset-centric scenario of data analysis using the ASpecD framework. Here, objects of each of the respective classes need to be instantiated, the properties set appropriately and the interaction of the dierent classes directly implemented in code, calling the respective methods. For comparison, Listing 1 shows a recipe performing the same analysis. Independent reproducibility is at the heart of the scientic method, [3,4,5,6,7] and hence all software developed for scientic data processing and analysis should be made available for the community. [67,68,69,70,35] This requires a permissive license allowing others to use and potentially modify the code. [65,64] As software is always in constant development, a version control system (VCS) such as git [115,66] is the next important requirement, [62,63,64,65] allowing to restore the exact version of the software that was used for a particular task. Only in conjunction with a scheme for version numbers can this be achieved, with one version number referring to one and only one actual state of the software. In mathematical terms, this is is a one-to-one or bijective mapping between version number and program version. ASpecD follows the semantic versioning [116] scheme. As software for scientic data analysis is intrinsically complex, even the ideal implementation where the code fully speaks for itself cannot convey the overarching design principles. Therefore, extensive documentation of both, the public interfaces (API) of the routines as well as the underlying concepts is essential for the software to be usable and to get used. [65,64] This can only be achieved by employing tools that automatically generate well-formatted documentation from the source code, such as Sphinx [117]. Finally, we need to ensure the software to be correct as much as possible. This requires automatic tests to be present together with a high test coverage. [62,89,90,80,63,91,65,92] The best way to achieve testability and high test coverage is by using a test-rst approach [93,94] and automatic testing via a unittest framework [95,94,96].</p><p>With this infrastructure in place, implementing the actual software can begin. An essential prerequisite of the modularity and hence easy extensibility of the ASpecD framework is its use of the object-oriented programming paradigm. [118,119] Next is choosing a programming language that is easy to use for non-experts and comes at the same time with a large set of libraries for scientic data processing and an active user community. This is why Python [120,121,65,122,123] has been chosen for implementing the ASpecD framework, in conjunction with the NumPy, [124,125] SciPy, [126] and matplotlib [127] packages. A last aspect worth mentioning is le formats. The time scales of reproducibility in science and of the availability of digital le formats dier quite substantially. Nevertheless, scientic data need to be accessible after years and even decades. Therefore, only fully documented and open formats should be used. Furthermore, information about the numerical data (i.e., metadata) need to be stored together with the actual numerical data that are the results of measurements or computations. Generally, structured text les should be used wherever possible [64], with the YAML format [111] providing a good compromise of being human-writable while at the same time being processable by machines. For storing oating-point numbers in binary format, the IEEE 754 standard [128] is an acceptable exception from this rule, as long as its use is restricted to numeric data. Key to reproducible science is to collect all necessary information (metadata) about the original measurement or calculation as well as each individual processing and analysis step. As long as this information is collected and saved in a structured way in open formats, converting it to exchange formats for deposit in public databases and sharing with other scientists is possible and can usually be automated as well.</p><p>The actual modules the ASpecD framework consists of are basically identical to its components described above. For more details, the reader is referred to the Supporting Information and the extensive documentation available online [99]. Both include details as well of how to write and develop analysis packages based on the ASpecD framework. A number of such packages that are currently actively developed in the authors' group are shortly outlined below, after having given a practical example of working with the ASpecD framework itself. The four spectra presented have been normalised to their low-energy absorption band. A detailed analysis of the data shown here is given in [129].</p><!><p>To provide a concrete example how working with the ASpecD framework and recipe-driven data analysis looks like, in this section a series of optical absorption spectra will be plotted with each of the datasets normalised to the intensity of its low-energy band. The actual analysis of the data used here is given in [129]. Similarly to what has been shown in Listing 1, the steps are (i) importing the data, here setting some options for the importer, (ii) normalising over a specic range (conveniently given in wavelength), and (iii) plotting the result. The recipe is provided in Listing 3, the resulting graphical representation is shown in Fig. 6.</p><p>A few comments on this recipe (Listing 3): Due to using the bare ASpecD framework and CSV les for the data, in this particular case one needs to provide both, the importer and the plotter with a bit more details. For the importer, separator refers to the decimal separator, here the comma rather than the dot. This can be handled conveniently and transparently in the recipe, as well as the number of header lines to skip. Using the UVVisPy package [130], the import would be reduced to basically a single line per dataset. Similarly, as the data imported contain no metadata, the plotter needs to be provided with axis labels explicitly. `Serving' this recipe not only results in the gure, but in a recipe history as well. The latter contains all implicit and explicit parameters of each task as well as details on the platform and version of each package used during the analysis. For the full recipe history (that in itself can be used as a recipe), see the Supporting Information. Both, data and recipe are made available online, [131] to facilitate getting familiar with the ASpecD framework and recipe-driven data analysis.</p><p>For more detailed examples regarding processing and analysis of continuous-wave electron paramagnetic resonance (cw-EPR) spectra, the interested reader is referred Listing 3: Concrete example of a recipe used to plot a series of optical absorption data each normalised to their low-energy absorption band. The result is shown in Fig. 6 1 format : to [104], containing a series of typical workows and corresponding recipes.</p><!><p>Currently, the implementation of analysis tools for data from both, time-resolved [52] and conventional continuous-wave [53] electron paramagnetic resonance (EPR) spectroscopy is under way in the authors' group, helping to further develop the framework. The results are provided open-source and free of charge, and two Python packages derived from the ASpecD framework focussing on EPR spectroscopy, namely the trepr [132] and cwepr [103,104] packages, are available online via PyPI and GitHub. Both packages are now actively employed in the authors' group on a daily basis. These packages dene an explicit description of the metadata relevant for the respective spectroscopic method, importers for the dierent vendor-specic le formats, and processing and analysis routines that extend the existing functionality provided by the ASpecD framework. A particular highlight, demonstrating the modularity and validity of the approach taken with the ASpecD framework, is the capability of processing both, cw-EPR and tr-EPR data within one recipe and creating joint plots with both types of spectra. Details of these two packages are [104] and will be published elsewhere.</p><p>Furthermore, there exists a package for optical absorption (UV/Vis) data, the UVVisPy package [130]. The FitPy package [102] for tting models to spectroscopic data is as well based on the ASpecD framework and can be used in connection with all derived packages for specic spectroscopic methods. See below for further details.</p><p>While implementation of the packages mentioned above is driven by the needs of the authors, the ASpecD framework is by no means restricted to these types of spectroscopic data. Extention to, e.g., mass spectrometry and IR and NMR spectroscopy, is straight-forward, particularly when using the pymetacode Python package [133] that helps with creating and maintaining the package structure. Given the large amount of Python packages available for these kinds of data, a Python package based on the AS-pecD framework could reuse their functionality for data import and analysis, while adding a unied representation of a dataset including all relevant metadata as well as a gap-less record of each and every processing and analysis step. While handling of these types of data is currently done mostly in proprietary environments, providing opensource alternatives is imperative for reproducibility as well as for implementing the FAIR principles [58].</p><!><p>As with most software projects, developing the ASpecD framework is an ongoing process, and a few concrete ideas for further development are worth mentioning here. This involves additional features, support for further spectroscopic methods by implementing specic derived packages, and embedding the ASpecD framework into the larger context of an open-source laboratory information system aiming at even fuller reproducibility.</p><!><p>Besides implementing further generally applicable processing and analysis steps and extending the ways plots can be customised, interfacing with a tting framework is probably most important, followed by a graphical user interface to recipe-driven data analysis.</p><p>Fitting models to data is a frequent requirement in spectroscopy, and the Python SciPy package [126] provides excellent capabilities to this end. Developing a Python package (`FitPy' [102]) providing a unied interface to the dierent tting strategies, including stochastic sampling of starting conditions, is currently under way in the authors' lab, and a rst working prototype is available. Fitting is an analysis step, and the models to be tted are derived from the abstract model class provided by the ASpecD framework. Independently, simulation routines for EPR spectra are developed (Python package `SpinPy' [134]). Using FitPy together with SpinPy or alternative simulation routines [135,136] will allow to fully automatically process and analyse EPR data all the way from data acquisition to the nal publication-ready gure. Of course, FitPy can use any kind of model, as long as it conforms to the specication of a model of the ASpecD framework. Hence, FitPy is in no way restricted to EPR spectroscopy. Given the growing number of models available within the ASpecD framework, tting even works out-of-the-box for a series of standard spectroscopic tasks.</p><p>Development of a graphical user interface to recipedriven data analysis based on web technologies is currently under way, for a preview of a working prototype see the Supporting Information. While designed to run locally, using web technologies and a web browser makes the interface platform-independent. Datasets can be loaded, added and removed from a recipe, tasks conveniently be selected, previewed, edited, and added to the recipe, and nally the recipe be cooked and the served results be accessed. Particularly for more complex tasks and longer recipes, the WebUI greatly enhances the user experience and largely circumvents problems originating from wrongly formatted recipes.</p><!><p>Admittedly, EPR spectroscopy is a niche technique. However, there are currently rather concrete plans to develop a Python package for NMR spectroscopy based on the ASpecD framework called nmraspecds [137], and to extend the UVVisPy package [130] for optical absorption data. As with the other packages, the most crucial aspect is to dene an explicit description of the metadata relevant for the respective spectroscopic method, stemming from a thorough understanding and practical experience, and importers for the dierent vendor-specic le formats. Processing and analysis routines extend the existing functionality provided by the ASpecD framework where necessary. The two key dierences of these packages based on the ASpecD framework to other, existing Python packages are the unied representation of a dataset including all relevant metadata and the gap-less record of all operations performed on the data.</p><!><p>Whereas the ASpecD framework focusses on reproducibility of scientic data processing, important aspects for a fully reproducible research such as long-term archival of data as well as unique and persistent identiers for accessing data and metadata, though logical extensions and essential facets of the FAIR principles, [58] have been set aside here. Unique identiers become important as soon as more than one dataset is involved in the analysis, and one possible solution may be a system similar to the digital object identiers (DOI) familiar from scientic publishing, though primarily in a local context. Other aspects include a fully automated and metadata-driven workow for routine processing, starting with the raw data and ending with reports posted, e.g., to an internal web page.</p><p>Due to its highly modular design, the ASpecD framework well integrates in such larger infrastructure. Existing ideas for a modular laboratory information management system (LIMS) built from open-source components and having a small footprint are currently actively implemented in the authors' group and will be detailed elsewhere. [138] 9. Conclusions Reproducible research as well as adhering to good scientic practice is of high demand. Here, we have presented a set of best practices as well as design rules for scientic software for data processing and analysis. These are implemented in a modular framework for analysing spectroscopic data, termed ASpecD and focussing on usability while minimising system requirements and detailed knowledge in software development that is notoriously scarce in science. The implementation is driven by own needs and based on long-standing experience with both, experimental science and the intrinsic limitations with regard to software engineering in academia. Therefore, we are optimistic that the ideas presented here will encourage other scientists to focus more on reliable and reproducible software-based data analysis, thus ultimately contributing to the overall quality of research. Providing concepts as well as their implementation that take care of most of the equally important and tedious tasks, the additional effort besides personal discipline is rather negligible. Finally, recipe-driven data analysis not requiring any programming skills renders truly reproducible research easily achievable for spectroscopists.</p><!><p>There are no conicts to declare.</p><!><p>Examples of recipe-driven data analysis, a datasetcentric approach to the ASpecD framework, implementation details, writing software based on the ASpecD framework, history of the example recipe, preview of the webbased graphical frontend for recipe-driven data analysis.</p><!><p>The ASpecD framework is provided open-source and free of charge under a BSD license. Details can be found on its website (https://www.aspecd.de/) together with a detailed documentation for both, users and developers (https://docs.aspecd.de/). The ASpecD framework is available via the Python Package Index (PyPI) (https://pypi.org/project/aspecd/) and the source code via GitHub (https://github.com/tillbiskup/aspecd). The latest version is always available via the DOI 10.5281/zenodo.4717937.</p>