{'fraction_non_alphanumeric': 0.040993591441131294, 'fraction_numerical': 0.013293787405328108, 'mean_word_length': 5.208977310095364, 'pattern_counts': {'":': 0, '<': 1, '<?xml version=': 0, '>': 1, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
DeepA2: A Modular Framework for Deep Argument Analysis with Pretrained Neural Text2Text Language Models

In this paper, we present and implement a multi-dimensional, modular framework for performing deep argument analysis (DeepA2) using current pre-trained language models (PTLMs). ArgumentAnalyst – a T5 model [Raffel et al. 2020] set up and trained within DeepA2 – reconstructs argumentative texts, which advance an informal argumentation, as valid arguments: It inserts, e.g., missing premises and conclusions, formalizes inferences, and coherently links the logical reconstruction to the source text. We create a synthetic corpus for deep argument analysis, and evaluate ArgumentAnalyst on this new dataset as well as on existing data, specifically EntailmentBank [Dalvi et al. 2021]. Our empirical findings vindicate the overall framework and highlight the advantages of a modular design, in particular its ability to emulate established heuristics (such as hermeneutic cycles), to explore the model’s uncertainty, to cope with the plurality of correct solutions (underdetermination), and to exploit higher-order evidence.


Introduction
Argumentative text analysis is an interpretation method for clarifying arguments (Fisher, 2004). Being studied in argumentation theory, logic, or epistemology, it is widely taught and applied as a key critical thinking skill in, e.g., law (Alexy, 1989), the humanities (Bruce and Barbone, 2011), social sciences (Fairclough and Fairclough, 2012), policy advice (Hansson and Hirsch-Hadorn, 2016), or public debate (Beck et al., 2019). This paper presents a computational approach for deep argument analysis, i.e., for reconstructing naturallanguage arguments from a given text, as in the following example (adapted from Siegel, 2018): It is unethical to destroy human embryos. The most basic argument supporting this claim just stresses that it is wrong to intentionally kill innocent human beings.
(P1) It is impermissible to kill innocent human beings. (P2) The human embryo is an innocent human being. (C) THUS: It is impermissible to kill the human embryo.
The literature on argument reconstruction (cf. Feldman, 1998;Scholz, 2000;Lau, 2011;Bowell and Kemp, 2014;Brun, 2014;Brun and Betz, 2016) characterizes deep argument analysis as: • a complex task involving a variety of subtasks, such as identifying reasons and conclusions in a text, formalizing sentences, checking validity of an inference, logical streamlining, or explicating implicit premises. • a non-conservative, creative task that goes beyond mere text annotation and essentially generates a new, more transparent text. • an iterative process through which reconstructions are built and revised step-by-step, and the solution space is gradually explored. • a hermeneutical task, guided by the principle of charity, which urges one to come up with an interpretation (reconstruction) as strong and plausible as possible. • assuming a normative background theory about what constitutes a strong and plausible argument in the first place. • being affected by severe underdetermination, both in terms of the process and the final outcome; in particular, there typically exist rival, yet equally legitimate reconstructions of one and the same text.
Given these special characteristics, deep argument analysis poses many challenges for machine models of natural language understanding. In this paper, we introduce a novel modular modeling approach for analysing complex argumentation that builds on recent pre-trained text2text transformers (Raffel et al., 2020). Our approach -DeepA2 (illustrated in Figure 1) -works by systematically decomposing a complex reconstruction problem to smaller text2text sub-tasks (see Section 3), which allows for emulating the types of interpretation strategies and heuristics studied in argument theory. Referring to the different components of a comprehensive argumentative analysis, we may also define tailor-made metrics for assessing argument reconstructions. To demonstrate the benefits of our approach, we construct a new argumentation dataset (AAAC) that exhibits several complex interpretive dimensions, show how to map other existing datasets into our framework (Section 4), and train and evaluate our main model, referred to as Argu-mentAnalyst, within DeepA2 (Section 5).
Our empirical results show: 1. ArgumentAnalyst generates -out-of-domainsemantically meaningful argument reconstructions, 70% of which are logically valid. By pooling alternative reconstructions, virtually every source text in the synthetic dataset can be reconstructed as a valid argument.
2. Modular generation chains which emulate iterative reconstruction strategies are highly successful: they yield, in particular, a more coherent interpretation of an argumentative text, exploit the text more thoroughly, and generally outperform one-step generation as soon as problems become difficult.
3. ArgumentAnalyst outperforms Entailmen-tWriter (Dalvi et al., 2021) on difficult Entailment-Bank problems with respect to telling apart relevant premises from distractors. 4. ArgumentAnalyst generates reliable higherorder evidence (Christensen, 2010) which can be used for diagnosing logical fallacies -despite the fact that ArgumentAnalyst is maximally charitable and is trained to reconstruct any input whatsoever as a logically valid argument, even if the input argument, taken at face value, is painstakingly fallacious.
In concluding this paper, we sum-up and interpret these findings as general vindication of DeepA2's modular, multi-angular design (Section 6).
Neural semantic parsing uses sequence models to formalize natural language sentences (Kamath and Das, 2019). Shin et al. (2021) show that PTLMs are zero-shot parsers, and that intermediate steps which rephrase and streamline the original input before parsing it to a formal language improve accuracy.
Argument mining is an active research field that studies computational methods for retrieving argumentative components from a text corpus (Wachsmuth et al., 2017;Moens, 2018;Potthast et al., 2019;Lawrence and Reed, 2020). Recently, work in this field has started to use PTLMs: Ein-Dor et al. (2020) and Gretz et al. (2020) succeed in retrieving relevant pro-or con-arguments for a given topic from a large corpus with a fine-tuned BERT model (Devlin et al., 2019). Using BERT, Bar-Haim et al. (2020) map argumentative texts to key points that succinctly summarize the argument's gist. Akiki and Potthast (2020) explore abstractive argument retrieval by means of text generation with GPT2 (Radford et al., 2019). Similarly, Syed et al. (2021) deploy BART (Lewis et al., 2019) to generate conclusions of argumentative texts on a challenging corpus compiled from Reddit and various online debate corpora. Rodrigues et al. (2020), revisiting the argument comprehension task (Habernal et al., 2014(Habernal et al., , 2018, demonstrate that identifying implicit premises -and deep argument analysis a fortiori -remains a hard, unsolved task. Recently, Chakrabarty et al. (2021) have shown that augmenting training data with discourse-aware commonsense knowledge improves the plausibility of automatically identified implicit premises. Such a knowledge-driven perspective is orthogonal to, and may eventually complement the logical approach adopted in this paper.

Problem Definition
Deep argument analysis of a given text seeks to answer the following central question: Can we conjectures: source: Socrates is mortal because every human is. argdown: source: Socrates is mortal because every human is. conjectures: Socrates is mortal formalize: premises: Socrates is human | If someone is human, then they are mortal
(2) If someone is human, then they are mortal.
Socrates is mortal F a | (x): F x -> G x Figure 1: Example text-to-text tasks for deep argument analysis, defined by DeepA2.
make sense of the text as a presentation of a rational argument? And if so, what exactly is the argument; and how precisely is it related to the text? In carrying out a deep argument analysis, one explicates, rephrases and rebuilds -even repairs -the text's argument in one's own words. That is why deep argument analysis is also referred to as rational reconstruction (cf. Leitgeb and Carus, 2021). The reconstructed argument forms, together with details about its logical properties and about its relation to the source text, a comprehensive argumentative analysis of a text. The latter can be seen as an interpretative hypothesis that is abductively inferred from a source text by means of an inference to the best explanation. Here is another example that illustrates how far a reconstruction may deviate from the original text that presents the argument (adapted from Brun and Betz, 2016): source text ; reconstructed argument So, the researcher's central dilemma exists in an especially acute form in psychology: either the animal is not like us, in which case there is no reason for performing the experiment; or else the animal is like us, in which case we ought not to perform on the animal an experiment that would be considered outrageous if performed on one of us.
(P1) If the animal is not like us, it is wrong to perform the experiment. (P2) If the animal is like us, it is wrong to perform the experiment. (C) THUS (with classical dilemma): It is wrong to perform the experiment.
A compelling argumentative analysis yields (i) a rational argument that is (ii) closely related to the source text. Deep argument analysis is, accordingly, guided by a dual goal (cf. Brun and Betz, 2016). An argument reconstruction should both be (i) systematically correct, i.e., the reconstructed argument itself is, e.g., transparent, deductively valid, non-circular, or doesn't contain irrelevant premises; and (ii) exegetically adequate, i.e., the reconstructed argument accounts for the original text, because, e.g., its premises merely reformulate parts of the text, or because its overall inferential structure can be traced within the source text.
The fact that there typically exists -regarding a specific text -a trade-off between these two goals is one major reason for the underdetermination of deep argument analysis and the plurality of legitimate reconstructions of a given text (cf. Brun and Betz, 2016).
Against this background, we may finally define the problem of Deep artificial argument analysis: Describe, analyse and implement an effective computational system for deep argument analysis!

Multi-angular Data
The DeepA2 framework is built upon a multiangular data structure (Tafjord and Clark, 2021) whose dimensions represent the essential components of a comprehensive argumentative analysis (see Section 3.1). Structured argumentative data is rendered as plain text (cf. Voigt, 2014). The different data dimensions, which are related as shown in Figure 2, are (with an illustrating example): argument source text (S) It is unethical to destroy human embryos. The basic argument supporting this claim just stresses that it is wrong to intentionally kill innocent human beings. verbatim reason statements in source text (R) it is wrong to intentionally kill innocent human beings (ref: (1)) verbatim conjectures in the source text (J) It is unethical to destroy human embryos (ref: (3)) argument reconstruction (A) (1) It is impermissible to kill innocent human beings.
(2) The human embryo is an innocent human being.
-with hypothetical syllogism from (1) (2) -(3) It is impermissible to kill the human embryo.  premises of the reconstructed argument (P) It is impermissible to kill innocent human beings | The human embryo is an innocent human being final conclusion of reconstr. argument (C) It is impermissible to kill the human embryo formalizations of premises (F) Each record in a DeepA2 dataset contains a source text plus a legitimate comprehensive argumentative analysis, which is, given underdetermination, not necessarily the only compelling reconstruction of the text; moreover, a dataset may contain different records with one and the same source text analysed in several ways. So, for example, an alternative, equally legitimate argument reconstruction of the above source text (S) may read: argument reconstruction (A) (1) If it is wrong to kill innocent human beings, then it is wrong to kill a human embryo.
(2) It is wrong to kill innocent human beings.
Beyond this structural and functional characterization, DeepA2 is agnostic about the nature and origin of the argumentative data. Synthetically generated, automatically retrieved, manually created datasets as well as translations of other databases are all compatible with the framework and can be used side by side.

Generative Modes and Chains
Given DeepA2's multi-dimensional data structure described in the previous section, a generative mode maps data from some input dimensions to a target dimension. For example, the mode S ; A takes a source text (S) as input and outputs an argument reconstruction (A), the mode R J ; A reconstructs the argument (A) given the verbatim reasons (R) and conjectures (J). All in all, we define and investigate 21 different generative modes (see Appendix B). Every mode represents a task on which a text-to-text model can be trained.
By taking some mode's output as another mode's input, modes can be concatenated into generative chains. For example, the output of modes S ; R and S ; J (reasons and conjectures from source) can be fed into mode R J ; A to reconstruct an argument. Such generative chains allow us to emulate different strategies (heuristics) for analysing a given argumentative text (see Appendix C for technical details).
Three generative chains which model distinct interpretative strategies, taking a source text (S) as sole input, are: While the chain straight, where no output ever serves as input to another mode, represents a simple baseline, hermeneutic cycle and logical streamlining mimic prominent, equally-named methods in argument analysis (cf. Bowell and Kemp, 2014;Brun and Betz, 2016). One goes through a hermeneutic cycle, generally speaking, if one revisits a text in view of its previous interpretation, as, for example, in steps S A ; R S A ; J , where the source text (S) is re-interpreted (identifying reason statements and conjectures) given the previously reconstructed argument (A), so as to subsequently re-reconstruct the argument itself (step R J ; A ). To logically streamline a reconstruction means to rephrase its conclusion or premises in order to make their logicosemantic structure more transparent. Such semantic clarification can be emulated by (i) formalizing a statement (e.g., A ; C C ; O C O ; K ) and (ii) using the keys (K) to retrieve the original statement from the generated logical formulas (such as in O K ; C ), from which the argument can be re-built (step P C ; A ).
For evaluation, we append to each generative chain the following sub-chain that formalizes the reconstructed argument: A generative chain can be construed as hypergraph on the dimensions of DeepA2's multiangular datasets, with each of its modes representing a directed hyper-edge. Summing up the number of input dimensions (except S) over all modes yields a simple graph centrality measure, which gauges a chain's sophistication. Thus, straight, hermeneutic cycle and logical streamlining display a sophistication of 0, 4, and 11, respectively.

Metrics
As discussed in Section 3.1, an argument reconstruction should both be sound and make sense of the text to-be-interpreted. In line with the dual goal of argument analysis, we propose metrics both for the systematic correctness and for the exegetic adequacy of a given analysis. The following metrics measure the degree to which a given generated argument is systematically correct: SYS-PP 1 if the argument is not a petitio principii (i.e., if no premise is identical with its final conclusion), 0 otherwise; SYS-RP 1 if the argument has no redundant premises (i.e., if no premise occurs more than once), 0 otherwise; SYS-RC 1 if the argument has no redundant conclusions (i.e., if no conclusion -intermediary or final -occurs more than once), 0 otherwise; SYS-US 1 if all statements in the argument other than the final conclusion are explicitly used in an inference, 0 otherwise; SYS-SCH ratio of sub-arguments which correctly instantiate the explicitly stated inference scheme (e.g., hypothetical syllogism); SYS-VAL 1 if the argument is globally valid (i.e., if the final conclusion deductively follows from the premises), 0 otherwise; All six systematic metrics can be computed automatically (SYS-SCH tries to parse the argument based on the inference schemes and templates used to construct the synthetic dataset in the first place; SYS-VAL passes the model-generated formalizations of premises and conclusion to a symbolic theorem prover (De Moura and Bjørner, 2008); and the remaining metrics check for string identity).
Whereas systematic metrics apply primarily to the generated argument (A), a reconstruction's interpretative adequacy will also depend on how reasons (R) and conjectures (J) coherently link the argument's components to the original text. As a first set of exegetic metrics, we thus propose EXE-MEQ 1 if the reasons and conjectures are mutually exclusive verbatim quotes from the source text, 0 otherwise; EXE-RSS semantic similiarity (BLEURT, see Sellam et al., 2020) of each reason statement and its counterpart premise in the reconstructed argument (if such exists, -1 otherwise); EXE-JSS semantic similiarity (see EXE-RSS) of each conjecture statement and its counterpart in the reconstructed argument (if such exists, -1 otherwise).
Each source text presents (more or less faithfully) an underlying target argument, which in turn marks some of the text's statements as 'target' reasons, others as 'target' conjectures. The following two metrics assess the degree to which a comprehensive argumentative analysis correctly predicts (R, J) those target reasons and conjectures.
EXE-PPR predictive performance (F1-score) for identifying (target) reason statements in the source text; EXE-PPJ predictive performance (F1-score) for identifying (target) conjecture statements in the source text.
An argument's final conclusion may be implicit or explicit in a given text. The ability to fully exploit a text can be measured by verifying whether the reconstructed argument's final conclusion is implicit (= prediction) if and only if the target argument's one is.
EXE-TE text exploitation, as measured by ability (F1-score) to reconstruct arguments with explicit final conclusions (prediction) if and only if the target final conclusions are explicit.

Models
Any text-to-text language model is compatible with the proposed DeepA2 framework. We refer to models used within the framework as ArgumentAnalyst. In this study, we train and evaluate the transformer model T5 (Raffel et al., 2020) with 770M parameters as implemented by (Wolf et al., 2020).

Limitations
In the DeepA2 framework, arguments are reconstructed from relatively short and isolated texts, disregarding both the broader context of the argument and domain-specific background knowledge. This limits the framework, as presented here, in important ways: Implicit premises that are explicated in an argument reconstruction can neither be checked for plausibility nor for agreement with the author's broader convictions. In addition, the framework cannot assess an argument's dialectic function in a wider debate. It seems worthwhile to explore according extensions of the framework in future research.

Datasets
For the experiments reported below, we synthetically create two artificial argument analysis corpora that comply with the DeepA2 framework (see also Appendix A): AAAC01 and AAAC02. In addition, we translate the synthetic RuleTaker (Clark et al., 2020) and the manually compiled EntailmentBank (Dalvi et al., 2021) datasets into our framework.
In argument analysis, one proceeds from a source text to its reconstruction. Creating the synthetic corpora, we reverse-engineer this process: Step 1. We sample, first of all, a possibly complex argument (A) from a set of valid inference schemes. In doing so, we use a multi-step templating strategy (inspired by Betz et al., 2021) to translate symbolic forms into natural language schemes (which were generated by local domain experts) and to substitute natural language terms for placeholders. Premises (P), conclusion (C) and their formalization (F, O, K) are side-products of such a construction of an argument.
Step 2. Given the fully explicit argument (A), we compose a text (S) that presents the argument in a more or less transparent and faithful way. Such text creation involves: rendering the argument tree as a linear story, leaving out premises or conclusions (implicit premises and conclusions), inserting irrelevant material (distractors), using templates that obfuscate the logical form of a sentence, limiting the use of premise and conclusion indicators (such as "therefore"), applying rule-based and automatic paraphrasing. In composing the argumentative text (S), we may record its reasons (R) and conjectures (J).
Given the synthetic and controlled nature of our dataset, which involved eliciting rule templates from a group of local domain experts, all data is assumed to be correct by construction. As an additional check of correctness on the logic of our examples, we ran a symbolic theorem prover (De Moura and Bjørner, 2008) over the argument formalizations to verify their validity. To ensure the fluency of the underlying language templates, all templates were hand verified by the authors.
Our two datasets AAAC01 and AAAC02 differ in the following ways: 1. predicates and names are sampled from different, disjunct domains (texts are about, e.g., allergies and family relations versus, e.g., badminton and cooking) to test a model's robustness to lexical diversity (Rozen et al., 2019); 2. similarly, AAAC01 applies automatic paraphrasing (Alisetti, 2021) to the final source text whereas AAAC02 doesn't; 3. AAAC02 allows for imprecise renditions of logical formulas, while AAAC01 sticks to plain formulations to test robustness to variations in description of rules.
Each dataset contains diverse texts and arguments. Broadly speaking, data records may differ in terms of properties of the argument (step 1 above) and properties of the argument's presentation (step 2). Along these two dimensions, we define five homogeneous subsets of the data: simple inference: arguments with a single inference step that neither involves negation nor compositional predicates; complex inference: arguments with four inference steps that heavily rely on syntactically intricate schemes (e.g., transposition, or de Morgan); plain presentation: all premises and conclusions are explicit in the source text which, in addition, contains no distractors; mutilated presentation: at least two premises and one conclusion are implicit, while the text contains two distractors and explicitly states the final conclusion; C&M: the argument's inference is complex, plus the text contains at least two distractors.
The RuleTaker and EntailmentBank datasets contain multi-hop inference trees (A). To import these into the DeepA2 framework, we create source texts (S) for the given arguments by means of simple templates (such as "{theory} All this entails: {hy-pothesis}") and record reasons (R) and conjectures (J) on the fly. Unlike AAAC and EntailmentBank, RuleTaker (as updated in Tafjord et al., 2020) contains an equal share of arguments for which (i) the conclusion follows from the premises, (ii) the conclusion contradicts the premises, (iii) the conclusion is independent of the premises.

Experiments and Results
As first and main experiment we train our base model (see Section 3.5) on the AAAC01 corpus, and evaluate the resulting ArgumentAnalyst model out-of-domain on AAAC02. ArgumentAnalyst undergoes multi-task training on 21 generative modes, which are interpreted as sequence-tosequence tasks (the training set-up is further described in Appendix B).
The evaluation of ArgumentAnalyst on AAAC02 proceeds in two steps: (1.) prediction: produces output in accordance with 16 different generative chains (Appendix C); (2.) metrics application: assesses the quality of the generated output by means of the systematic and exegetic metrics of the DeepA2 framework (see Section 3.4). Table 1 reports the ability of ArgumentAnalyst to generate systematically correct and exegetically adequate argument reconstructions. We obtain similar global results with the three chains straight, hermeneutic cycle, and logical streamlining, whose generated reconstructions mainly differ in terms of internal coherence (EXE-RSS, EXE-JSS) and text exploitation (EXE-TE). However, the different generative chains complement each other, as shown by pooling, which does not only outperform individual chains, but nearly attains oracle performance.
Moreover, ArgumentAnalyst produces much better reconstructions of simple inferences and plain presentations -compared to complex inferences and mutilated presentations, i.e., difficult problems (cf. Table 5 in App. D). In addition, within one and the same subset, substantial differences show up between the three generative chains. Globally speaking, hermeneutic cycle outperforms the other two chains for difficult problems. Is ArgumentAnalyst capable of reliable selfevaluation? We have validated the logic metric (SYS-VAL), which passes on a self-generated formalization of the reconstructed argument to a theorem prover, in three ways: First of all, ArgumentAnalyst correctly recognizes target arguments as valid (with accuracy 92.7%), which has been verified by running the formalization subchain on target data. Secondly, virtually every generated argument with all-correct scheme instantiations (i.e., SYS-SCH = 1) is also -and correctly -recognized as logically valid. Thirdly, a manual analysis (humanin-the-loop) of 100 generated arguments with incorrect scheme instantiation (i.e., SYS-SCH < 1) reveals a high rate of false negatives: roughly one half of all inferences that are not automatically identified as an instantiation of the given scheme actually do correctly instantiate it. The accordingly adjusted global ratio of correct scheme instantiations (Table 1) equals roughly 0.65 (rather than 0.31-0.33), which is consistent with the ratio of logically valid arguments being 0.72-0.73.
Do reconstructed arguments exhibit basic semantic flaws? Regarding the full dataset, Argument-Analyst produces nearly flawless argument reconstructions, committing basic errors (petitio, redundancy, unused statements) only very rarely (Table 1). And even for very difficult problems, two thirds of all generated arguments display no basic flaw whatsoever (Table 5,

Are reconstructed arguments logically valid?
Roughly 70% of all arguments generated by one of the three chains are logically valid (Table 1). More importantly, though, for virtually every source text in the dataset, there is at least one chain (out of 16) which reconstructs the text as a valid argument (pooling). Given that logical validity can be automatically assessed, the pooled system may thus guarantee to yield a valid reconstruction. Concerning different problem types (Table 5), hermeneutic cycle clearly outperforms the other chains as soon as the problem gets difficult. Additional analysis shows that ArgumentAnalyst can also cope with underdetermination, as 68% of all generated arguments whose final conclusion differs (BLEU ≤ .8) from the target argument's one -i.e., arguments that are not reconstructed as expected given the target data -are still logically valid.
Are the generated interpretations internally coherent? The generative chain hermeneutic cycle yields comprehensive argument reconstructions where premises (P) and conclusions (C) fit much better to detected reasons (R) and conjectures (J) than straight or logical streamlining (EXE-RSS, EXE-JSS). This holds globally (Table 1), as well as for easy, and for difficult problems ( Table 5). Note that the oracle baseline for metrics EXE-RSS, EXE-JSS is well below 1, which reflects the fact that source texts may present arguments in highly mutilated ways; it is nearly attained by pooling the 16 different generative chains (Table 1).
Can ArgumentAnalyst detect reasons and conjectures, and fully exploit the text? The evaluation demonstrates that reason/conjecture detection on AAAC02 is a relatively easy task (EXE-PPR, EXE-PPJ).  In contrast, fully exploiting a text (i.e., generating an argument with implicit final conclusion if and only if the underlying target argument has an implicit final conclusion, EXE-TE) is seemingly more challenging (Table 1). Again, hermeneutic cycle achieves best text exploitation, performing, however, clearly below oracle baseline -which may simply reflect the degree of underdetermination in the AAAC02 corpus.
In a second experiment we train two models on the imported EntailmentBank (task1 and task2) dataset (see Section 4), namely: (1.) our base model (T5), which yields ArgumentAnalyst EB ; (2.) the ArgumentAnalyst model pretrained on AAAC02 (resulting in an intermediary pre-training set-up similar to Phang et al., 2018;Geva et al., 2020), which yields ArgumentAnalyst AAAC,EB . Since the EntailmentBank data doesn't contain formalizations, we can only train on 14 modes, which are interpreted as sequence-to-sequence tasks (see Appendix B). We evaluate the models on task2 of EntailmentBank only, which contains problems with a relatively large number of distractors, and proceed in two steps as before: prediction (with 11 different generative chains) and metrics application. Dalvi et al. (2021) report the ability of EntailmentWriter (a fine-tuned T5-11b model) to correctly distinguish relevant premises of an argument from distractors in terms of a F1-score, which corresponds to our metric EXE-PPR. That's why the sole focus in this second experiment is on EXE-PPR. Table 2 describes the ability of ArgumentAnalyst models to correctly tell apart relevant premises from mere distractors in the EntailmentBank task2 dataset for two generative chains (straight, which directly outputs reason statements, and hermeneutic cycle, which tries to reconstruct the argument first and uses both source text and argument to identify reasons), and compares this with the performance of EntailmentWriter (scores from Dalvi et al., 2021). The results, shown separately for arguments with a specific number of inference steps, let us draw three conclusions: First, ArgumentAnalyst outperforms Entailmen-tWriter on difficult problems with more than 4 inference steps / sub-arguments.
Second, using the sophisticated chain hermeneutic cycle improves predictive performance compared to the simple straight chain.
Third, the chain hermeneutic cycle (unlike straight) generally benefits from intermediary pretraining on AAAC -caveat: not so for arguments with more than 4 steps. This latter observation might be due to the fact that the AAAC02 corpus, by construction, doesn't contain arguments with more than 4 steps, so that pre-training biases the model towards shorter arguments.
In a third experiment we explore the following hypothesis: Informative higher-order evidence. The degree to which ArgumentAnalyst struggles in reconstructing a given argument (presented in the source text) as logically valid is a reliable in-dicator for whether the original argument is fallacious or not.
To test this hypothesis, we apply ArgumentAnalyst (trained on AAAC02, see above) to the RuleTaker data as imported into the DeepA2 framework (see Section 4): ArgumentAnalyst produces -by means of 13 generative chains -comprehensive reconstructions, to which the systematic and exegetic metrics are applied. RuleTaker contains an equal share of arguments whose conclusions follow from (label=valid), contradict (label=contradiction), or are independent of (label=neutral) the corresponding premises. Now, informative higher-order evidence would allow us to correctly predict these labels. And this is exactly what we observe: First, if reconstructions of one and the same source text which are independently generated with different chains agree (disagree), then the original argument tends to be valid (invalid). Second, by training simple classifiers on our argumentative metrics and further properties of the reconstructions, we robustly achieve a predictive accuracy 10% above the random baseline. While this is far below the SOTA results of tailor-made RuleTaker (Clark et al., 2020) and ProofWriter (Tafjord et al., 2020) models on this data, our findings nonetheless confirm the above hypothesis.

Conclusion
In this paper, we have presented and implemented a multi-angular, modular framework for deep argument analysis (DeepA2). It allows for defining a large variety of generative modes by combining different dimensions of the data. These modes, in turn, can be concatenated into complex generative chains. ArgumentAnalyst -a text-to-text model set up and trained within the DeepA2 frameworkyields plausible reconstructions of argumentative texts. Our empirical findings vindicate the overall framework and highlight the following advantages of a multi-angular, modular design in general: First of all, modular chains may emulate established, well-proven, typically piece-meal, scholarly techniques for text analysis (heuristics), which hence may provide normative, methodological guidance in setting up NLP systems. Secondly, by defining and implementing different modular chains, and investigating the plurality of generated solutions, one can systematically explore the system's uncertainty as well as the tasks's underdetermination. Thirdly, monitoring the sys-tem during modular computation yields diagnostically useful information (e.g., intermediary results) which not only describes the model's performance on the given problem, but which additionally allows us -as higher-order evidence -to characterize (e.g., classify) the original problem in the first place. Fourthly, breaking down a complex task into sub-tasks with intermediary results that can be further processed and re-combined helps to overcome input size limitations of neural language models. Fifthly, modular generation with meaningful modes allows users to follow the system, comprehend generated solutions, verify sub-steps and detect errors -the NLP system becomes a transparent, explainable AI (Miller, 2019). Finally, modular NLP systems as described by DeepA2 may be connected to a user-interface which promises fine-grained interactive control of modular generations and seamless cognitive cooperation of AI and human experts in analysing texts.

A Synthetic Argument Data
The AAAC datasets used in this study are publicly available via Huggingface's Hubhttps: //huggingface.co/datasets/debatelab/aaac -where the construction of the datasets is documented meticulously. A synthetically generated AAAC record, which nicely illustrates the underdetermination of argument reconstruction, with two implicit premises, one distracting statement and a simple (one-step) argument (formatted as presented to the model): source: It is not the case that Tracy is not an admirer of Fullerton and Tracy has seen La Habra. Plus, if someone loves Chico, then they haven't visited Monterey, owing to the fact that loving Laguna Beach is sufficient for not having visited Monterey. reasons: loving Laguna Beach is sufficient for not having visited Monterey (ref: (2)) conjectures: if someone loves Chico, then they haven't visited Monterey (ref: (4)) argdown: (1) If someone is an admirer of Chico, then they are an admirer of Laguna Beach or a visitor of Stockton.
(2) If someone admires Laguna Beach, then they haven't visited Monterey.
(3) If someone has visited Stockton, then they haven't visited Monterey.

B Training Set-up
By interpreting a generative mode as a sequenceto-sequence task, we may translate a multi-angular DeepA2 dataset (e.g., AAAC01) into a multi-task sequence-to-sequence format, on which a sequenceto-sequence model can be trained. For each record in the multi-angular DeepA2 dataset, we randomly sample 14 modes in accordance with the weights provided in Table 3 and add, for each mode, a corresponding sequence-to-sequence record to the training data. This results, for AAAC01, in a sequence-tosequence training dataset with 14 × 16.000 records.

C Iterative Prediction with Generative Chains
Generative chains are implemented with a dynamic dictionary (9 keys, corresp. to the dimensions of DeepA2 data), which is initialized with the source text, provides input for the generative modes, and is updated after each generative step with the mode's generated output. Output is generated with beam search decoding and beam width 2. Table 4 displays all generative chains we resort to in this study, all of which are used in the first experiment. The second experiment makes use of chains 1-11. The third experiment deploys chains 1-13. Table 5 assesses ArgumentAnalyst's reconstructions on specific subsets of the AAAC02 dataset (defined in Section 4) for three representative generative chains. Table 6 details the performance of Argument-Analyst on the entire AAAC02 dataset as measured by tailor-made argumentative metrics. Table 7 shows the corresponding performance on out-of -sample eval data AAAC01.

D Additional Results
Distinguishing four mutually exclusive subsets of AAAC02, Tables 8-11 detail the the quality of ArgumentAnalyst's reconstruction for easy and difficult problems. Tables 12-15 present the corresponding out-of-sample performance on the equally partitioned AAAC01 dataset (eval split).