{'fraction_non_alphanumeric': 0.034336709177294325, 'fraction_numerical': 0.010283820955238809, 'mean_word_length': 5.249145424357847, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
IPAL: Breaking up Silos of Protocol-dependent and Domain-specific Industrial Intrusion Detection Systems

The increasing interconnection of industrial networks exposes them to an ever-growing risk of cyber attacks. To reveal such attacks early and prevent any damage, industrial intrusion detection searches for anomalies in otherwise predictable communication or process behavior. However, current efforts mostly focus on specific domains and protocols, leading to a research landscape broken up into isolated silos. Thus, existing approaches cannot be applied to other industries that would equally benefit from powerful detection. To better understand this issue, we survey 53 detection systems and find no fundamental reason for their narrow focus. Although they are often coupled to specific industrial protocols in practice, many approaches could generalize to new industrial scenarios in theory. To unlock this potential, we propose IPAL, our industrial protocol abstraction layer, to decouple intrusion detection from domain-specific industrial protocols. After proving IPAL's correctness in a reproducibility study of related work, we showcase its unique benefits by studying the generalizability of existing approaches to new datasets and conclude that they are indeed not restricted to specific domains or protocols and can perform outside their restricted silos.


INTRODUCTION
Digitized industries play an essential role in today's society across various domains such as water treatment and distribution, power delivery, chemical processing, or manufacturing [59]. To realize their functionality, industries rely on (near) real-time communication of process values and commands using a large pool of specialized industrial protocols, such as Modbus or EtherNet/IP [57]. The increasing reliance of these protocols on the Internet Protocol (IP) to foster automation, remote control, and optimized processes, however, also moves traditionally air-gapped industrial networks closer to cybersecurity threats [59]. Indeed, a growing number of cyber incidents with detrimental environmental damage and risk to human life [55,116] highlight the importance and urgency of adequately protecting industrial communication networks.
As a non-intrusive, retrofittable, and cheaply deployable security solution, Intrusion Detection Systems (IDSs) offer a great additional layer of defense that is already well-established in traditional environments, such as office or data-center networks. While these traditional IDSs (e.g., Zeek [94]) offer a certain benefit to industrial networks, they also reach their limitations: Stealthy attacks differing just enough from normal operations to cause serious harm are hardly detectable by them [114]. Meanwhile, industrial protocol's rather deterministic and predictable nature offers unique opportunities for process-aware or semantic intrusion detection [49]. Hence, Industrial IDSs (IIDSs) leveraging these regularities are necessary to protect industrial networks. Consequently, a large research community has gathered [34,49,58,65,84,90,98,114], proposing specialized solutions for a wide variety of industrial scenarios.
However, although different industrial domains exhibit similar predictable communication and process patterns, research on IIDSs is highly tailored to specific domains and communication protocols. While technical and engineering reasons for such tight coupling might exist, we find this strong interdependence rather surprising and contrary to intuition. From an IIDS perspective, all industrial protocols essentially exhibit the same distinct characteristics: They primarily exchange sensed data and commands using a small set of well-defined communication patterns [121]. Additionally, while specifics of benign and malicious behavior change between scenarios, these can mostly be trained prior to deployment.
Although there exists no fundamental conceptual reason for the dependence of IIDSs on specific industrial protocols, IIDSs still specialize to few protocols, limiting their applicability to single arXiv:2111.03438v2 [cs.CR] 11 Jul 2022 scenarios. Moreover, besides contrary claims [33,46,61,68], it is seldom tested whether a specific IIDS even works outside the precise scenario and industrial protocol it was developed for. Consequently, the overall progress in research on IIDSs is slowed down due to niche solutions that do not improve on prior work from other industrial fields, wasting tremendous advancements in detecting cyber attacks on industrial systems.
Contributions. To break up the prevalent research silos of protocol-dependent and domain-specific IIDSs and ultimately transfer the advancements of IIDS solutions to new industrial scenarios, we make the following contributions in this paper: • Beginning with a survey of 53 IIDSs from related work, we identify the characteristics of industrial communication they rely on to detect anomalies and attacks. Thus, we theoretically identify the potential for protocol-independent and domain-agnostic intrusion detection across existing IIDSs (Sec. 3). • To turn these results into practice, we introduce the IPAL industrial protocol abstraction layer. IPAL sits between industrial protocols and IIDSs, captures all aspects relevant for intrusion detection in a unified representation, and thus facilitates protocolindependent and widely applicable IIDSs (Sec. 4). • We showcase the practical applicability and correctness of IPAL in a reproducibility study of eight IIDSs from related work. We show that IPAL provides all required information, and we simultaneously reproduce scientific results as an independent party, an important but often neglected step in research (Sec. 5). • We illustrate the various benefits for IIDS research of IPAL in three case studies. With IPAL, we are able to compare different IIDSs, even from opposing research branches, study to what extent they generalize to new scenarios, and, for the first time, transfer the advancements of IIDSs to new protocols and domains (Sec. 6).
Availability Statement. To let IPAL become the foundation of future work on industrial intrusion detection, we make our source code available. This includes tools to transcribe industrial protocols [43] and existing datasets into IPAL [41], as well as IIDSs implementations from our reproducibility study [42] (to the extent permitted by licenses or authors).

INDUSTRIAL INTRUSION DETECTION
As a foundation for our work, we begin by recapitulating the core idea behind intrusion detection and what makes it special in industrial networks. Based on this, we identify key limitations of industrial intrusion detection research that prevent the broad application of improvements achieved for isolated industrial scenarios.

Intrusion Detection
Intrusion (or anomaly) detection is the art of automatically uncovering cyber attacks or other suspicious activity by passively monitoring a system's behavior, e.g., w.r.t. communication [110,119]. The core assumption behind Intrusion Detection Systems (IDSs), which implement intrusion detection, is that cyber attacks lead to distinctively different (anomalous) system behavior than benign activity. As their passive nature allows retrofitting them to existing deployments easily, IDSs are popular to complement preventive security measures, i.e., integrity or authentication.
Traditionally, IDSs were designed for office, server, or data-center networks with largely fluctuating traffic and usage patterns [108]. Even though this makes it challenging to define benign activity comprehensively, there is a decent understanding of how individual attacks are performed on these networks, as attacks and attack patterns are typically used across many different networks, e.g., by widely-used malware. Thus, standard methods to reliably detect such (known) attacks are signature or rule-based approaches, which compare user and network behavior against a list of attack indicators, with well-established implementations such as YARA [117], Suricata [44], Zeek [94] (previously Bro [106]), and Snort [100].

Promises of Industrial Intrusion Detection
Intrusion detection is particularly attractive for industrial networks, as retrofitting other (active) security measures often results in costly hard-or software modifications, major downtime, or the need to redo acceptance tests. W.r.t attack and system behavior, however, we observe exactly opposite conditions for intrusion detection in industrial networks: While rather deterministic industrial tasks lead to more predictable communication patterns [57], attacks are often more subtle and scenario-specific, severely limiting predictability and thus signature or rule-based approaches.
Most importantly, attacks against industrial networks are often specifically designed for a single or few targets and hardly generalize across domains (as, e.g., malware does), making it hard to catch them by predefined rules. Furthermore, industrial settings are susceptible to subtle attacks [114,126] where attackers send legitimate traffic (as allowed by rules) but at the "wrong" time to trip the system into a dangerous state. Thus, IDSs designed for typical IT networks cannot simply be transferred to industrial settings [128].
Still, industrial scenarios provide unique opportunities to detect intrusion and anomalies through different means, e.g., the tight coupling of industrial processes such as water treatment, power delivery, chemical processing, or manufacturing, with the physical environment via sensors and actuators. To this end, process-aware or semantic IIDSs incorporate information from the physical and/or communication level to uncover highly specialized and subtle attacks and anomalies. Therefore, IIDS approaches typically "learn" a model of benign behavior for a specific facility and alert deviations from this model. Such models may incorporate additional system knowledge, e.g., that certain actuator combinations never occur, resulting in an IIDS not requiring individual attack knowledge.

Limitations of Current IIDS Research
Given these promises of IIDSs to detect sophisticated and safetycritical attacks on industrial networks and processes, a plethora of different research fields for IIDSs in various industrial domains has been established [49,90,114], including industrial control systems [34,58,65], intra-vehicular communication [84], or water treatment [98]. Across these various research fields, we find that individual IIDSs are tailored to specific domains and communication protocols. Consequently, advancements made for one scenario often cannot generalize to other industrial protocols or domains, limiting their widespread use, slowing down innovation, and the overall progress in securing industrial networks. We identify two core problems that lead to these limitations of current IIDS research. (a) The current state-of-the-art requires a tailored implementation of IIDSs for each protocol.
(b) Protocol-independent IIDSs would allow separating protocol parsing and IDS development. Figure 1: To overcome the limitations of current IIDS research, protocol-independent IIDSs promise to break up the strong interdependence between IIDS approaches and the specific industrial protocol and domain they operate in.
First, compared to traditional computer networks, we observe a huge heterogeneity in industrial domains, each with unique and custom communication protocols for legacy reasons. Thus, current IIDSs generally focus on one specific domain and industrial protocol combination, presenting yet another (novel) approach in isolation without building on top of prior research findings. E.g., two similar approaches independently propose to use Probabilistic Suffix Trees for Modbus [123] and IEC-104 [79], highlighting that work is done twice across different industrial scenarios.
Second, IIDS research across different industrial domains suffers from an evaluation bias, where vast inconsistencies in used evaluation methodologies make comparisons across publications hardly possible [49,78,114]. Moreover, the use of only a few datasets (e.g., SWaT [50]) or many private ones steers the collective research efforts in a direction where solutions are optimized and biased towards specific scenarios. This problem is already known for traditional IDSs [92] but is even more severe in industrial settings, where systems expose very narrow yet domain-specific behavior [111].
Consequently, although different approaches claim broader applicability (without proof) [33,46,61,68], the actual generalizability of proposed IIDSs to new settings within the same domain or across domains remains mostly unexplored [36,78]. Thus, the various research streams in the area of IIDSs are indeed disjoint. As a result, the overall research field is slowed down, as new approaches are designed for niches without improving on previous research and are biased to datasets from specific scenarios. It thus remains unknown to what extent the tremendous advancements in the detection of cyber attacks on industrial systems can or cannot be generalized and transferred from their isolated niches to a broader scale and thus sustainably increase security for many industrial networks.

THE CASE FOR PROTOCOL-INDEPENDENT INDUSTRIAL INTRUSION DETECTION
Currently, the IIDS research landscape is scattered across heterogeneous industrial domains (cf. Sec. 2.3), leading to a situation as depicted in Fig. 1a, where IIDSs are developed in isolation for specific industrial protocols and domains. Consequently, research on IIDSs is limited w.r.t. generalizability, evaluation scenarios, widespread use, and coherence between research streams. To overcome these limitations, in this paper, we make a case for protocol-independent IIDSs, as shown in Fig. 1b, where IIDSs do not have to care about the underlying protocol anymore while simultaneously being transferable to various scenarios with little to no additional effort. Our proposal for protocol-independence in IIDSs is motivated by two observations: First, for the purpose of intrusion detection, essentially all industrial protocols show similar functionality, i.e., the exchange of sensor readings and actuator commands. Second, in principle, the underlying detection approaches of IIDSs work similarly as they each know or learn a model of a specific industrial process's behavior and detect anomalies by comparing their models to the monitored physical process.
Given these observations, we postulate that a large portion of existing work is not necessarily bound to the specifics of the industrial domain and protocol they are designed for and may generalize well to other domains and protocols. Thus, we are convinced that tackling the diversity in industrial protocols is key to breaking up the isolated silos in the IIDSs research landscape and hence allows the wide-ranging application of IIDSs across domains and protocols.
To realize this goal, we first highlight the benefits of protocolindependent IIDSs for the research community (Sec. 3.1) before we survey 53 state-of-the-art IIDS approaches to analyze their potential for generalizing across industrial domains and protocols (Sec. 3.2).

Benefits of Protocol-Independent IIDSs
Although IIDSs are specifically tailored to a single industrial protocol and domain nowadays, this does not imply that their benefits are necessarily restricted to a single combination. Protocol-independent IIDSs (cf. Fig. 1b) instead promise to operate detached from specific industrial protocols or domains and thus exhibit several advantages.
First, this would allow applying IIDSs to multiple industrial protocols within the same domain. E.g., power grids that historically rely on different protocols such as IEC-104 (covered by IIDSs [40,[79][80][81]) or DNP3 (covered by [46,96]) would no longer require separate IIDSs. Moreover, if the underlying industrial protocol could be exchanged, the potential to generalize detection methodologies across domains not initially thought of would become directly accessible.
Also, by designing IIDSs with protocol-independence in mind, developers no longer have to extract relevant data from a specific industrial protocol in a time-consuming and error-prone process. Instead, this process can be left to domain experts who know the ins and outs of specific network protocols, which has already proved beneficial for prominent intrusion detection datasets, e.g. SWaT [50] providing pre-processed process data from network traffic.
Finally, it becomes easier to reproduce results, validate findings in different scenarios, and fairly compare IIDSs based on common metrics, thus constituting to sound scientific experiments [49,113]. Protocol-independence allows to efficiently validate results from prior work and expand upon them, a step that is often neglected in research [18,113], especially when the original scenario is unavailable, flawed, or too narrow in scope.
Consequently, decoupling IIDSs from the underlying industrial protocol promises to address the transferability of IIDSs and their generalizability to new scenarios. It can also streamline future research efforts on IIDSs by enabling extensive evaluations, validations, or removing potential biases with regard to specific protocols or domains. However, so far, it remains open how big the potential for realizing protocol-independent IIDSs actually is.

Potential for Protocol-Independent IIDSs
To identify the potential for protocol-independent IIDSs, we first study to which extent existing IIDSs can generalize to new industrial protocols and domains, which is directly related to the protocol and process information they operate on. However, while existing surveys identify the need for generalizing IIDSs [36], they do not investigate whether existing work shows potential in this regard. Bridging this gap, we survey 53 IIDSs covering a wide range of detection methods, industrial protocols, and domains. We specifically focus on identifying which information they use for detection.
As our goal is to identify the potential for protocol-independent IIDSs, we specifically target process-aware or semantic IIDSs (leveraging knowledge about the physical process), as these are mostly independent of the used protocol. Contrary, we exclude IIDSs leveraging specific characteristics of physical communication [71,72].
Survey Structure. In the following, we structure and discuss our survey, as depicted in Tab. 1, along two broad categories of processaware and semantic intrusion detection: Communication-based approaches (Sec. 3.2.1) which incorporate characteristics of industrial network traffic and process state-aware approaches (Sec. 3.2.2) restricted to the current (physical) state of an industrial setting.
For each approach, we identify the input format, i.e., network packets (P) or aggregated process states (S). We complement this with information an IIDS operates on, i.e., (i) timing information of exchanged packets, (ii) knowledge about communication partners and type of communication, as well as (iii) which values are communicated, also including the process state. We identify whether additional packet features are required (e.g., packet lengths), whether an externally provided process model is required, as well as whether it trains on anomalous (A) or benign-only (B) traffic. Finally, we list the number of evaluation scenarios, the number of IIDSs a novel approach compares to, the intended industrial domain and protocol, as well as the availability of source code and evaluation dataset.

Communication-based Intrusion
Detection. Different streams of research propose to leverage the inherent communication patterns found across industrial domains. We identify three categories of how regularities caused by the periodic distribution of, e.g., new sensor measurements, can be utilized for anomaly detection. First, irregularities in timings indicate potential malicious activity. Secondly, the order of message sequences also exposes repeating patterns that can be learned to identify expected normal communication. Finally, a class of classifier-based approaches grade each packet as benign or anomalous based on a set of predefined features.
Our analysis shows that all three categories are used across different industrial domains. Analyzing the type of information these IIDSs operate on, we observe that timing-and sequence-based approaches mostly rely on just four data types: timing information, communicating entities, message types, and often detailed information about transmitted data. In addition, classifier-based approaches often incorporate the exchanged values. Rarely do such IIDSs use further packet features, e.g., packet sizes or checksums for their classification. Still, none of the approaches take advantage of protocol-specific information, such that, in theory, they could be employed for other protocols and scenarios (for Chu et al. [31], this claim cannot be validated as their automatic feature extraction approach could potentially result in the use of arbitrary features).
Overall, communication-based approaches are not based on the particular structure and behavior of the underlying industrial protocol. Instead, despite IIDSs being tailored to specific protocols, they nearly exclusively use protocol-agnostic information to detect anomalies. Hence, in theory, virtually all considered communicationbased IIDSs work protocol-independent and thus show potential for deployment in a wide range of industrial domains.
However, such potential for protocol-independence remains mostly unexplored in practice. As a rare exception, Lin et al. evaluate their IIDS [81], checking for irregular timings between similar messages, for three industrial protocols. Still, they focus on a single domain (power grids) and thus do not examine generalizability. Generally, despite theoretical generalizability to other domains and protocols, communication-based IIDSs are not compared against each other or evaluated across different domains. Consequently, researchers cannot determine which IIDSs show the best performance and should be used as a benchmark for novel proposals, improved in future research, or deployed in practice.

Process
State-aware Intrusion Detection. In contrast to the communication-based approaches, process state-aware IIDSs solely base on (time-)series of industrial system's physical states. A state comprises the combined process information of all sensors and actuators aggregated over multiple packets to assess whether a process' physical state indicates an anomaly over time. Thus, IIDSs from this category do not rely on packet-specific information and already naturally abstract from many industrial protocol's characteristics. To identify attacks, process state-aware approaches leverage the repetitiveness and predictability of physical processes, e.g., Programmable Logic Controllers (PLCs) controlling a pump to keep a water reservoir's fill level within certain bounds [39].
While process state-aware IIDSs operate on a more restricted set of information than communication-based IIDSs, their detection methodologies show more diversity, which we broadly classify along four categories. The first category of approaches defines critical states of an industrial system, e.g., through externally provided system states by domain experts, and raises an alarm if such a state is reached. Complementary, the second category of IIDSs attempts to predict future states based on past observations, raising an alarm if the behavior significantly deviates from the prediction. The third category of approaches takes a more local view by investigating correlations between an individual or a small set of sensor readings over time, searching for indicators of anomalous activities, such as outliers from learned clusters. Finally, as for network-based approaches, a category of approaches train classifiers to distinguish between genuine and anomalous behavior based on process state.

PROPOSING IPAL -AN INDUSTRIAL PROTOCOL ABSTRACTION LAYER
Given the enormous potential for realizing protocol-independent IIDSs (cf. Sec. 3.2) and thus capitalize on their manifold benefits (cf. Sec. 3.1), we set out to turn this vision into reality. To this end, we propose IPAL, our design of an industrial protocol abstraction layer to decouple intrusion detection from domain-specific industrial communication protocols. As shown in Fig. 2 Sender of the network packet (e.g., IP and port) destination Receiver of the network packet (e.g., IP and port) message type Fine-granular industrial packet's message type activity Abstracted message type (e.g., request or response) responds to List of related IPAL messages this message responds to process data Arbitrary number of process variables and their values 1) operate on a broader set of information but still only rely on packets with (direct) influence on the physical process, such that other packets (e.g., diagnostic data or TCP handshakes) can be ignored. As detailed in the following, ten features suffice to represent industrial network packets in a common format while preserving all information required by process-aware IIDSs. Meta Data. The first observation in our survey regarding the abstraction is the incorporation of metadata across all IIDS categories. Thus, IPAL includes a packet's timestamp and length. In addition, a unique identifier (id) is included to disambiguate packets. Since several IIDSs require labeled training data differentiating benign and malicious packets, IPAL includes the malicious field.
Addressing Information. Besides meta data, communicationbased IIDSs require features to identify the communicating entities. Thus, IPAL includes the source and the destination of a single packet. These fields are arbitrary strings that, in most cases, represent an IP-port combination, but can be extended, e.g., by adding Modbus's unit identifier field, to further disambiguate devices. Likewise, destination can remain empty for broadcast protocols.
Message Identification. One relevant aspect found during our survey is that communication-based IIDSs leverage message types of industrial packets on different levels of abstraction (cf. Sec. 3.2.1). Some IIDSs [23,24] analyze the sequence of specific, reoccurring messages, and further IIDSs [81] only differentiate between requests or responses. Therefore, IPAL captures a fine-granular message type, as well as a more generic activity. Here, four activities (requests, commands, and their respective answers) suffice [121], while the message type closely matches the differentiation schemes used by the underlying protocols. Lastly, the responds to field lists the ids of all IPAL packets a given message is a response to.
Process Data. The most important feature used by both types of IIDSs is the process state. In IPAL, the field process data collects all process variables as well as their current values as communicated within a single packet. Depending on the transcribed packet, process data thus can range, e.g., from a single temperature to an array of sensor readings and actuator commands. As process state-aware IIDSs demand a snapshot of the entire system's state, we support state aggregation over several IPAL messages to realize a similar extraction as, e.g., through active polling [45].
A few rare cases exist where additional features (e.g., checksum validity tests [31]) are included in the decision-making process of otherwise process-aware IIDSs. These checks are, however, already covered by traditional IDS (e.g., Zeek [94]) and do not contribute to information about the physical process monitored by process-aware IIDSs, and can thus be skipped to avoid redundant work. Consequently, the ten features of IPAL suffice to preserve all information required by the process-aware IIDSs surveyed in Tab. 1.

Transcribing Industrial Protocols
IPAL, our abstract representation of industrial network traffic, promises to decouple process-aware intrusion detection from underlying industrial protocols. Yet, to realize protocol-independent IIDSs, real-world network traffic must be transferred into this abstract representation. As shown in Fig. 2, we designed a transcriber [43] that automates this conversion. While integrating specific industrial protocols into IPAL requires a deep protocol understanding, this work needs to be performed only once.
We have already incorporated nine industrial protocols (Modbus, EtherNet/IP, MQTT, IEC-104, DNP3, S7, IEC 61850-GOOSE, IEC 61162-450, and NMEA 0183) used in popular datasets. Thus, future IIDS research can focus on modeling industrial processes, while a broad set of datasets from different industrial scenarios is readily available for evaluation through IPAL [41]. In the following, we discuss challenges faced while implementing our transcriber.
Most features of IPAL can be directly extracted from network packets. However, certain protocols require special attention. E.g., in ModbusTCP, multiple sub-devices may operate behind a single TCP connection. Thus, source and destination need to include not only the IP and port but also the device's unit identifier. Also, while Modbus's function code directly corresponds to the message type, the activity has to be derived from the traffic direction, as function codes for requests and responses are identical.
A more challenging aspect is the extraction of the actual process values from industrial communication because of their encoding. To this end, transcribers can contain custom rules to extract and post-process process values from observed communication, similar to StreamPipes [125]. Such rules are small code snippets that allow, e.g., to interpret two 16-bit registers (the biggest data type in Modbus) as a single 32-bit float. Thus, we can integrate the necessary flexibility to define the interpretation of potentially multiple related variables and even directly annotate the interpreted value with a more descriptive and human-readable name.

Developing IIDSs with IPAL
To leverage its benefits, we provide a framework to implement protocol-independent IIDSs on top of IPAL [42]. We observed in our survey (cf. Sec. 3.2) that IIDSs are either based on communication traffic or on the global process state. As shown in Fig. 2, for communication-based IIDSs, our framework passes each packet to the IIDS after abstracting it into an IPAL message.
For process state-aware IIDSs, we need to aggregate multiple packets over time to obtain the current state of the entire process. Thus, our framework simply caches the process values included within the most recent IPAL messages. We considered different state aggregation methods, but simply outputting the most recent process values in regular intervals (e.g., each second) yields the best results and allows us to convert EtherNet/IP traffic included in a newer run of the SWaT testbed (without attacks) to a state that closely matches the pre-processed state information provided by the dataset authors. Furthermore, as some common datasets (e.g., SWaT [50]) already provide a state representation, our framework optionally allows restoring these pre-computed states directly.
Our framework enables the realization of communication-based and process state-aware IIDSs on IPAL messages. IPAL already supports nine industrial protocols from various domains. Thus, IIDSs developed for or adjusted to IPAL can be evaluated for various scenarios based on current and future datasets, accelerating research on IIDSs and moving them closer to deployment (cf. Sec. 3.1).

Limitations of Protocol Abstraction
Any form of abstraction involves the risk of introducing an information loss, as crucial features may be discarded. In our case, IIDSs taking advantage of a feature not covered by our abstraction would likely not be compatible with IPAL. However, considering that IPAL's features are derived from an extensive survey of 53 scientific publications (cf. Sec. 3.2), we expect the number of currently developed IIDSs that cannot be realized on top of IPAL to be low.
Still, our choice of features captured in IPAL could potentially narrow the scope of future research. However, for dominantly used datasets, we already observe a restriction in available information to pre-processed data, e.g., only providing timing and process-state information (cf. SWaT in Tab. 1), presumably to facilitate easy utilization of the dataset. Contrary to intuition, the adoption of IPAL could even lead to the availability of more features by encouraging the release of unprocessed packet captures, as IPAL provides the tooling to easily process raw data according to researchers' needs. In this vein, the IPAL framework is publicly available [42,43] such that the extracted feature set can be extended in the future if deemed necessary. A similar trend has been observed for Netflow [32], whose abstraction revolutionized network monitoring and introduced newer versions as the need for more features arose.
From a different perspective, protocol abstraction comes with the potential risk of introducing transcription or abstraction inaccuracies affecting the final detection performance. Yet, as we show in the following (Sec. 5), such inaccuracies are generally not introduced by IPAL, and the IIDS performance is independent of the use of our abstraction layer. Consequently, the value of IPAL homogenizing a split-up research landscape far outweighs the potential drawbacks arising from introducing an abstraction layer (so early) for the growing industrial intrusion detection research field.

PRACTICAL APPLICABILITY OF IPAL
We derived IPAL from the requirements of existing IIDSs and discussed its theoretical applicability for industrial network protocols. Now, we complement this viewpoint by showing IPAL's practical applicability and correctness by performing a reproducibility study of eight IIDS approaches from previous work and reproducing their evaluation results on top of IPAL. Thereby, we also contribute to reproducing scientific research results as an independent party. While deemed extremely important [18], this is often a challenging and tedious task with little reward and thus rarely performed [36,97].
IIDS selection criteria. To conduct a comprehensive reproducibility study, we set out to reproduce at least three representative approaches for each of the two major categories in our survey (cf. Tab. 1), covering all features provided by IPAL. Since we re-evaluate existing IIDSs, we rely on the availability of the original evaluation dataset or tools for attack generation. Also, as re-implementing IIDSs from scratch can be challenging, we focus on open source approaches and only implement IIDSs ourselves if necessary. Thus, our selection was driven by the availability of code or datasets.
Note that we reproduce evaluation results and thus do not invent new attack methodologies. This implies that the reproducibility results are not necessarily comparable, as the authors used fairly different evaluation procedures. Still, we will show in Sec. 6 how IPAL enables conducting a side-by-side evaluation of different IIDSs.
In the following, we briefly argue why we selected each IIDS, summarize its core idea, provide re-implementation and evaluation details, and compare our results to the original publication. Tab. 3 summarizes our IIDS selection and reproduction results.

Inter-arrival Time (Communication)
IIDSs from the communication-based category utilize data on a perpacket basis and consider features such as communicating entities and message types. An ideal representative for this category is the inter-arrival time (IAT) approach by Lin et al. [81], which utilizes periodic traffic patterns in industrial protocols such as Modbus, S7, and IEC-104 to detect, e.g., packet-injection attacks. To this end, the approach measures the mean inter-arrival time, i.e., the elapsed time between two packets of the same type (e.g., requests and responses), as well as the maximum temporal deviation (range) between packets with the same content and checks for timing violations.
As no implementation of the inter-arrival time approach was available, we re-implemented it based on the paper [81]. To reproduce the results, we use the only publicly available dataset from the original work [81], which evaluates S7 traffic [105]. As only the attack-free dataset was available, we performed an identical 1/10 train-test dataset split and injected "malicious" packets according to the authors' description. Our reproduced results on top of IPAL (cf. Tab. 4) closely match those of the original paper. We attribute the remaining differences to randomness and minor implementation uncertainties w.r.t. to attack generation. Overall, IPAL provides everything needed to re-implement and reproduce this IIDS.

DTMC (Communication)
As an additional communication-based IIDS, we consider an approach relying on Discrete-time Markov Chains (DTMCs) modeling the sequence of network packets for a single connection [40]. This approach leverages a core property of industrial networks: Devices such as PLCs perform several communication steps in series, e.g., reading values from remote devices and adjusting setpoints later on, resulting in periodic packet sequences repeating in similar order. We adapted the DTMC approach's publicly available source code [30] to operate on top of IPAL. While a full reproducibility study is infeasible without the original dataset, we can still validate whether the original implementation and the version on top of IPAL produce identical results when presented with the same input. To this end, we generated our own IEC-104 network trace using a simulation framework for power distribution grids [56] and the attack tool by the authors to add equivalent attacks to our trace [29]. We performed the same procedure as in the original paper [40], i.e., a 50/50 train/test split, applying the attack tool ten times for each attack type and counting the number of state and transition violations compared to the trained DTMC. While not comparable to the original evaluation, IPAL correctly preserves all information resulting in identical attack coverage compared to the original DTMC implementation, as shown in Tab. 5.

Classifiers (Communication)
Complementing the IAT and the DTMC approach, IIDSs using machine-learning classifiers may operate on additional information such as the packet length. Furthermore, they additionally require traces of past attacks for training. Thus we selected the work of Perez et al. [83] as a representative for this category. In their work, they compare three different machine learning algorithms against each other (Random Forest (RF), Support Vector Machine (SVM), and Bidirectional Long Short Term Memory (BLSTM)).    For our reproducibility study, we adapted the implementations of the machine-learning classifiers published by the authors [99] to operate on top of IPAL. Furthermore, we use the same evaluation dataset [87], hyper-parameters, and pre-processing steps as in the original paper [83]. In Tab. 6, we compare the original results of the paper with our replication of the original evaluation as well as the same evaluation on top of IPAL. All three approaches perform equivalently, and the minor deviations can be inferred from randomly selecting a different train and test dataset. Thus, for these three approaches and the previous two communication-based IIDSs, we showed that IPAL correctly preserves all relevant information to achieve equivalent attack detection performance.

PASAD (Process state)
In contrast to the thus far considered IIDSs, Process-Aware Stealthy Attack Detector (PASAD) [16] detects structural changes in data series and thus represents the process state-aware category. PASAD's core idea is that legit data series span a vector space, and the distance to the mean of all training vectors indicates an anomaly. An alarm is raised if the distance (departure score) exceeds the maximum observed distance during training (threshold).
Two implementations of PASAD are available, the authors' version in Matlab [62] and a re-implementation in Python [97]. We realized PASAD for IPAL based on the Python version and used the original Matlab implementation to generate reference data. PASAD was trained and evaluated visually on output logs of the TEP [35], parts of the SWaT datasets [50], and private data of a real water treatment facility. We verify our implementation on the public TEP datasets and exemplary discuss one example in Fig. 3. The results obtained on top of IPAL are identical to the original implementation [62], and both detect the attack simultaneously. By reproducing these results, we show that IPAL is able to preserve all information required by a process state-aware IIDSs such as PASAD. 20

Seq2Seq-NN (Process state)
Extending on the idea of PASAD, the approach by Kim et al. [68] requires knowledge about multiple actuators and sensors to predict the expected future states of an industrial system. More specifically, sequence-to-sequence Neural Networks (Seq2Seq-NN), known from language translation, are provided with the recent history of system states (e.g., the last 99 seconds) and predict the next state. Then, the distance between the predicted and actual state is calculated, and the error aggregated over time, resulting in an anomaly score. If the anomaly score exceeds a threshold, the IDS raises an alarm. For our Seq2Seq-NN implementation on top of IPAL, we adapted its publicly available version [67], extracted the original anomaly scores as a comparison baseline, and used the authors' pre-trained models for the evaluation on IPAL. We exemplarily compare the original implementation and the one on top of IPAL w.r.t. their ability to detect an attack on stage 3 of SWaT [50] in Fig. 4. There the anomaly score is identical, and the attack is detected simultaneously, showing IPAL correctly providing all information for Seq2Seq-NN.

TABOR (Process state)
Complementing PASAD and Seq2Seq-NN, TABOR focuses on one sensor and models the impact of actuators on this sensor [82]. To this end, TABOR combines three distinct detection approaches into one IIDS: An out-of-alphabet check alerts if unseen data or data out of regular boundaries is observed, a timed automaton captures the sequence and duration of linear sensor value segments to enable anomaly detection, and a Bayesian network couples sensor segments with actuator states to identify unexpected combinations.
We were able to retrieve the implementation of TABOR from the authors for our re-implementation on top of IPAL. TABOR was evaluated on the SWaT dataset [50], which was divided into smaller logically coherent units (models). Here, we concentrate on reproducing model 1 as its intermediate results in the original paper [82] Table 7: TABOR, on top of IPAL, detects the identical nine attack scenarios (TP) with slightly different coverage percentage (CP) and penalty scores (PS) (cf. Tab. 2 in [82]).
ease verifying the correctness of our implementation. We achieved equivalent results for our version of TABOR on top of IPAL (cf. Tab. 7). Our re-implementation scores are identical in true positives (TP) and false positives and only deviates marginally within the coverage percentage (CP) and penalty scores (PS), probably due to minor temporal deviations in the alarms. We confirmed manually that the alerts overlap between the original and our implementation. Concluding, IPAL provides the necessary information to realize even complex, combined process state-aware IIDSs.

Summary and Lessons Learned
To showcase the practical applicability and correctness of IPAL, we performed a reproducibility study of eight IIDSs covering the two IIDS categories with at least three representatives (cf. Tab. 1). By (re-)implementing a diverse set of IIDSs on top of IPAL, we have shown that IPAL provides all information required by various types of IIDSs. By reproducing previous work's results, we have shown that IPAL operates correctly and neither reduces functionality nor detection rates (cf. Sec. 4.4). Consequently, IPAL provides a reliable foundation for protocol-independent IIDSs, thus capitalizing on the benefits of an abstract representation (cf. Sec. 3.1). As indicated in Tab. 3, successfully reproducing others' research requires the availability of the original implementation. Only in rare cases does an evaluation dataset suffices. While we were able to successfully reproduce seven IIDS approaches (and even achieve fully identical results for two of them), we could not reproduce the evaluation results of the DTMC approach [40] since the evaluation dataset was not available to us. Still, as the original source code and attack tool were available, we could use another dataset to show that our re-implementation on top of IPAL produces exactly the same results as the original implementation.
While detection performance is of utmost interest, adding abstraction layers, such as IPAL, introduces overheads. For a fast IIDS such as DTMC, we exemplary compared the execution time against the original implementation. IPAL yields an overhead of just 10.1% for the benefits of increased applicability. This overhead is expected to shrink for more complex IIDSs.
Overall, the considered eight IIDSs not only rely on the requirements derived from our survey (Sec. 4) but also cover four distinct protocols (Modbus, EtherNet/IP, S7, and IEC-104) from four datasets [35,50,87,105]. Besides contributing to the important task of reproducing scientific research results as an independent party [18,36], we lay the foundation to ease the protocol-independent development, testing, and evaluation of IIDSs by providing a rich foundation for broad evaluations.

Generalizing Communication-based IIDSs
The first case study demonstrates IPAL's ability to transfer communication-based IIDSs to new and realistic scenarios. Therefore, we study the two approaches IAT [81] and DTMC [40] from our reproduction study (Sec. 5). Both were previously not compared against each other and were evaluated on artificial attacks retroactively integrated into network traces. Thus, their effectiveness in realistic scenarios is currently unknown.
To this end, we utilize novel datasets from realistic industrial systems not yet covered by both IIDSs. The first dataset resembles a Denial of Service (DoS) attack against a power grid communicating with the IEC 61850-GOOSE protocol [20]. In the same setting, more advanced attacks are performed based on a combination of message modification, injection, and replaying, comprising our second dataset [20]. Finally, we look at a Machine-in-the-Middle (MitM) attack dataset based on ARP poisoning to manipulate the data within Modbus packets [109]. Note that these datasets already contained these attacks and were specifically designed for evaluating the performance of security solutions. Our transcriber (cf. Sec. 4.2) transparently translates the GOOSE and Modbus network traces into IPAL, obviating the need to modify the existing IIDS implementations. Tab. 8 summarizes the results on these three new datasets.
DoS. All IIDSs detect the packets injected by the DoS attacks to a high degree. For the IAT's mean model, these results are expected, as this attack is similar to the artificial flooding attack evaluated in the original publication [81]. Surprisingly, even the IAT's range model achieves good detection rates, even though it performed worse in the artificial scenario (cf. Tab. 4). However, the DTMC achieved the best performance, classifying all packets correctly despite no similar attack being discussed in the original paper outperforming an IIDS specifically designed to detect such attacks. Advanced. The second scenario considers advanced data manipulation attacks against a power grid similar to the artificial attacks evaluated in both original publications [40,81]. Still, neither of the IAT models can sufficiently detect these attacks, despite performing well in the artificial attacks originally evaluated. This behavior can be explained by the low number of data manipulations and variable retransmission timings of the GOOSE protocol, leading to a larger variety of benign inter-arrival times. On the other hand, DTMC achieves great detection rates, confirming the results from the original publication in a new industrial domain and protocol.
MitM. Finally, we consider a MitM attack manipulating data received by industrial controllers [109]. While such an attack is a realistic threat in practice, it has neither been considered by IAT nor by DTMC thus far. Still, the IAT's mean model performs especially well in this scenario without issuing any false alarm. Contrary, DTMC is not able to detect a single malicious packet since this attack does not change the sequence of observed messages.
These results indicate that artificial attacks, retroactively integrated into a dataset [40,81], do not necessarily carry over to more realistic attack scenarios, even when considering a similar attack behavior. Yet, these new attack types can be detected well in some instances. This observation highlights that IIDSs have to be studied more extensively to understand which attacks they can detect and which not. Additionally, our case study highlights that one IIDS does not necessarily have to be superior in every attack scenario. Thus, cooperative IIDSs, i.e., several detecting specific attack classes, may improve overall performance (cf. Sec. 6.3). Finally, we showed the adaption from one domain or protocol to another scenario, confirming the enormous potential for protocol-independent and domain-agnostic IIDSs as previously identified in Sec. 3.1.

Generalizing Process state-aware IIDSs
Our second case study focuses on process state-aware IIDSs. Since the generalizability of IIDSs has not been evaluated sufficiently before (cf. Sec. 2.3), we show how IPAL eases the adaption of IIDSs to new domains and thus understand how IIDSs transfer to new scenarios. Furthermore, as Urbina et al. [49] already observed, approaches are insufficiently compared in the literature. E.g., PASAD is evaluated only visually [16], while TABOR defines its own metrics [82], and Seq2Seq-NN counts an attack as detected even 15 minutes after the attack ended [68] (cf. Sec. 5). Therefore, we evaluate these approaches against each other in a detailed comparison.
From our reproducibility study, we have three process stateaware IIDSs (TABOR [82], PASAD [16], and Seq2Seq-NN [68]) at hand, which were all evaluated on the SWaT dataset [50]. With IPAL, we can apply these to two new datasets (WADI [7] and HAI [104]). Fig. 5 depicts the alerts of all three approaches on these three datasets and Appx. A summarizes further in-depth results.
SWaT. We compare the IIDSs on the first stage of the SWaT dataset, on which they were initially evaluated. Here, we see significant differences between the operation of each IIDS. TABOR detects 9 attacks, most of which are captured by Seq2Seq-NN too. Seq2Seq-NN detects 10 attacks while still having 17 false alarms. PASAD can detect 6 scenarios, some of which were neither detected by TA-BOR nor by Seq2Seq-NN. Besides detection performance, we also notice differences in how these IIDSs emit alerts. PASAD buffers recent states internally, and Seq2Seq-NN accumulates deviations over time, delaying alerts for both approaches. Contrary, TABOR has great precision w.r.t. the attack time, yet alarms are announced retrospectively due to internal segmentation. Such differentiation only show in an in-depth side-by-side analysis as enabled by IPAL.
WADI. While the WADI dataset [7] is not unknown (cf. Tab. 1), none of the three approaches evaluated it. WADI is similar to SWaT, and we can show how simple transferring these IIDS to a similar domain with IPAL is again considering the first process stage of WADI only. Here, TABOR detects 4 scenarios, detects PASAD only 2 attacks and Seq2Seq-NN 8 scenarios. During the training of PASAD, computational limits were reached due to the larger dataset size, which might contribute to the lower detection quality. Still, the initial results of transferring an IIDS to a new dataset with IPAL are promising without the need to design new IIDSs from scratch.
HAI. After transferring these IIDSs to a similar domain, we now study how they generalize to other industrial settings. Thus, we consider the fairly new HAI dataset [104] based on a hardware-inthe-loop simulator. As some stages of HAI are of a similar domain as SWaT and WADI, we consider the remaining boiler process used for power generation. This dataset is much noisier than SWaT or WADI and less seldomly exhibits clear repetitive patterns challenging all IIDSs. TABOR does not generalize well, as it flags most parts of the dataset as an anomaly, while Seq2Seq-NN detects 2 scenarios with 5 false alarms. PASAD also detects 2 scenarios but with far more false positives (22). Transferring these IIDSs to HAI with IPAL reveals their unexplored potential to generalize to new scenarios.
IPAL enables a detailed look at how different IIDSs compare against each other. We even observed how differently IIDSs emit alerts motivating the use of well-chosen metrics [60]. Besides this in-depth comparison, IPAL allows to apply IIDSs to new datasets quickly, and we showed that Seq2Seq-NN indeed generalizes.
What sets it apart from other datasets is that it comprises of network attacks (DoS flooding, MitM to manipulate packet data, and TCP scanning), as well as, physical attacks (e.g., pump breakdowns) and is thus well-suited to analyze and compare both IIDS types. We visualize the results of applying the five IIDSs to the WDT dataset in Fig. 6 (Appx. B provides further in-depth metrics). Similar to Fig. 5, alerts raised by process state-aware IIDSs are visualized as time ranges. In contrast, individual packets flagged as malicious by communication-based IIDSs are marked with ×. We highlight the attack types in the dataset with different colors.
Without further tweaks, IAT range and DTMC do not produce reliable results, as indicated by the continuous emitting of alerts (in grey). Upon closer manual inspection, these "false" alerts can be traced back to either a missing communication sequence in the training data for DTMC or a missing margin of error for a lowerbound check in the IAT range model. Since these alerts (in grey) are easily circumventable, we excluded them from the following evaluation and considered only alerts marked with black ×.
Process state-aware. We expect process state-aware IIDSs to perform best on the physical attack type since they operate on aggregated state information (cf. Sec. 3.2.2). Indeed, three out of the seven attacks detected by PASAD and five out of nine attacks detected by TABOR are of the physical type. Interestingly, Seq2Seq-NN does not detect a single physical attack even though it should be capable of doing so as it analyses the physical state. Meanwhile, network attacks are covered insufficiently by process state-aware IIDS, as only attacks directly aiming at physical consequences, such as some MitM and few DoS attacks, are detected. In total, TABOR and PASAD detect three MitM attacks each.
Communication-based. Like process state-aware IIDSs, communication-based IIDSs detect MitM attacks, despite not having an understanding of the underlying physical process. DTMC detects three MitM instances, IAT-Range five attacks, and IAT-Mean only two. Another observation is that these IIDSs detect attacks early, likely even shortly before the attack has a real physical impact (which is detected by process state-aware IIDSs). Regarding DoS, all except one instance are detected by IAT and DTMC. Scanning, since typically considered out of scope for semantic IIDSs (cf. Sec. 2.2 and Sec. 3.2), is, as expected, hardly detectable by any approach.
Both research branches benefit from each other through IPAL. Overall, process state-aware and communication-based IIDSs detect overlapping and contradicting flavors of attack types. Thus, they complement each other nicely, either to widen the covered attack types or to realize redundancy for equivalent attacks -better than a single research branch can achieve on its own. Here, IPAL not only facilitates the generalization to another dataset but also, for the first time, enables the interworking of contrary research streams.

FURTHER RELATED WORK
Besides research on IIDSs covered extensively in Sec. 3.2, our work draws inspiration from different streams of related research. Existing tools from traditional network monitoring, such as NetFlow [32], lack essential features and are thus not powerful enough for IIDSs. Furthermore, adapting traditional, rule-based IDS approaches [94,100,107] to industrial domains [22,27,[46][47][48]122] is also closely related to our work. These rule-based IDSs, however, only detect traditional (known) attacks (cf. Sec. 2.2) and thus only complement process-aware IIDSs that are able to detect even (stealthy) attacks exploiting physical processes to cause harm [49,114].
Furthermore, past analysis of IIDSs [5,49,90,114], partially with a focus on specific domains [34,58,65,84,98], already describe problems such as data heterogeneity and evaluation bias in IIDS research (cf. Sec. 2.3). To analyze the ability of existing IIDSs to detect unseen sensor spoofing attacks, the only reproducibility study (considering only model-free state process-based IIDSs) known to us [36] shows significant differences between claimed generalizability and reality. These works, in addition to reports on issues with widely-used datasets [112], motivated us to quantify these problems and ultimately mitigate them through protocol independence.
The problem of protocol heterogeneity in industrial communication is not exclusive to intrusion detection. Proposals to address this issue (e.g., PLC4X [45] and StreamPipes [120,125]) do, however, only extract insufficient information for IIDSs and consume valuable bandwidth through polling, a limited good in many industrial scenarios [17,118]. Other work in this context is concerned with the interoperability of devices [52,69,77,127] and proposes translation between different data representations. We do, however, expect these related fields to benefit from IPAL in the future too, e.g., to reduce development costs and improve protocol support.
Lastly, Ryšavý et al. [101] propose a library to ease industrial network data preprocessing for intrusion detection. Since their approach requires IIDS developers to implement custom data extractors for each protocol, this solution does not tackle protocolindependence, nor does it facilitate generalizability. Still, to motivate the library, the authors similarly acknowledge the problem of heterogeneity within the IIDS research landscape [101] and likewise strive to move towards an interconnected IIDS community.

CONCLUSION
A growing number of cyber attacks against industrial networks not only inflict substantial financial and environmental damage but even put human lives at risk [55]. Intrusion detection is regarded as a promising complementary protective measure to timely detect such attacks, which is especially well suited for industrial settings due to their repetitive processes and predictable network behavior. Consequently, a large research community gathered around industrial intrusion detection. The produced research advancements are, however, surprisingly scattered, as many solutions are proposed for distinct communication protocols in specific industrial domains, thus hindering their transfer to other industrial domains.
To better understand this phenomenon, we survey 53 scientific IIDSs and identify an unexplored potential: While practically operating in protocol and domain-dependent silos, theoretically, neither do IIDSs operate on the information of certain communication protocols nor are their fundamental detection methodologies specific to individual domains. Consequently, there is a huge potential for protocol-independent IIDSs to protect industrial networks across scenarios, not only to improve the research landscape but also to contribute towards widespread real-world deployments.
To unleash this potential, we propose IPAL, our industrial protocol abstraction layer that decouples intrusion detection from domain-specific industrial communication protocols. To this end, IPAL transcribes protocol features relevant for intrusion detection into a common abstract representation. To show the applicability and correctness of our approach, we conducted a reproducibility study of eight IIDSs from related work, proving that IIDSs can indeed be implemented on top of IPAL. Finally, with the ability to transfer IIDSs seamlessly across different industrial protocols and domains, we studied how existing IIDSs generalize to new scenarios. We find that existing works are (i) indeed not confined to specific domains or protocols, but also that (ii) the type of attacks a given system can detect is not sufficiently studied, thus (iii) motivating the idea to cooperatively combine approaches from previously disjoint detection domains. With our work, we lay the foundation to break up protocol-dependence of IIDSs research and enable further studies regarding the generalization and application of IIDSs in new research or real-world scenarios.

A PERFORMANCE COMPARISON OF PROCESS STATE-AWARE IIDSS
In Sec. 6.2, we used IPAL to compare three different process stateaware IIDSs against each other on the SWaT dataset [50]. To study their potential to generalize to new industrial domains, we applied these IIDSs to two new datasets (WADI [7] and HAI [104]). Our analysis in Fig. 5 depicts the alerts of these three approaches visually, which provides a decent intuition on how these approaches work in general. To additionally confirm our observations objectively and in further detail here, Tab. 9 summarizes different metrics for all evaluated settings from that case study. Detected attacks counts the number of scenarios overlapping with an alarm and false alarms are all alarms not overlapping with an attack. Penalty score accumulates for all true positives the time in seconds with regions not labeled as attack within the dataset, based on the definition by Lin et al. [82]. The remaining traditional metrics (accuracy, precision, recall, and f1-score) were calculated one by one for each entry within the evaluation dataset. While these metrics may be valuable for randomized datasets, they struggle to infer the quality of time-aware datasets [60], especially since approaches such as PASAD or Seq2Seq-NN detect attacks with a slight delay.

B COOPERATIVE COMMUNICATION-BASED AND PROCESS STATE-AWARE IIDSs
In Sec. 6.3, we used IPAL to compare three process state-aware IIDSs (TABOR [82], Seq2Seq-NN [68] & PASAD [16]) against the complementary research branch of communication-based IIDSs represented by the IAT [81] and DTMC [40] approaches. To study which type of alerts these two IIDS directions detect best, we applied them to the WDT dataset [37], as it consists of network and physical attacks. As before, we provide further details by summarizing the different metrics for this case study (cf. Fig. 6) in Tab. 10. For a short description of the used metrics, refer to the previous Appx. A. Note that process state-aware IIDSs alert over time-ranges while communication-based IIDSs mark individual network packets. This results in poor performance of all communication-based IIDSs w.r.t. recall and the F1-score since this type of IIDS emits short alerts while the metrics favor coverage over the entire attack's range. Again, as described in Sec. 6.3, the circumventable false alerts from the IAT range and DTMC approach (cf. grey annotations in Fig. 6) are neglected for this evaluation.