{'fraction_non_alphanumeric': 0.027235203066263727, 'fraction_numerical': 0.008329033684676052, 'mean_word_length': 5.079991037418776, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 1, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
An Evaluation of Indicators of Railway Intelligent Transportation Systems using the Group Analytic Hierarchy Process

Intelligent Transportation Systems (ITS) have a significant and promising role in the overall improvement of transport performances. However, in the absence of a set of key performance indicators (KPI), it is very difficult to evaluate ITS and the effects of their application objectively. Therefore, the first aim of the paper is to develop an appropriate KPIs for assessment of railway ITS and the monitoring of their impacts. Based on that, a tool for the evaluation of the importance of these indicators is proposed. For that purpose a set of twenty-four indicators was grouped by topics under the economic, social, and environmental dimensions of sustainability. The paper thoroughly describes their development and classification. An evaluation of indicators based on expert opinions via the Group Analytic Hierarchy Process (GAHP) method is presented. Results obtained by the GAHP method illustrate the importance or significance of indicators for railway ITS.


Introduction
During recent decades, the ITS development and deployment have been progressing rapidly throughout the world. In the field of transport, the term ITS refers to the application of the Information and Communication Technologies (ICT) for information and data streaming in real time, all with the aim of providing an intelligent use of infrastructure and vehicles, as well as traffic management improvement. To be more precise, ITS apply advanced technologies such as electronics, communication, information, image processing, and various sensors that can not only help to improve transportation safety, mobility, efficiency, and productivity, but also to reduce transportation impacts on the environment [1] .
It could be said that ITS actually represent systems working in synergy to improve performance [2] . In railway transport, transport performance, there is a need for overall analysis of all ITS impacts. The benefits from the implementation of ITS have already gained much attention; and likely or potential results generated by investment into particular systems require further study [4] . Summarizing the effects of ITS can be very useful for decision makers of all kinds. The effects of ITS can be observed as a decision support tool both in making the right decisions in terms of ITS deployment or development and in recognizing the areas of improvement or evaluation of ITS. On the other hand, the lack of simple and efficient access to ITS impacts can be a crucial factor regarding the slow deployment of ITS and of a reluctance to invest in ITS. However, most guidelines do not explain in detail how the impacts should be measured or evaluated because many benefits are inherently difficult to measure or even define properly [5] . Therefore, the need to research and analyze the impacts generated by the implementation of ITS has become an urgent task.
Key performance indicators (KPI) are to be defined with the aim of monitoring the effects of ITS -i.e., improvements in transportation performance. Without specific indicators that represent the impact of ITS, it is difficult to determine whether the transportation performance has improved as well as what needs to be done to make it better.
The measurement of the ITS impacts is mainly performed through the development of the specific indicators for individual technologies. Certain steps in development and the summarizing of a comprehensive set of KPI of ITS for road transport have already been presented by [6] . Unfortunately, the key performance indicators for measuring the impacts of ITS on railway transportation are still missing [6] . The most important aspect regarding the indicators is that the railways can learn from one another and can undertake the appropriate initiatives for improving their technology.
This paper aims to identify a comprehensive set of indicators for assessing and monitoring the impacts of ITS on railway transport.
The concept of sustainability has been widely applied and usually attempts to integrate environmental, social and economic concerns [2] . In this paper, sustainability is used in regard to the environmental, social, and economic set of indicators used to assess the railway ITS. The paper has a twofold aim: (1) to identify a key set of indicators for railway ITS evaluation; and (2) to evaluate the indicators based on expert opinions by the Group AHP method.
The AHP method has proven to be an appropriate and popular technique for determining weights (significance) in multi-criteria problems. With the aim of avoiding partiality of any expert and due to the fact that different experts focus on different indicators of railway ITS, the authors have decided to use the Group AHP method. The wide use of the AHP method in decision making is reflected by its numerous citations of Satty [3] , with more than 2000 references [8] . In the field of ITS, the AHP method was used in [9] to investigate the preferences of outcomes in comparison with the broader goals by members of identified stakeholder groups for the FAST-TRAC system. Moreover, AHP was used to prioritize and weigh of various performance criteria in order to form groups of decision alternatives through the Dempster-Shafer Theory to form an intelligent transport sustainability index [2,10] . In the procedure of proposing a new two-stage multi-criteria analysis approach to assess the best safety innovation projects [11] , used AHP for both deriving criterion weights and ranking alternatives based on stakeholder opinions.
Regarding the application of AHP in the railway field, according to a systematic review of Multi-Criteria Decision Making (MCDM) techniques [12] , the AHP method was until now (from 1993 to 2015) used only two times for evaluations in the field of railway traffic or railway transportation. Mandic et al [13] . proposed an improvement of the two-phase multicriteria model for Serbian railways, and Gercek et al [14] . used AHP for an assessment of the rail transit networks. After that, AHP was used in Krmac and Djordjević [15] for the evaluation of Train Control Information Systems (TCISs) and their benefits. However, based on the knowledge of the authors, at the moment of writing of this paper the application of the AHP and/or the Group AHP for the evaluation of railway KPIs in literature does not exist.
The research questions and the motivation in terms of the development of KPIs for railway ITS and application of the proposed technique regarding the evaluation of KPI is presented in the next section. Section 3 presents developed KPI and provides their detailed descriptions. Section 4 describes the background of the AHP and group decision making with AHP, while the Section 5 presents the results of this study. Finally, Section 6 makes concluding remarks and offers proposals for future work.

Research Questions and Motivation
The main research question that fits with the main task of the ITS evaluation and represents both the research motivation and set goals, was "Which are the most important indicators for assessing, monitoring and choosing of railway ITS technologies that enhance the efficient implementation of the objectives and strategies of sustainability and which indicators best describe the effects of railway ITS?" The research implies the examination of the existing literature in terms of the evaluation of ITS in transport, as well as the development of the indicators for assessing or selecting particular ITS. The literature was further reviewed in terms of monitoring or measuring the effects of ITS. The overall literature review was focused on railway ITS. All identified indicators were classified under appropriate themes defined for each sustainability dimension. Finally, the assessment of indicators were performed by means of judgements obtained from experts in the fields of railways and ITS.
In the literature, papers that focus on proposing, developing and using indicators for ITS evaluation, can be found, but mainly they concern the road transport. Moreover, some steps towards creating a joint set of indicators, especially for road transport, have already been made, but the agreement on the importance and priorities among ITS indicators, as well as potential ITS benefits, are still missing [6] . As for the railway ITS indicators, nothing from the above has been done yet. Therefore, the focus of the authors' study was to create a list of indicators for railway ITS and provide answers as to what is the importance of railway ITS indicators and which indicators have the top priority in assessing such technologies and their effects.
These questions are rather difficult to answer as indicators are different and multidimensional. Usually, for assessing ITS, models that have been designed to follow only a single criterion (e.g., maximize traffic efficiency, or minimize cost of implementation, etc.) were developed. Therefore, the authors have used multidimensionality as a guideline for selecting the method of the evaluation of indicators. Thus, the Analytical Hierarchical Process (AHP), as the most favorable multi-criteria method for evaluating indicators was used. The AHP can be used for quantitative assessment and providing an overall measure of collective judgment of various relevant experts. The pairwise comparison of data of group of experts' assessments of priorities was gathered through a survey carried out among selected transportation professionals and experts from Slovenia

Development of Indicators
The development of indicators for evaluation of railway ITS and for monitoring or measuring their effects on the performance of railway transport was based on literature review of project reports on road transport indicators, studies of assessments of ITS, and literature on railway ITS. Depending on the type of considered ITS and impacts analyzed by reviewed literature developed indicators were associated with different themes -costs, system condition, operational efficiency, mobility, safety, reliability, pollution reduction, and energy efficiency -that were also defined. After performing interviews and consultations with experts in the ITS field, individual indicators were developed.
Twenty-four indicators were created and their objectives defined wherever possible. These indicators were first grouped by dimensions of sustainability -i.e., economic, social and environmental -and then by eight indicator themes within these dimensions (see Table 3). The indicators mostly referred to the cost of implementation of railway ITS and the effects of railway ITS on railway transport performance. Developed indicators reflected the EU transport policy and objectives of sustainable transport development. The presented set of indicators could be applied to an entire network, part of the network or a specific rail section. Data for some of the presented indicators could be collected from statistical databases such as Eurostat, while for others only from the authorities, managers, operators, ministries, etc. Additionally, some indicators that do not directly describe the correlation with ITS, such as the number of locomotives as the main source of GHG emission and the average weight of train which has an effect on increased level of CO2 emission, were also introduced [16] .

The description of developed indicators and themes
When considering railway efficiency, it could be useful to concentrate on railway infrastructure and those railway operations that have a direct effect on costs and revenues and take into account that automation and technology implementation are the major drivers of efficiency [17] . Indicators included in the economic dimension refer to the investment costs and costs of maintenance and operation of ITS technologies. These indicators represent direct costs of ITS implementation and are usually measured and controlled through cost benefit analysis [5,18] . The indicators costs of operation and maintenance of railway and the number of employed in operations indirectly reflect the changes of the costs due to an ITS implementation, because the operational and maintenance costs of railway may be reduced by introducing an automated track maintenance technology [17,19] while the number of employed in operations may be reduced by introducing remotely controlled safety devices. The railway provides employment to a large number of workers, which significantly supports economic development. The inclusion of this indicator therefore assists in understanding the ITS efficiency in terms of the number of employees and is thus included in the economic dimension [20] .
The number of tracks, as the most important feature of railway infrastructure, the time windows for maintenance and the average headway between trains all contribute to the increase of rail capacity. With ITS, the total kilometers of rail increase the efficiency of track utilization which results in a linear increase of revenues, as well as the costs of reconstruction and maintenance [19] . Additionally, one of the most relevant efficiency indicators that the economy of railway depends on is the traffic volume -i.e., the number of kilometers. High utilization of railway assets, such as the number of locomotives leads to more efficient railways. Moreover, network complexity -i.e., certain network characteristics such as the number of level crossings, the number of interlocking systems, the number of control centers and especially system failures have long-term impacts on infrastructure maintenance costs [17] . Additionally, the implementation of ITS may have a significant influence on the increase of the average speed and train length, which have a positive impact on revenue. Therefore, they are also included in the economy dimension. As tracks that can handle heavier loads contribute to the reduction of costs of freight railways and to the improvement of transport capacity, the indicator average weight of train was also included as economic [17] .
To meet the needs of society, aside from its predominant function -i.e., transportation of goods and peopletransport should fulfill several additional functions [20] . Even though railway is treated as a relatively saf mode of transportation, the number of accidents was introduced as railway accidents may occur due to system error. However, there are many accidents caused by lack of the safety awareness of ordinary people, such as crossing accidents [21,22] .
Since the reliability of railway is the most important requirement of railway customers [23] , and delays of trains negatively impact on the degree of the reliability, the indicator average delay of train(s) was introduced. On the other hand, it is expected that the adequate use and the implementation of ITS shall increase reliability. Additionally, mobility measures should reflect the ability of people and goods to reach different destinations and is mainly concerned with travel time [24] , so the travel time and frequency of trains may be favorable indicators which point to mobility changes.
The challenge of climate change and the depletion of natural resources are indispensable elements of any sustainable strategy. Railway transport uses less energy and emits fewer air pollutants than other transport modes [24] .
ITS can help reduce negative environmental impacts through the optimization of journeys, speed regulation, decreasing stops, delays and accidents, as well as through improving the performance of locomotives and drivers [16,25,26,27] . Based on this, indicators placed in the environmental dimension are the amount of GHG emissions and the energy consumption.

A Description of Analytical Hierarchical Process
The Analytical Hierarchical Process (AHP) is a Multi Criteria Decision Making method introduced by [28] . It is an effective tool for dealing with complex decision making. AHP aims to deconstruct a complex decision problem into a hierarchy of smaller constituents. Through the AHP a hierarchy representing relevant aspects of the problem (criteria, sub-criteria, attributes and alternatives) is constructed. The goal or mission concerning the problem is placed at the top of this hierarchy. Other relevant aspects (criteria, sub-criteria, attributes, etc.) are placed in the remaining levels. AHP is based on pairwise comparisons of the decision elements helping decision makers to set priorities and make the optimal decision [29,30] . Hence, with AHP significances (relative weights) among criteria that are hierarchically non-structured and vice versa in terms of those belonging to a higher level are determined [31] . Based on that, in this paper an AHP process was proposed for the purpose of evaluating the importance of KPI for railway ITS.
The process of the AHP method consists of several steps [32,33,34,35]  In matrix A each entry is a positive value with a reciprocal, that is, aji=1/aij,, and aii=1 for all i, j = 1,2...n. A matrix is consistent if a ij = a ik a kj , i, j, k = 1, ... n holds among its entries, which means that matrix A does not contain errors. Then, priorities are identified among all elements as compared to the higher level of the hierarchy.
Quantification of the priorities among all elements is possible by using judgements from a fundamental scale, the so called Saaty's scale (Table 1) proposed by [36] . For n matrix, n n−1 2 judgments are necessary. 4. exploring the priorities of each criterion among all solutions that can be calculated using the eigenvalues -in contrast to other multicriteria techniques, the AHP permits an assessment of the degree of inconsistency present in the judgements expressed by the decision maker in the reciprocal positive pairwise comparison matrix [36] ; 5. checking the consistency of the judgments -Consistency is determined by consistency index CI which can be calculated by equation (1), where λ max is the largest eigenvalue. Judgment consistency can be checked by the value of consistency ratio CR (2), where RI represents random index, the value of which depends on matrix level.
Average random consistency is shown in Table 2.  (3), and are usually slightly larger; e.g., for n=15 the value of RI by formula is 1.72 [31,37] .

Group Decision Making with AHP
Since the introduction of the AHP method, various extensions and combinations with other MCDM techniques have been carried out. The AHP can also be used for group decision-making process that involves multiple actors, scenarios, and decision elements [38,39,40] . Group decision making is based on facilitated participation of various experts and integration of various experts' opinions [40] . In order to avoid bias of individual opinions of experts the Group AHP was used for the evaluation of KPI.
A set of decision makers were marked as D = d 1 ,d 2 ,…,d m , while λ = λ 1 ,λ 2, …,λ m refers to the weight of the vector of decision makers, whereas λ k > 0, k = 1,2,…,m and k=1 m λ k = 1 , if A k = a ij k nxn is a judgment matrix provided by the decision maker d k (k=1,2,…,m.) [41] . As for group decision making in AHP, the two most useful methods for obtaining collective opinion are the Aggregation of Individual Judgments (AIJ) and the Aggregation of Individual Priorities (AIP). The concepts of AIJ and AIP are used depending on the group intention to behave as a synergistic unit or as a group of individuals, respectively [41,42,43] . According to [42] AIJ is most often performed using weighted geometric mean, whereas AIP is typically performed using the weighted arithmetic mean [40,43] . For Saaty [44] the use of geometric mean provides an effective way to aggregate group decision weights. The authors [41] state that, in terms of AIJ, decision makers use the weighted geometric mean method to aggregate individual judgement matrices to (k) λ k . Detailed overview of the levels of aggregation and procedures of aggregation through the use of the above mentioned is described in [45] .

Results and Discussion
In order to evaluate KPI of railway ITS, a survey was designed to evaluate preferences for indicators of railway ITS by transportation professionals from the Railway of Slovenia, Ministry of Traffic and Infrastructure, the ITS Association of Slovenia, as well as railway and ITS researchers and experts from Slovenia. These professionals were asked to provide a pairwise comparison judgments for indicators of railway ITS. The priorities and level of consensus amongst them were derived from their survey responses.
The survey consisted of 24 indicators of railway ITS. All defined themes and dimensions, associated indicators and their descriptions, as well as calculated weights that represent the significance of various indicators are summarized in Table 3. The first five indicators were classified under the theme costs, the next seven under the theme system condition and the final six under the theme operational efficiency. All these indicators were classified in the economic dimension. A total of four indicators were classified within themes of mobility, safety, and reliability under the social dimension. Two indicators -pollution reduction and energy efficiency -were placed within the environmental dimension. Some of these indicators represent benefits of ITS, as well as the European Commission goals for sustainable transportation system. Each of these indicators could be associated with an individual or specific set of railway ITS technologies. Having in mind the above information, the examinees were asked to provide numerical pairwise comparison judgments of their relative preferences for each of the indicators.
From these 24 indicators, 276 [n(n-1)/2=276] pairwise comparisons were generated and set in an optimum order. The pairwise comparisons were recorded using a scale from "1" (equal importance) to "9" (extreme importance) for each indicator (see Table 1). With the individual preference data in hand, a collective preference analysis was aggregated using the weighted geometric mean in Group AHP technique. 10 questionnaires was distributed, 9 were returned filled out. In any case, 5 questionnaires were sufficient and consistent for ensuring adequate results according to the Group AHP method. Naturally, there was a possibility of inconsistency and necessary revision through repeated poll until achieving the necessary consistency was needed. Moreover, the group decision making and the evaluation of a set of indicators for ITS technologies could result in irrelevancy of the results due to inadequate knowledge of the participants or incomplete information.
The output of the Group AHP method is relative significance for each indicator of railway ITS based on scores of individual experts assigned to specific indicator. From the questionnaires that were returned, 5 of 9 were eliminated from the analysis due to judgement inconsistency/presence of inconsistency greater than 10%, calculated using Super Decision software.
In terms of the economic dimension of sustainability, the importance of each indicator was observed on the level of three themes. The reason is that consistency cannot be achieved in the case of comparison of higher matrix couplesi.e., larger number of indicators. However, when compared to other dimensions of sustainability (the social and environmental dimensions), the indicator significance was considered on the level of these dimensions. The total sum of weights for each group of indicators within the individual theme or individual dimension equals 1.
During the aggregation of individual evaluations in a group, to each expert was assigned the same importance -i.e., λ k = 0.2.

Table 3. Indicators of railway ITS and their importance
Within the economic dimension, the indicator operational and maintenance costs of railway (E4) associated to the theme Costs resulted as the most important for the railway ITS evaluation, followed by the operating/enforcement costs of ITS (E2). The indicators investment costs of ITS (E1) and maintenance cost of ITS (E3), with the same significance occupy the third position. The indicator ranked as least important in this group of costs is the number of employed in operations and traffic of railway (E5).
The indicators associated to System condition were classified in the following way: as the most significant indicator was the failures of system (E12), followed by the total kilometers of rail with ITS (E6), the number of tracks (E8), the number of interlocking systems (E10), the number of control centers (E11), the number of level crossings (E9), and the number of locomotives (E7). Finally, within the third theme under the economic dimension, Operational efficiency, the most significant indicator is average speed of train (E13), the second ranked is traffic volume (E18), followed by the reduction of time windows for maintenance (E15), average headway between trains (E16), average length of train (E17), and average weight of train (E14).
The most significant indicator within the social dimension is the number of accidents (S3) followed by the average delay of train(s) (S4), travel time (S1), and finally frequency of trains (S2).
As for the environmental dimension, the most significant indicator is the energy consumption (En2), while the second ranked is the indicator amount of GHG emissions (En1)

Conclusion
This paper develops and evaluates indicators for the evaluation of railway ITS by sustainability using the method Group AHP. Consideration of grades given by individual experts can offer some of the ideas for inconsistencies in terms of experts' judgements. Namely, years of experience in the field of railway ITS and railway, as well as the position of the expert, have significant impact on their valuation of indicators for railway ITS evaluation. In order to avoid the partiality of individual examinees, the aggregation of individual grades into a group grade for each indicator is necessary. In this way, a realistic image of the significance of each indicator is obtained. In such circumstances, the indicator valuation based on the Group AHP method provides useful information and enables the Group AHP method to be suitable for that purpose.
As for group valuation of indicators by AHP method, the analysis revealed the most important -i.e., the most significant -indicators for evaluating ITS by sustainability. The developed indicators for railway ITS evaluation offer the possibility of making comparative performance assessments of existing ITS operations, conducting ITS gap analysis, and developing performance-based investment plans that lead to effective ITS deployment decisions. However, it is important to keep in mind that during the development of indicators for ITS evaluation, indicators illustrate mutual dependence. For instance, the reduction of train delays has impact on capacity increase, etc. Therefore, the future work will be focused on the application of the Fuzzy Cognitive Maps (FCM) approach for construction of mutual connections between different indicators for railway ITS evaluation.