{'fraction_non_alphanumeric': 0.022222222222222223, 'fraction_numerical': 0.004878048780487805, 'mean_word_length': 5.12571934484285, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Utilizing radar graphs in the visualization of simulation and estimation results in network meta‐analysis

Traditional visualization in meta‐analysis uses forest plots to illustrate the combined treatment effect, along with the respective results from primary trials. While the purpose of visualization is clear in the pairwise setting, additional treatments broaden the focus and extend the results to be illustrated in network meta‐analysis. The complexity increases further in situations where all potential contrasts in the network are compared to a predefined fixed value of interest, such as the 95% coverage evaluated against the nominal value of 95% in simulation studies. We propose utilizing radar graphs to illustrate results from network meta‐analysis in cases where the interest lies in the comparison of estimated results (after fitting a network meta‐analysis in a specific data set) or a performance measure (simulation study) to a pre‐defined fixed reference value. Accounting for the complex high‐dimensional data structure, the general picture of the full network is captured at once without increasing the space needed for visualization. Especially in large simulation studies, where multiple scenarios need to be visually combined to gain an overview on different scenarios, this type of illustration facilitates the discussion of results. Further properties, such as the expected variation due to the Monte‐Carlo error or the differentiation between directly and indirectly estimated treatment contrasts in simulation studies, as well as the indication of well‐connected and sparsely connected treatments in an applied network meta‐analysis, can additionally be included in the visualization. While we used the radar‐graph mainly for a simulation study, other applications are suitable whenever relative contributions of treatment (contrasts) are of interest.


| BACKGROUND AND PREVIOUS WORK
Communication through visualizing data or analysis results is an important step in a quantitative workflow and is one of the most effective ways to summarize and communicate complex results 12,3 or results from large data sets. 4 Visual data presentations make up a considerable amount of the results section in scientific publications. At best, they help to access, understand, and interpret the main message of a research paper quickly. 5 Visualization, as every form of communication, therefore needs careful evaluation of the message that should be illustrated, 1,3 careful adaption to the specific setting and audience, as well as careful composing of the graph itself, maximizing the data-to-ink ratio. 6 Meta-analysis is a technique to summarize results from various studies commonly applied in the medical context quantitatively. 7 Due to the limited availability of individual participant data, the summary measure is usually estimated from aggregated results of studies identified through a systematic review process. The interest in meta-analysis in the medical context commonly lies in comparing two (pairwise meta-analysis) or more than two (network meta-analysis) treatments for one medical condition, using relative treatment effects. One advantage of network meta-analysis over multiple pairwise comparisons is the possibility to model directly observed comparisons and indirectly informed evidence simultaneously. 8 However, depending on the number of treatments in a network meta-analysis, the number of treatment effects can easily be large. Consequently, presenting and discussing estimation results is complex. Another characteristic of network meta-analysis is that it potentially provides more precise effect estimates than pairwise meta-analyses by borrowing strength across contrasts. This borrowing of strength can be summarized in the recently introduced BoS-statistic, 9,10 a measure that evaluates the variance of a contrast estimated by a network meta-analysis relative to its pairwise counterpart. This means that the BoS-statistic for one treatment contrast needs to be interpreted relative to those of all other contrasts in the network, as well as relative to aspects of the network structure itself, such as the geometry and connectivity of the network. Communicating the results by means of a BoS-statistic can, therefore, be challenging, especially for large networks or if several endpoints are considered simultaneously.
Visualization in (pairwise) meta-analysis typically consists of forest plots that illustrate the individual studies' treatment effects and standard errors for an outcome along with those of the combined effects resulting from a quantitative synthesis. 11 Additionally, funnel plots to detect publication bias and stacked horizontal bar charts illustrating the risk of bias are commonly used in the discussion of meta-analytic results. 12 Further visualization techniques, such as network graphs or contribution graphs, are commonly applied in network meta-analysis, on which Chaimani et al. 13 provided an overview. Besides these visualization techniques, several graphs exist that were designed to illustrate estimation results in network meta-analysis.
Tan et al. 14 identified estimated treatment effects and ranks of the treatments within the network as relevant for reporting and proposed different graph types summarizing these aspects. They proposed to use a summary forest plot matrix in which ranks of the treatments are displayed at the diagonal, while forest plots of the combined effects of each, pairwise, and network meta-analysis, are shown below and the numerical estimates of the latter above the diagonal. As an alternative to this visualization, they additionally proposed a summary forest plot table, where combined effects for all pairwise metaanalyses are calculated and then arranged in a traditional-style forest plot ordered by the treatment ranks. With both graphs, they provide an informal way of accessing the borrowing-of-strength by comparing direct and mixed evidence while at the same time visualizing the treatment ranks for one outcome at a time. Both illustrations, however, cannot be easily applied to large networks, as they considerably increase in size with the number of treatment contrasts evaluated. In addition, Tan et al. 14 suggested the median rank chart that uses a color-intensity scheme to emphasize the ranks of the treatments in a network meta-analysis for a certain outcome. The median rank chart is easily understandable; however, it has a relatively low data-to-ink ratio coding solely information on ranking. All three of their proposed visualizations illustrate one outcome at a time, and while the median rank chart might be adapted by using, for example, different colors and intensities within a color, there is no obvious way to modify the summary forest plot matrix or table to include multiple outcomes. Likewise, as a tool to visualize rankings of treatments from a network meta-analysis, Veroniki et al. 15 introduced the rank-heat plot. Contrary to the median rank chart, 14 it visualizes the summarized rankings of a network metaanalysis for all considered outcomes by using a circular xaxis to facilitate comparability of treatments within large networks. It provides an easily accessible and clear illustration for rankings potentially varying with the outcome that has an increased data-to-ink ratio as compared to the median rank chart. However, an application to estimation or simulation results where the interest lies in comparing more subtle differences (eg, BoS values) is not straightforward: The radii are used to code different outcomes but do not carry numerical information, while the ranking itself is coded using colors.
Besides visualizations for the treatment ranking, there exist also other graphs designed specifically to illustrate one particular estimation result. Consistency, that is, the agreement between directly and indirectly estimated treatment effects, is an important assumption in network meta-analysis that commonly needs to be evaluated. As a tool to identify and visualize potential inconsistency, Krahn et al. 16 proposed the net-heat plot. In this graph, a colored version of the matrix of estimated treatment effects is used, which helps to identify treatment contrasts or designs that contribute to inconsistency in the network. Using this idea, it is, therefore, possible to assess the validity of estimation results and to identify inconsistency hotspots in the network. It is used to illustrate results for one endpoint at a time and can become difficult to display in large networks with many potential treatment contrasts. Law et al. 17 proposed to visualize the fitted results of a network meta-analysis by visualizing similarity between treatments through distance by evaluating closeness between treatments through different distance measures and the definition of a threshold value. To do so, they group treatments into network communities that can be visually identified through color-coding. With this visualization, well-and weakly-defined clusters in the network can be easily detected. However, grouping treatments into communities is ambiguous and may result in qualitatively different results depending on the threshold used. Furthermore, both the net-heat plot and the illustration of communities based on distance cannot be easily transferred to illustrate other estimation results from network meta-analysis.
During the last decades, simulation studies have been increasingly important in statistical research 18 and are also used in (network) meta-analysis. Results of simulation studies are evaluated focusing on the relative performance of candidate methods in varying scenarios or the evaluation concerning the robustness when model assumptions are not met. 18 Visualizing such summarized results of simulation studies is particularly challenging in network metaanalysis. Varying over multiple parameters of interest results in a large set of scenarios, adds further dimensions, and increases the complexity in an initially already complex data structure, making it even more difficult to visualize all relevant properties simultaneously. In this setting, a careful encoding of dimension through visual properties 3 and a high data-to-ink ratio 6 is even more important than in the evaluation of results from an empirical network meta-analysis. Specifically for simulation studies of network meta-analyses, Rücker and Schwarzer 19 proposed the nested loop plot. This visualization technique is constructed by ordering all simulation scenarios lexicographically and arranging them consecutively along the horizontal axis of the graph. Thereby, all simulation parameters are incorporated in the same plot by nesting parameters into each other and looping through all possible combinations, making it possible to identify overall results and influential parameters. This visualization focuses on illustrating network-level performance measures. As the illustration of overall trends relies on ordering parameters by the strength of their influence hierarchically, transferring this visualization to contrast-level summaries might lead to a loss of illustrative power. Similarly, spotting of an overall trend might be challenging in situations where many parameters influence results without a clear hierarchical structure.
We propose to utilize radar graphs as a visualization tool for empirical and simulation studies in network meta-analyses. We will discuss the aims when using radar graphs in Section 2. In Section 3, application of radar graphs is illustrated by an applied network metaanalysis and a simulation setting. Section 4 concludes with a discussion of the properties of this visualization method.

| USING RADAR GRAPHS IN NETWORK META-ANALYSIS
The value of visualization in quantitative research, best practices in creating graphs in a scientific context, and discussion on common (avoidable) mistakes in designing visual data summaries have been discussed in the literature. 3,6,12 In network meta-analysis, and especially in simulation studies for network meta-analysis, the complexity of the data structure poses a challenge in creating visual summaries, complicating communication and interpretation of results. A recent publication 1 defines the three main ingredients of a graphical representation as having a clear purpose, providing a clear visualization of the data and making the message transported by the visualization obvious. The purpose of a radar graph in network meta-analysis is (a) to compare treatments (or treatment comparisons) to each other, for example, for a particular outcome or statistic, and (b) to compare a set of quantities estimated in a collection of network meta-analysis with a pre-specified value, for example, coverage of 95% confidence intervals in a simulation study. We aim at illustrating estimation or simulation results for network meta-analyses like Tan et al. 14 or Rücker and Schwarzer, 19 without summarizing these results any further like Law et al. 17 or concentrating on certain aspects of network meta-analytic results like Krahn et al. 16 or Veroniki et al. 15 .
To provide a clear visualization of the data and to support the communication of the message conveyed by the data, a radar graph combines several useful properties. The circular x-axis is defined by all treatments or treatment contrasts of the network, using the same idea for illustrating potentially large networks as Veroniki et al. 15 Values on this circular x-axis always need to be on the same scale. For example, in a network meta-analysis of binary data, treatment effects are either all on the scale of (log) odds ratios (OR) or all on the scale defined by a different effect measure, such as risk difference or (log) risk ratio. The treatments (or treatment contrasts) depicted on the x-axis are not naturally ordered. This means that an additional property of the data, for example, the magnitude of the treatment effects, the ranking, or the alphabetical order of their names, can be included in the visualization to sort the x-axis in a meaningful way. Unlike in cases where the x-axis has a natural order (eg, time or sentiment), the ordering of the x-axis can therefore be used to structure the visualization of the data and emphasize the message. To compare these treatments (or treatment comparisons) to each other, differences between values with respect to their distance to center of the radar graph are shown on the continuous y-axis and can be directly interpreted. If the comparison to a pre-specified target value is of interest, the deviations from this predefined value are depicted on the yaxis and are again interpretable. If a common target value exists, such as the nominal level in simulation studies, it should be added to the graph as a separate element. Further visual properties, such as color, brightness, transparency, line-or point-type, are still available to illustrate important data properties, such as estimation method or type of contrast in the network or can be used to provide orientation for the eye, for example, by color-coding grids. If important for data properties, the network graph is added directly to the visualization. R-Code and the necessary data to reproduce the examples are provided in the Supplement.

| ILLUSTRATION OF APPLICATION
We exemplify the proposed visualization by two examples from empirical network meta-analyses and one example from a simulation study. The first empirical network considers treatments against acute mania and was presented in Cipriani et al. 20 and reanalyzed by Efthimiou et al. 21 The network consists of 68 trials comparing 13 active drugs and placebo in a mixture of two-and three-armed trials. There were two primary outcomes, acceptability and efficacy of the treatment (both measured after 3 weeks). During the analysis, the surface under the cumulative ranking curve (SUCRA) 22 was computed for each treatment and endpoint to obtain a ranking between the investigated treatments. As not all 68 trials reported both outcomes, the underlying network structure differs between both outcomes. The results of these two outcomes could be informatively illustrated using a standard scatterplot, although treatments that are not part of the network in both outcomes would be lost. In situations where more than two outcomes are of interest, or in situations where the networks differ substantially concerning the treatments included, a scatterplot would not be an optimal choice. The radar graph in this setting (Figure 1) depicts the treatments on the (circular) x-axis and the F I G U R E 1 Radar graph to illustrate SUCRAs for two different endpoints in a network meta-analysis. Treatments on the circular x-axis and SUCRAs on the y-axis. Due to different endpoints reported in the trials, the network structure differs and is added for each of the endpoints [Colour figure can be viewed at wileyonlinelibrary.com] SUCRA value on the y-axis for each of the two outcomes in a separate panel. For orientation, SUCRA values are coded through colors using the grid lines of the plot. In this figure, treatments are ordered using SUCRA values of the efficacy endpoint. The general picture for both endpoints is very similar: Some treatments perform for acceptability and efficacy, while others are ranked low for both endpoints. However, SUCRA values are generally higher for acceptability than for efficacy. The difference in the underlying network of trials between the two endpoints can be seen in the graph, as one of the treatments, Gabapentin, is missing from the evaluation of efficacy. To connect the SUCRA value with the respective network visually, the two different network graphs are added inside of the radar-graphs. From the visual impression of these network graphs, it is also obvious that the network on acceptability is more densely connected than the one on efficacy.
In a second example (Figure 2), we utilize a radargraph to visualize results from an analysis of the BoS measure. Bangalore et al. 23 investigated antihypertensive drugs and the risk of cancer by performing a network meta-analysis including eight groups of treatments from more than 40 trials and over 250 000 patients. In the empirical network, 11 of the 28 contrasts are only observed indirectly. Of the remaining 17 treatment contrasts where direct evidence is available, eight are informed by only one trial, while the other nine contrasts are informed by two or more studies. Here, we used the graph-theoretical approach by Rücker 24 to estimate treatment effects and their confidence intervals using ORs as an effect measure. Subsequently, the BoS-statistic was calculated for each treatment in the network. The radar graph in this setting ( Figure 2) depicts all treatment contrasts (directly and indirectly observed ones) on the circular x-axis and the respective BoS value on the y-axis. Then, the BoS-statistic is approximated by the ratio of squared confidence interval lengths for a treatment contrast estimated through network meta-analysis divided by those of the pairwise meta-analysis. In this example, contrasts are ordered alphabetically, and BoS values are color-coded. By utilizing a radar graph in this setting, it is F I G U R E 2 Radar graph to illustrate BoS in a network meta-analysis. Contrasts between different classes of antihypertensive drugs on the circular x-axis, BoS-statistics on the y-axis. For further information on the network structure and the presence of multi-arm trials, a network graph is added to the visualization [Colour figure can be viewed at wileyonlinelibrary.com] possible to grasp all contrasts simultaneously and to identify relations between the connectedness (network graph on the side) and the amount of borrowing. Additionally, the type of contrast (indirect, direct from one trial, direct from more than one trial) is distinguished by the color and shape of the points.
To illustrate the utilization in a simulated data set, we use coverage as the target performance measure. A network of eight trials was simulated from a mixture of pairwise and multiarm trials as described in a previous work. 25 Besides the between-trial heterogeneity, which is varied over none, moderate, and substantial, the number of trials informing direct contrasts is varied from 1 over 2 to 5. Furthermore, the geometry and network density were varied in the original simulation study; however, we will use only one combination of geometry and network density to illustrate the visualization idea. The radar graph for the simulated data ( Figure 3) depicts all treatment contrasts in the network on the circular x-axis and the respective coverage values averaged over 2000 repetitions on the y-axis. Different panels are used to differentiate the settings, where heterogeneity is varied over the columns and the number of trials that inform a direct contrast over the rows. Different methods used to estimate the treatment effects (contrasts) are coded by using different colors. Additionally, the Monte-Carlo error is visualized by drawing a grey uncertainty area around the nominal value of 95% calculated by using the Monte-Carlo error. The differentiation between contrasts that are informed by direct information and those only indirectly informed is illustrated by using increased transparency for indirectly informed contrasts. This type of visualization facilitates the general comparison over different scenarios along with the variation of one or two of the parameters. It is also possible to compare the performance of the methods with each other for one scenario or specific types of contrasts (direct vs indirect) in that scenario.

| DISCUSSION
We propose to use radar graphs in the visualization of estimation results or simulation studies on network meta-analysis. We used these radar graphs having two specific purposes in mind, namely, to compare treatments (or treatment contrasts) to each other, or to compare a set of quantities to a predefined target value. In addition, we want to be able to include large networks in the plot. Like Veroniki et al., 15 we use a circular axis to illustrate large networks with a high data-to-ink ratio. However, contrary to the rank-heat plot, our visualization is not restricted to illustrating rankings of treatments within a network. Furthermore, radii code illustrated values, therefore allowing for an additional characteristic of the network to be included in the plot by using color. Like Tan et al., 14 we are interested in a graph that can be used to visualize different properties of a network metaanalysis. However, we concentrate on large networks that are not easily visualized by summary forest plot matrices due to the need to increase the matrix dimension with the number of treatment contrasts. Furthermore, as we are interested in both simulation studies and network meta-analyses estimated from specific data sets we want to extend beyond the forest-plot type visualization as in the summary forest plot table. Like Rücker and Schwarzer, 19 we therefore need to think about a way to illustrate variation over simulation scenarios. However, as we potentially want to include results on the treatment contrast level, the overall trend in the iteration over parameters might be potentially lost when using the nested-loop plot. The radar graphs shown in the examples above have several visual properties in common. One shared feature is that there is no natural ordering for values on the x-axis. Therefore, treatments or treatment contrasts in the network can be sorted by an additional criterion as in Figures 1 and 2. Furthermore, the values depicted on the x-axis all share the same scale making it possible to compare y-values between different treatments or treatment contrasts. Of interest are not the details of deviations in absolute numbers, which would be better communicated by using tables, but also the overall development or deviations from symmetry, which can be used to spot abnormalities. Further visual properties, such as colors identifying different types of contrasts ( Figure 2) or estimation methods (Figure 3), point shapes ( Figure 2), transparency (Figures 2, 3), or background polygons identifying the target value ( Figure 3) may be added when necessary. Furthermore, due to the circular shape of the visualization, it is possible to add network plots within the radar graph making it possible to identify structural differences between results with different underlying networks (Figure 1). Using radar graphs to visualize estimation or simulation results in network meta-analysis has several limitations. While the arrangement around a circular axis is useful when visualizing large networks on relatively small space, it has the drawback that deviations on the circular axis by judging radii do not allow detailed absolute value comparisons. In this sense, radar graphs are very specialized in the way they illustrate data and can therefore not be used in cases where subtle differences are of interest. In some settings, color-coded may be used for orientation; however, in large simulation studies, this might not be an option. In this aspect, using a radar graph is similar to the nested-loop plot by Rücker and Schwarzer 19 in that it should be used to get a general impression on the relative relation of contrasts in a network meta-analysis or identify a trend. Like Rücker and Schwarzer, 19 we are thinking of settings where a high data-to-ink ratio is required to evaluate multiple outcomes or scenarios. A limitation of using radar graphs is certainly that no more than two simulation parameters can be illustrated in one plot by faceting. However, as the Radar graph -illustrates results on a circular x-axis using radius -focus: treatments, contrasts, or network-level summaries -suitable for large networks -color codes additional property -limitations in the detailed evaluation of differences here R Code available in supplement to publication method of Rücker and Schwarzer 19 relies on determining the nesting structure hierarchically, trends are difficult to spot in situations where many of the simulation parameters influence a performance measure. Meaning to expand rather than to replace existing visualization techniques, we provide an overview of the discussed visualizations in Table 1. With the radar graph in mind, we think of situations where the deviation from a predefined fixed (theoretical) value is of interest rather than the illustration of an absolute result and where a general overview over the network of treatments is the aim rather than a detailed comparison of specific treatments.

| CONCLUSIONS
Radar graphs are specifically designed to illustrate differences between the values on the x-axis (at best on a common scale). This suits the needs in network metaanalysis, where comparisons between contrasts or treatments are necessary. This visualization makes it possible to illustrate graphically in small space the complex data structure (eg, the network graph) along with estimation or simulation results. Deviations between the values on the x-axis (treatments or contrasts) or a predefined fixed value are easy to illustrate in this setting. However, it needs to be kept in mind that details in differences are better depicted on a horizontal scale where the eye can follow a straight line. We propose to utilize radar graphs as an extension to the existing graphical tools in network meta-analysis in settings where the interest lies in comparing unordered values on the x-axis as treatments and contrasts on the same scale either to each other or to a predefined fixed value.