{'fraction_non_alphanumeric': 0.04600694444444445, 'fraction_numerical': 0.006786616161616162, 'mean_word_length': 4.083433614119535, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 2, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Optimal statistical decision for Gaussian graphical model selection

Gaussian graphical model is a graphical representation of the dependence structure for a Gaussian random vector. It is recognized as a powerful tool in different applied fields such as bioinformatics, error-control codes, speech language, information retrieval and others. Gaussian graphical model selection is a statistical problem to identify the Gaussian graphical model from a sample of a given size. Different approaches for Gaussian graphical model selection are suggested in the literature. One of them is based on considering the family of individual conditional independence tests. The application of this approach leads to the construction of a variety of multiple testing statistical procedures for Gaussian graphical model selection. An important characteristic of these procedures is its error rate for a given sample size. In existing literature great attention is paid to the control of error rates for incorrect edge inclusion (Type I error). However, in graphical model selection it is also important to take into account error rates for incorrect edge exclusion (Type II error). To deal with this issue we consider the graphical model selection problem in the framework of the multiple decision theory. The quality of statistical procedures is measured by a risk function with additive losses. Additive losses allow both types of errors to be taken into account. We construct the tests of a Neyman structure for individual hypotheses and combine them to obtain a multiple decision statistical procedure. We show that the obtained procedure is optimal in the sense that it minimizes the linear combination of expected numbers of Type I and Type II errors in the class of unbiased multiple decision procedures.


Introduction
Gaussian graphical model is a graphical representation of the dependence structure for a Gaussian random vector. It is recognized as a powerful tool in different applied fields such as bioinformatics, error-control codes, speech language, information retrieval and others [4]. There are two aspects related to the study of the graphical model: the computational and algorithmic aspect, and the statistical aspect. The computational aspect has become increasingly popular due to the growing interest in large scale networks [5]. A central question in the statistical aspect is how to recover the structure of an undirected Gaussian graph from observations. This problem is called the Gaussian graphical model selection problem (GGMS problem). A comprehensive survey on different approaches to this problem is given in [3]. Most of the constructed statistical procedures for GGMS are based on asymptotically optimal estimations of correlations [1], [2], [12]. However, as far as we know, there are no results related to the optimality of statistical procedures for GGMS for a fixed sample size. This is the subject of this paper.
The quality of statistical procedures for GGMS can be measured by two types of errors: false edge inclusion (Type I error) and false edge exclusion (Type II error). Traditional measures of quality used in GGMS are: FWER (Family Wise Error Rate), FDR (False Discovery Rate), FDP (False Discovery Proportion) and others [3]. Most of them are connected with the Type I error (false edge inclusion). It is clear, that the quality of GGMS procedures has to be related to the difference between two graphs: the true graph and selected graph. Therefore in GGMS it is important to take into account both types of errors (Type I and Type II errors). The quality of a statistical procedure can be measured in this case by the linear combination of expected values of the numbers of Type I and Type II errors. We refer to a selection statistical procedure which minimizes this value for a fixed sample size as optimal. The main goal of this paper is to find an optimal statistical procedure for GGMS. To achieve this goal we consider the graphical model selection problem as part of the multiple decision theory [8]. The quality of statistical procedures is measured by the risk function. To take into account the numbers of Type I and Type II errors we consider the additive loss function. We prove that in this case the risk function is a linear combination of expected values of the numbers of Type I and Type II errors. Subsequently, we construct tests of a Neyman structure for individual hypotheses and combine them to obtain a multiple decision statistical procedure. We show that the obtained procedure minimizes the risk function in the class of unbiased multiple decision procedures.
The paper is organized as follows. In Section 2 we give basic definitions and notations.
In Section 3 we describe the multiple decision framework for the Gaussian graphical model selection problem and prove the representation of the risk function. In Section 4 we construct the tests of a Neyman structure for individual hypotheses. In Section 5 we combine individual tests in the multiple decision procedure and prove its optimality. In Section 6 we discuss the proposed approach for different model selection problems.

Problem statement
Let X = (X 1 , X 2 , . . . , X p ) be a random vector with the multivariate Gaussian distribution from N(µ, Σ), where µ = (µ 1 , µ 2 , . . . , µ p ) is the vector of means and Σ = (σ i,j ) is the covariance matrix, σ i,j = cov(X i , X j ), i, j = 1, 2, . . . , p. Let x(t), t = 1, 2, . . . , n be a sample of size n from the distribution of X. We assume in this paper that n > p, and that the matrix Σ is non degenerate. The case n < p has a practical interest too [10], but it is not considered in this paper. The undirected Gaussian graphical model is an undirected graph with p nodes. The nodes of the graph are associated with the random variables X 1 , X 2 , . . . , X p , edge (i, j) is included in the graph if the random variables X i , X j are conditionally dependent [7], [1]. Gaussian graphical model selection problem consists of the identification of a graphical model from observations.
The inverse matrix for Σ, Σ −1 = (σ i,j ) is known as the concentration or precision matrix for the distribution of X. For simplicity we use the notation ρ i,j = ρ i,j•N (i,j) . The problem of pairwise conditional independence testing has the form: According to [7] the partial correlation can be written as Note that the problem of pairwise conditional independence testing (1) is equivalent to The Gaussian graphical model selection problem can be formulated now as multiple testing problem for the set of hypotheses (1).

Multiple decision approach
In this Section we consider the GGMS problem in the framework of decision theory [11].
According to this approach we specify the decision statistical procedures and risk function.
Let X = (X 1 , X 2 , . . . , X p ) be a random vector with multivariate Gaussian distribution from N(µ, Σ). In the GGMS study observations are modeled as a sequence of random vectors X(t), t = 1, 2, . . . , n where n is the sample size and vectors X(t) are independent and identically distributed as X. Let x = (x i (t)) be observations of the random variables X i (t), t = 1, 2, . . . , n, i = 1, 2, . . . , p. Consider the set G of all p × p symmetric matrices represent adjacency matrices of all simple undirected graphs with p vertices. The total can be formulated as a multiple decision problem of the choice between L hypotheses: The multiple decision statistical procedure δ(x) is a map from the sample space R p×n to the is included in the graphical model), and Any multiple decision statistical procedure δ(x) based on the simultaneous inference of individual edge tests (1) can be written as According to [11] the quality of the statistical procedure is defined by the risk function.
By Ω S we denote the parametric region corresponding to hypothesis H S . Let S = (s i,j ), Q = (q i,j ), S, Q ∈ G. By w(S, Q) we denote the loss from decision d Q when hypothesis H S is true, i.e.
w(H S ; d Q ) = w(S, Q), S, Q ∈ G Assume that w(S, S) = 0, S ∈ G. The risk function is defined by where P θ (δ(x) = d Q ) is the probability that decision d Q is taken.
As mentioned before, for the GGMS problem it is important to control Type I and Define the individual loss as

0, otherwise
To take into account both types of errors we suggest the total loss w(S, Q) is defined as: It means that the total loss from the misclassification of H S is equal to the sum of losses from the misclassification of individual edges: The main result of this Section is the following theorem Theorem 1 Let the loss function w be defined by (5), and a i,j = a, b i,j = b, i = j, i, j = 1, 2, . . . , p. Then where Y I (S, δ), Y II (S, δ) are the numbers of Type I and Type II errors for model selection by the statistical procedure δ when the true decision is d S .
Proof. One has where N I (Q) is the number of Type I errors when the procedure δ(x) takes decision d Q and θ ∈ Ω S , and N II (Q) is the number of Type II errors when the procedure δ(x) takes decision d Q and θ ∈ Ω S .

Uniformly most powerful unbiased tests for individual hypotheses
In this Section we briefly present the uniformly most powerful unbiased tests for individual hypotheses (1). More details are given in our paper [6]. Consider the statistics The joint distribution of statistics S k,l , k, l = 1, 2, . . . , N, n > p is given by the Wishart density function [1]: if the matrix S = (s k,l ) is positive definite, and f ({s k,l }) = 0 otherwise. The Wishart density function can be written as: According to [9] (Ch. 4) the uniformly most powerful unbiased (UMPU) test for hypothesis h i,j has the form: where the critical values c ′ i,j , c ′′ i,j are defined from the equations where I is the interval of values of s i,j such that the matrix S = (s k,l ) is positive definite, and α i,j is the significance level of the test.
It is shown in [6] that the constructed UMPU test is equivalent to the following partial correlation test where r i,j is the sample partial correlation, and q i,j is the (α i,j /2)-quantile of the beta distribution Be( n−p 2 , n−p 2 ). Finally, we need to specify the optimality and unbiasedeness of the test (9). Let ω i,j be the set of parameters defined by Let ϕ i,j be a test for individual hypothesis h i,j with significance level α i,j . The test ϕ i,j is reffered to as unbiased [9] if The test ϕ umpu i,j defined by (9) is unbiased and the following inequality holds: for any unbiased test ϕ i,j .

Optimal multiple decision procedures
According to Wald [11] the procedure δ * is referred to as optimal in the class of statistical for any S ∈ G, θ ∈ Ω S , δ ∈ C.
In this paper we consider the class of w-unbiased statistical procedures. Statistical procedure δ(x) is reffered to as w-unbiased if one has Risk(S, θ; δ) = E θ w(S; δ) ≤ E θ w(S ′ ; δ) = Risk(S ′ , θ; δ), for any S, S ′ ∈ G, θ ∈ Ω S . The following theorem describes the optimal procedure in the class of w-unbiased multiple decision procedures for Gaussian graphical model selection.
Theorem 2 Let the loss function w be defined by (5). Let the procedure δ ou be defined by is defined by (9) and Then procedure δ ou is an optimal multiple decision statistical procedure in the class of wunbiased procedures for Gaussian graphical model selection.
Proof. We use the general approach by Lehmann [8] and give a direct proof. Let δ be a statistical procedure defined by (3) where Now we prove that δ ou is a w-unbiased multiple decision procedure. One has from relation (14) and unbiasedness (10) of ϕ umpu Therefore, Risk(s ′ i,j , θ; ϕ umpu i,j ) = Risk(S ′ , θ; δ ou ), for any S, S ′ ∈ G, θ ∈ Ω S .
Finally, we prove that δ ou is optimal in the class of w-unbiased statistical procedures.
The main result of the paper is the following Theorem 3 Let 0 < α < 1, and the loss function w be defined by (5) with a i,j = 1 − α, b i,j = α, i, j = 1, 2, . . . , p. Then for any w-unbiased multiple decision statistical procedure δ defined by (3)-(4) one has for any S ∈ G, θ ∈ Ω S . Here Y I (S, δ), Y II (S, δ) are the numbers of Type I and Type II errors for model selection by statistical procedure δ.

Concluding remarks
The main result of the paper states that statistical procedure δ ou gives the minimal value for the linear combination of expectations of numbers of Type I and Type II errors for any true Gaussian graphical model S. It is interesting to compare Gaussian graphical model selection procedures known in literature with respect to this criteria for different S. Our experiences shows that the structure of S will play an important role in this comparison.
This will be a subject for a forthcoming study.