{'fraction_non_alphanumeric': 0.035353643633557016, 'fraction_numerical': 0.011620178804965376, 'mean_word_length': 4.6299336149668076, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 1, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 6, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Matrix-Monotonic Optimization Part II: Multi-Variable Optimization

In contrast to Part I of this treatise [1] that focuses on the optimization problems associated with single matrix variables, in this paper, we investigate the application of the matrix-monotonic optimization framework in the optimization problems associated with multiple matrix variables. It is revealed that matrix-monotonic optimization still works even for multiple matrix-variate based optimization problems, provided that certain conditions are satisfied. Using this framework, the optimal structures of the matrix variables can be derived and the associated multiple matrix-variate optimization problems can be substantially simplified. In this paper, several specific examples are given, which are essentially open problems. Firstly, we investigate multi-user multiple-input multiple-output (MU- MIMO) uplink communications under various power constraints. Using the proposed framework, the optimal structures of the precoding matrices at each user under various power constraints can be derived. Secondly, we considered the optimization of the signal compression matrices at each sensor under various power constraints in distributed sensor networks. Finally, we investigate the transceiver optimization for multi-hop amplify-and-forward (AF) MIMO relaying networks with imperfect channel state information (CSI) under various power constraints. At the end of this paper, several simulation results are given to demonstrate the accuracy of the proposed theoretical results.

Matrix-monotonic optimization is an interesting framework that takes advantage of monotonic property in positive semidefinite matrix set to derive the optimal structures of optimization variables [1], [17]- [20]. In Part I [1], we focus our attention on single-variable optimization problems. However, for many practical optimization problems there are multiple matrix variates to optimize. For example, in multiuser multiple-input multiple-output (MU-MIMO) communication systems, the transceiver optimization processes of the downlink and uplink involve multiple matrix variables, namely the equalizer matrices and precoder matrices [21]- [24]. For multi-carrier MIMO systems, in each subcarrier there is a precoder matrix and an equalizer matrix [17]. Moreover, in multi-hop communications the forwarding matrix of each relay has to be optimized [25], [26].
This fact inspires us to take a further step and to investigate the optimization problems hinging on multiple matrixvariables. Generally speaking, solving an optimization problem having multiple matrix-variables is more challenging than its single matrix-variable counterpart. How to solve this kind of optimization problems has attracted substantial attention both across the wireless communication and signal processing research communities [21]- [23]. In contrast to single matrix-variable optimizations, for multiple matrixvariable optimization in most cases it is impossible to derive the optimal solutions in closed-form. Iterative optimization algorithms or alternating optimization algorithms have neeb widely used to solve this kind of optimization problems [23]- [27]. Unfortunately, there is no general-purpose mathematical tool or framework that can cover all the kinds of optimization problems. In some cases, similar to the single-variable case, for multiple matrix-variable optimization first the optimal structures of the matrix variables have to be derived, based on which the optimization can be significantly simplified and the corresponding convergence rate can be substantially improved.
In this paper, we investigate in detail, how to exploit the hidden monotonicity in positive semidefinite matrix fields to derive the optimal structures of the multiple matrix variables. Based on the optimal structures, the optimizations of multiple matrix variables can be significantly simplified. In our work, it is revealed that for many optimization problems associated with multiple matrix variables, the matrix-monotonic optimization framework still works. We also would like to point out that the authors of [17] also investigate how to apply matrix-monotonic optimization to optimization problems associated with multiple matrix variables. However, it is worth highlighting that the previous contribution [17] only considers a simple sum power constraint. By contrast, our work in this paper is significantly different from that in [17], since here diverse power constraints are taken into account, such as the multiple weighted power constraints of [20], the shaping constraint of [28], [29] and so on. Additionally, more scenarios are also taken into account. Furthermore, in addition to the multi-hop systems investigated in [17], in this paper, the MU-MIMO uplink and distributed sensor networks are also considered.
The main contributions of this paper are enumerated in the following. These contributions distinguish our work from the existing related works.
• Firstly, we investigate precoder optimization in the uplink of MU-MIMO communications under three different power constraints, namely the shaping constraint, joint power constraint and multiple weighted power constraints. Based on the matrix-monotonic optimization framework, the optimal structures of the matrix variables can be derived. Then the optimization can be substantially simplified and can be efficiently solved by an iterative algorithm. In each iteration based on the optimal structure, the optimal solutions of the remaining variables are standard water-filling solutions. We cover the precoder optimization under per-antenna power constraint as its special cases. • Secondly, we investigate the signal compression matrix optimization problem in a distributed sensor network under the above three power constraints. For this data fusion optimization, there exist correlations between the signals transmitted from different sensors. This makes the corresponding optimization problem significantly different from that in the MU-MIMO uplink. Moreover, in contrast to [27], where at each sensor only the sum power constraint is considered, in our work more general power constraints are taken into account, namely the shaping constraint, joint power constraint and multiple weighted power constraints. This is our main contribution. Based on the matrix-monotonic optimization framework, the optimal structures of the compression matrices can be derived and the optimal solutions of the remaining vectors are found to correspond to water-filling solutions. • Thirdly, we investigate the robust transceiver optimization problem of multi-hop amplify-and-forward (AF) cooperative MIMO networks, including both linear and nonlinear transceivers. For the linear transceivers, various kinds of performance metrics are taken into account, namely the additively Schur-convex and Schur-concave scenarios [25], [26], [30]. On the other hand, for nonlinear transceivers, both decision feedback equalizers (DFE) and Tomlinson-Harashima precoding (THP) are investigated. In contrast to [28], [31], various power constraints are taken into account in the robust transceiver optimization instead of the simple sum power constraint. Based on the proposed framework, the optimal structures of the robust transceiver design can be derived, based on which the robust transceiver optimization can be efficiently solved. Hence our contribution fills a void in the robust transceiver design literature of multi-hop AF MIMO systems under multiple weighted power constraints.
The remainder of this paper is organized as follows. In Section II, the basic properties of the framework on matrixmonotonic optimizations are given first. Following that, the MU-MIMO uplink is investigated in Section III. Compression matrix optimization for distributed sensor networks is discussed based on matrix-monotonic optimization in Section IV. In Section V, robust transceiver optimization is proposed for multi-hop AF MIMO relaying networks separately under shaping constraints, joint power constraints and multiple weighted power constraints. Several numerical results are given in Section VII, Finally, the conclusions are drawn in Section VIII. Notation: To be consistent with our Part I work [1], the following notations and definitions are used throughout this paper. The symbols Z H , Z T , Tr(Z) and |Z| denote the Hermitian transpose, transpose, trace and determinant of matrix Z, respectively. The matrix Z 1 2 is the Hermitian square root of a positive semidefinite matrix Z, which is also a positive semidefinite matrix. For an N × N matrix Z, the vector λ(Z) is defined as λ(Z) = [λ 1 (Z), · · · , λ N (Z)] T where λ i (Z) denotes the ith largest eigenvalue of Z. The symbol [Z] i,j denotes the ith-row and jth-column element. On the other hand, the symbol d(Z) denotes the vector consisting of the diagonal elements of Z. The identity matrix is denoted by I. In this paper, Λ always represents a diagonal matrix, and Λ ց and Λ ր represent a rectangular or square diagonal matrix with the diagonal elements in descending order and ascending order, respectively.

II. FUNDAMENTALS OF MATRIX-MONOTONIC OPTIMIZATION
In this paper, we investigate a real valued optimization problem with multiple complex matrix variables {X k } K k=1 which is generally formulated as where ψ k,i (·), 1 ≤ k ≤ K, 1 ≤ i ≤ I k , denotes the constraint functions. Similar to the single-variate matrix-monotonic optimization investigated in Part I [1], all constraints considered in this paper are right unitarily invariant, i.e., for arbitrary unitary matrices Q X k 's, In the following, several specific power constraints are given.
The general power constraint model is one of the main contributions of this work.

A. The Constraints on Multiple Matrix Variables
The simplest constraint is sum power constraint, i.e., the sum power across all transmit antennas is smaller than a predefined threshold. For example, in MU-MIMO uplink communications, each mobile terminal has a sum power constraint such as It is obvious that the sum power constraint is right unitarily invariant. Moreover, in order to constrain the fluctuation of the eigenvalues of X k X H k , the following joint power constraint will be used [28], [29] The difference between the sum power constraint and the joint power constraint is that there is an additional maximum eigenvalue constraint. It is obvious that the joint power constraint is right unitarily invariant. From the circuit viewpoint, each amplifier is connected to one distinct antenna. It is not reasonable to use the sum power as a constraint as the powers cannot be shared between different antennas. In other words, the individual power constraint or per-antenna power constraint is more practical, which is formulated as [21], [31], [32] X k X H k n,n ≤ P k,n , n = 1, · · · , N.
The per-antenna power constraint is also right unitarily invariant. It is worth highlighting that the per-antenna power constraint cannot include the sum power constraint as its special case.
In order to build a more general constraint model including more specific power constraints as its special cases, multiple weighted power constraints are given in the following [1], [20] Tr where I k is the number of weighted power constraints for the kth variable X k . The positive semidefinite matrices Ω k,i 's are weighting matrices. The multiple weighted power constraints are right unitarily invariant as well. Finally, in order to constrain the transmit signals to be in a desired region, shaping constraint can be used. Shaping constraint is a constraint on the covariance matrix of transmitted signals. Specifically, the shaping constraint on a matrix variable X k is defined as [28], [33] where R s k is the desired signal shaping matrix [28], [33]. The shaping constraint (7) is right unitarily invariant as well. Under these power constraints, in the following we give the properties which are the basis of application of the framework of matrix-monotonic optimization. From a mathematical perspective, the more complicated power constraints will significantly change the feasible set compared to that of the sum power constraint. This is because the sum power constraint is both right unitarily invariant and left unitarily invariant, however the general power constraints are only right unitarily invariant. In other words, the symmetry of sum power constraint does not exist for the general power constraints such as multiple weighted power constraints. It is clear that under multiple weighted power constraints the extreme values and the optimal solutions are significantly different from that under the sum power constraint. The multiple weighted power constraints model also includes the sum power constraint model as its special case. Note that the sum power constraint model is not a special case of the perantenna power constraint model. One model can include two different constraint models as its special cases. This is also an advantage of the multiple weighted power constraints model.

B. Matrix-Monotonic Properties
The framework of matrix-monotonic optimization aims at exploiting the monotonicity in positive semidefinite field to derive the optimal structures of the matrix variates. As the constraints in Opt. 1.1 are right unitarily invariant, defining X k = F k Q X k Opt. 1.1 is equivalent to the following optimization problem In our work, Opt. 1.2: satisfies the following properties. For the kth optimal unitary matrix Q X k , the objective function in Opt. 1.2 can be transferred into a function of with g 0,k (λ(F H k Π k F k )) being a monotonically decreasing function with respect to λ(F H k Π k F k ) for k = 1, · · · , K. The optimal F k is a Pareto optimal solution of the following vector optimization subproblem which is equivalent to the following matrix-monotonic optimization problem [1], [17] Opt. 1.4: max where Π k is independent of F k . Then, based on the results of Part I [1], the optimal structure of F k can be derived. Based on the optimal structures, the optimization problem can be substantially simplified. To elaborate a little further, given the optimal structures, the optimization problem Opt. 1.2 associated with multiple matrix variables can be efficiently solved in an iterative manner. It is worth noting that given the optimal structures, an iterative algorithm is still needed to solve Opt. 1.2 and in most cases the iterative algorithms used are iterative water-filling algorithms [53], [54]. Suffice to say that the convergence of this kind of algorithms can be guaranteed, but in a general case after convergence only covergence to the local optimum of the final solutions can be guaranteed. Based on Part I [1], in the following the fundamental results for Opt. 1.4 are given, which constitute the basis for the following sections. Shaping Constraint For the shaping constraint, Opt. 1.4 becomes the following optimization problem [28] Opt. 1.5: The following lemma reveals the optimal structure of F k for Opt. 1.5 with the shaping constraint.
Lemma 1 When the rank of R s k is not higher than the number of columns and the number of rows in F k , the optimal solution F opt,k of Opt. 1.5 is a square root of R s k , i.e., Joint Power Constraint Under the joint power constraint, Opt. 1.4 can be rewritten as The Pareto optimal solution F opt,k for Opt. 1.6 is given in Lemma 2.
Lemma 2 For Opt. 1.6 with the joint power constraint, the Pareto optimal solutions satisfy the following structure where the unitary matrix U Π k is specified by the EVD every diagonal element of the rectangular diagonal matrix Λ F k is smaller than √ τ k , and U Arb,k is an arbitrary unitary matrix having the appropriate dimension.

Multiple Weighted Power Constraints
Under the multiple weighted power constraints, Opt. 1.4 becomes Note that the weighted power constraints include both the sum power constraint and per-antenna power constraints as its special cases. The Pareto optimal solution F opt,k for Opt. 1.7 is given in Lemma 3.

Lemma 3
The Pareto optimal solutions of Opt. 1.6 satisfy the following structure where U Arb,k is an arbitrary unitary matrix of appropriate dimension, i Ω k,i , the nonnegative scalars α k,i are the weighting factors that ensure that the constraints Tr Ω k,i F k F H k ≤ P k,i hold and they can be computed by classic subgradient methods, while the unitary matrix U Π k is specified by the EVD In this paper, we focus our attention on the optimization problems of multiple complex matrix variates. In order to overcome the difficulties arising from the coupling relationships among the multiple matrix variates, the right unitarily invariant property of the constraints in Opt. 1.1 is exploited to introduce a series of auxiliary unitary matrices. Each auxiliary unitary matrix aligns its corresponding matrix variable to achieve extreme objective values. As a result, the optimal solutions of the matrix variables are Pareto optimal solutions of a series of single-variate matrix monotonic optimization problems. Then the optimal structure of each matrix variable can be derived, based on which the original optimization problem can be solved efficiently in an iterative manner. In the following, three specific optimization problems will be investigated, namely transceiver optimization for the multi-user MIMO (MU-MIMO) uplink, signal compression for distributed sensor networks and transceiver optimizations for multi-hop amplifyand-forward (AF) MIMO relaying networks. Generally speaking, an auxiliary unitary matrix aligns its lefthand side and righthand side with its corresponding matrix variables. The three examples are specifically chosen for characterizing the effects of the matrix variates on the auxiliary unitary matrices. Specifically, in the transceiver optimization of the MU-MIMO uplink, when optimizing the kth matrix variate, the other matrix variates only affect the righthand side of the corresponding unitary matrix. As for signal compression in distributed sensor networks, when optimizing the kth matrix variate, the effects of other matrix variates are only on the lefthand side of the corresponding unitary matrix. Finally, as for transceiver optimizations in AF MIMO relaying networks, when optimizing the kth matrix variate, the other matrix variates affect both sides of the corresponding unitary matrix.

III. MU-MIMO UPLINK COMMUNICATIONS
The first application scenario for the matrix monotonic optimization theory is found in MU MIMO uplink communications. In the MU MIMO uplink system of Fig. 1, K multiantenna aided mobile users communicate with a multi-antenna assisted base station (BS) [34]- [37]. The BS recovers the signals transmitted from all the K mobile terminals. The sum rate maximization problem associated with this MU-MIMO uplink can be formulated as follows [21], [34]- [36] Opt. 2.1: min where H k is the MIMO channel matrix between the kth user and the BS, X k is the precoding matrix at the kth user, and R n is the additive noise's covariance matrix at the BS. For the kth user, the positive definite matrix W k is the corresponding weighting matrix. Different from the work in [21], the power constraints considered in our work are more general than the per-antenna power constraints in [21]. Similar to Opt. 1.2, The objective function of Opt. 2.2 satisfies the following property, which can be exploited to optimize the multiplematrix variables where we have Therefore, based on (21) for the kth matrix variate F k Opt. 2.2 can be written in the following equivalent formula The matrix F H k H H k K −1 n k H k F k can be interpreted as the matrix version SNR for the kth user [31]. Based on Matrix Inequality 4 in Part I [1], we have The equality holds when the unitary matrix Q X k equals where the unitary matrices U SNR,k and U W k are defined based on the following EVDs From the multi-objective optimization viewpoint, the optimal solutions of Opt. 2.3 belong to the Pareto optimal solution sets of the following optimization problems for which is equivalent to Based on Lemma 1 in Section II, we readily conclude that for 1 ≤ k ≤ K, when the rank of R s k is not higher than the number of columns and the number of rows in F k , the optimal solution F opt,k of Opt. 2.3 is a square root of R s k .
2) Joint Power Constraint: We have I k = 2 and Based on Lemma 2 in Section II, we readily conclude that for 1 ≤ k ≤ K, the optimal solution F opt,k of Opt. 2.3 satisfies the following structure where the unitary matrix V H k is defined based on the SVD and every diagonal element of the rectangular diagonal matrix Λ F k is smaller than √ τ k . The diagonal matrix Λ F k can be efficiently solved using a variant water-filling algorithm [52], [54].
3) Multiple Weighted Power Constraints: In this case, we have Then based on Lemma 3 in Section II, we conclude that for 1 ≤ k ≤ K, the optimal solution F opt,k of Opt. 2.3 satisfies the following structure where the unitary matrix V H k is defined by the following SVD and the matrix Ω k is defined as The diagonal matrix Λ F k can be efficiently solved using waterfilling algorithms [53].

IV. SIGNAL COMPRESSION FOR DISTRIBUTED SENSOR NETWORKS
In the distributed sensor network illustrated in Fig. 2, the K sensors transmit their individual signals to the fusion center [38]- [47]. Specifically, the kth sensor transmits its signal x k to the fusion center, when the channel between the kth sensor and the fusion center is H k . The fusion center recovers the transmitted signals x k for 1 ≤ k ≤ K. In contrast to the scenario of MU-MIMO communications, there exist correlations among x k [27], and the correlation matrix is denoted by Note that the correlations among the signals make the optimization approach of this application totally different from that of the MU-MIMO application.
To maximize the mutual information between the received signal at the data fusion center and the signal to estimate, the signal compression can be formulated as Opt. 3.1 [27], given as where F k is the signal compression matrix at the kth sensor, R x k is the covariance matrix of the signal x k transmitted from the kth sensor, and R n k is the covariance matrix of the additive noise n k for the kth sensor signal received in its own time slot at the fusion center. Note that if all the sensors send signals at the same frequency, all the R n k are identical. If the sensors use different frequency bands, the noise covariance matrices R n k are different. Note that in [27], only the simple sum power constraint is considered, while in our work the more general multiple weighted linear power constraints are taken into account. In other words, the result derived in this section for signal compression in distributed sensor networks is novel.
For the general correlation matrix C x , it is difficult to directly decouple the optimization problem. A natural choice is to take advantage of alternating optimization algorithms among X k for 1 ≤ k ≤ K. To simplify the derivation, a permutation matrix P k is first introduced, which reorders the block diagonal matrix diag so that the following equality holds The computation of P k and the definition of Ξ k are provided in Appendix A. The permutation matrix P k aims at moving the term X H k H H k R −1 n k H k X k at the top of the block diagonal matrix. The permutation matrix P k is determined by the position of the term Note that a permutation matrix is also a unitary matrix. By further exploiting the properties of matrix determinants, Opt. 3.1 becomes equivalent to Opt. 3.2 of (40).

Opt. 3.2: min
In order to simplify Opt. 3.2, we divide P k C −1 Combining (39) and (41) leads to Further exploiting the fundamental properties of matrix determinants [27], [56], we have the following equality where Based on (43), the alternating optimization of F k for 1 ≤ k ≤ K can be performed. Specifically, the optimization problem Opt. 3.2 is transferred into: for 1 ≤ k ≤ K, It can be seen that by exploiting its block diagonal structure, the multiple-matrix-variate matrix-monotonic optimization of Opt. 3.1 is transferred into several single-matrixvariate matrix-monotonic optimization problems in the form of Opt. 3.3. For 1 ≤ k ≤ K, by introducing the auxiliary variable the optimization problem Opt. 3.3 is transferred into: Based on Matrix Inequality 3 in Part I [1], we have based on which the optimal unitary matrix Q X k equals [17] Q opt,X k = U SNR,kŪ where the unitary matrices U SNR,k andŪ H Φ k R k are defined by the following SVD and EVD, From the multi-objective optimization viewpoint, the optimal solutions of Opt. 3.4 belong to the Pareto optimal solution sets of the following optimization problems for 1 ≤ k ≤ K [17] Opt. 3.5: max which is equivalent to the following matrix-monotonic optimization problem: Opt. 3.6: max Based on the fundamental results of the previous sections derived for matrix-monotonic optimization, we have the following results. Clearly, the optimal X k equals 1) Shaping Constraint: We have I k = 1 and Based on Lemma 2 in Section II, we have when the rank of R s k is not higher than the number of columns and the number of rows in F k , the optimal solution F opt,k is a square root of R s k .
2) Joint Power Constraints: We have Based on Lemma 2 in Section II, the Pareto optimal solutions F opt,k satisfy the following structure where every diagonal element of the rectangular diagonal matrix ΛF k is smaller than √ τ k . The diagonal matrix ΛF k can be efficiently solved using a variant water-filling algorithm [52], [54].

3) Multiple Weighted Power Constraints: We have
Based on Lemma 3 in Section II, the Pareto optimal solutions F opt,k satisfy the following structure where Ω k is given by (36), whileV H k is defined by the following SVD, respectively, The diagonal matrix ΛF k can be efficiently solved using waterfilling algorithms [53], [54].

Remark 1
The results of this paper can also be applied to more complex scenarios. For example, when the CSI between a sensor and its data fusion center is imperfect, k are the estimated CSI and the channel estimation error, respectively. The correlation matrix Ψ k is a function of both the channel estimator and of the training sequence. Based on the proposed framework, the optimal structures of the optimal solutions for the robust signal compression matrices at different sensors can also be readily derived.

V. MULTI-HOP AF MIMO RELAYING NETWORKS
Multi-hop relaying communication is one of the most important enabling technologies for future flexible and highthroughput communications, such as machine-to-machine, device-to-device, vehicle-to-vehicle, internet of things or satellite communications [28], [48]. The key idea behind multihop communications is to deploy multiple relays to realize the communications between the source node and destination node [48], [49]. Before presenting our third application of transceiver optimization for multi-hop communications, we first highlight the difference between our work presented in this section and the previous conclusions in [28], [31].
• We consider a more general power constraint which includes both the per-antenna power constraint in [31] and the shaping constraints in [28] as its special cases.

Index
Objective function Optimal Q X 1 • The channel estimation errors are realistically taken into account in our work. By contrast, in [31] the CSI is assumed to be perfectly known. To the best of our knowledge, the robust transceiver optimization for multi-hop communications even under the per-antenna power constraint is still the problem not yet fully solved in the existing literature. Therefore, the results presented in this section is novel and significant.
The K-hop AF MIMO relaying network is illustrated in Fig. 3, where the source, denoted as node 0, communicates with the destination, represented by node K, with the help of the (K − 1) relays, which are nodes 1 to (K − 1). Denote the signal sent by the source as x 0 , which has the covariance matrix of σ 2 x0 I. Then the signal model in the kth hop, for 1 ≤ k ≤ K, can be expressed as where x k is the signal received by node k, H k is the channel matrix of the kth hop, and n k is the additive noise of the corresponding link with the covariance matrix σ 2 n k I, while X k is the forwarding matrix of node (k − 1). Note that S 1 is the source's transmit precoding matrix. When the channel estimation error is considered, based on a practical channel estimation scheme [15] the CSI of the kth hop is expressed as where H k and H W,k Ψ 1 2 k are the estimated CSI and the channel estimation error of the kth hop, respectively. Furthermore, Ψ k is the covariance matrix of the channel estimate, and the elements of H W,k follow the independent and identical complex Gaussian distribution CN (0, 1). For notational convenience, let us define the new variables F 1 Q X1 = X 1 , with the associated unitary matrix Q X1 , and F k Q k for 2 ≤ k ≤ K as where Q k is the associated unitary matrix, and clearly K 1 2 n0 M 0 = σ x0 I. Based on these definitions, as proved in Appendix B the MSE matrix of the data detection at the destination is expressed as, [28], [31] Based on the MSE matrix given in (74), both the linear and nonlinear transceiver optimization problems [28], [31] can be unified into the general optimization problem Opt. 4.1 given in (75). Various objective functions typically adopted for Opt. 4.1 are listed in Table I. For linear transceiver optimization, to realize different levels of fairness between different transmitted data streams, a general objective function can be formulated as an additively Schur-convex function [31], [51] or additively Schur-concave function [31], [51] of the diagonal elements of the MSE matrix, which are given by Obj. 3 and Obj. 4 [31], [51], respectively. The additively Schur-convex function f convex A-Schur (·) and the additively Schur-concave function f concave A-Schur (·) represent different levels of fairness among the diagonal elements of the data MSE matrix. When nonlinear transceivers are adopted for improving the BER performance at the cost of increased complexity, e.g., THP or DFE, the objective functions of the transceiver optimization can be formulated as a multiplicative Schur-convex function or a multiplicative Schur-concave function of the vector consisting of the squared diagonal elements of the Cholesky-decomposition triangular matrix of the MSE matrix, that is, Obj. 5 and Obj. 6 [31], [51], respectively, where L is a lower triangular matrix. The multiplicatively Schur-convex function f convex M-Schur (·) and the multiplicatively Schur-concave function f concave M-Schur (·) reflect the different levels of fairness among the different data streams, i.e., different tradeoffs among the performance of different data steams [17]. The detailed definitions of f convex A-Schur (·), f concave A-Schur (·), f convex M-Schur (·) and f concave M-Schur (·) are given in Appendix C. This appendix makes our work self-contained.
The constraints ψ k,i (F k ) ≤ 0 are right unitarily invariant, and the power constraint model of Opt. 4.1 is more general than the power constraint models considered in [17], [28], [31]. For linear transceivers with the objective functions Objs. 1-4 in Table I, C = I is an identity matrix, while for nonlinear transceiver optimization with the objective functions Obj. 5 and Obj. 6 in Table I , C is a lower triangular matrix. Specifically, we assume that the size of C is N × N . Then, for nonlinear transceivers, the optimal C satisfies [31] where L is the triangular matrix of the Cholesky decomposition of the following matrix [31] LL The optimal unitary matrices Q X k can be derived based on majorization theory. Specifically, the optimal Q k for k > 1 are derived as [26], [28], [31] where the unitary matrices V A k and U A k are defined by the following SVDs The optimal Q X1 is determined by the specific objective function, and various Q opt,X1 associated with different objective functions are also summarized in Table I. Here, the unitary matrix U Arb denotes an arbitrary matrix having the appropriate dimension. The unitary matrix U W is the unitary matrix defined by the following EVD The unitary matrix U DFT is a DFT matrix [55], [56]. Finally, the unitary matrix U GMD ensures that the triangular matrix of the Cholesky decomposition of has the same diagonal elements [31]. Given the optimal Q opt,X k and C opt , the objective function of Opt. 4.1 can be rewritten as [28] f In (81) f Eigen (·) is a monotonically decreasing function with respect to the eigenvalue vector The specific formula of f Eigen (·) is determined by the specific performance metrics. For example, for sum MSE minimization f Eigen (·) equals .
In addition, for sum rate maximization f Eigen (·) equals .
Hence, given Q opt,X k and C opt , Opt. 4.1 is transferred into Since the objective function of Opt. 4.2 is a monotonically decreasing function of λ F H k H H k K −1 n k H k F k , it can be decoupled into the following sub-problems: for 1 ≤ k ≤ K, Clearly, Opt. 4.3 is equivalent to the following matrixmonotonic optimization problem In this application, by exploiting its cascade structure, we are able to transfer the associated multiple-matrix-variate matrixmonotonic optimization problem into several single-matrixvariate matrix-monotonic optimization problems. Based on the fundamental results of the previous sections, we readily have the following results.
1) Shaping Constraint: We have I k = 1 and As proved in Part I, based on Lemma 1 in Section II, it is concluded that when the rank of R s k is not higher than the number of columns and the number of rows in F k , a suboptimal solution F opt,k that maximizes a lower bound of the objective of Opt. 4.4 is a square root of R s k . When Ψ k = 0 the lower bound is tight and then the suboptimal solution will be the Pareto optimal solution of Opt. 4.4.
2) Joint Power Constraint: We have As proved in Part I, based on Lemma 2 in in Section II for the general case Ψ k ∝ I, a suboptimal solution that maximizes a lower bound of the objective of Opt. 4.4 satisfies the following structure where Ψ k = σ 2 n k I + P k Ψ k . It is worth noting that when Ψ k = 0 or Ψ k ∝ I, the corresponding lower bound is tight. In other words, in that case the suboptimal solution is exactly the Pareto optimal solution of Opt. 4.4. The unitary matrix V H k is the right unitary matrix of the following SVD and every diagonal element of the rectangular diagonal matrix Λ F k in (89) is smaller than the following threshold τ k σ 2 n k + P k λ min (Ψ k ) / σ 2 n k + P k λ max (Ψ k ) . (91) The diagonal matrix Λ F k can be efficiently solved using a variant water-filling algorithm [53], [54].
3) Multiple weighted power constraints: We have As proved in Part I, based on Lemma 3 in Section II, we conclude that the Pareto optimal solutions F opt,k satisfy the following structure where the unitary matrix V H k is defined by the SVD and the matrix Ω k is defined by The diagonal matrix Λ F k can be efficiently solved using waterfilling algorithms [53], [54].

VI. DISCUSSIONS
In this paper, we have investigated three representative examples for the proposed framework of multi-variable matrixmonotonic optimization. Based on the proposed matrixmonotonic framework, the structure of the optimal solutions for the three largely different optimization problems can be derived in the same logic. The distinct difference between our work and existing work is that more general power constraints have been taken in account. Taking more general power constraints into account is definitely not trivial extensions. From physical meaning perspective, the considered optimization under more general power constraints includes more MIMO transceiver optimizations as its special cases. Moreover, from a mathematical viewpoint, the optimization with more general power constraints is more challenging. It is impossible to extend the existing results in the literature to the conclusions given in this paper via using simple substitutions. From convex optimization theory perspective, adding one more constraint may not change the convexity of the considered optimization problem. Specifically, adding one more linear matrix inequality on a SDP problem, the resulting problem is still a SDP problem. Adding a quadratical constraint on a QCQP problem, the resulting problem is still a QCQP. The story is totally different for the matrix-monotonic optimization framework as the matrix-monotonic optimization framework aims at deriving the structure of the optimal solutions. One more constraint will change the feasible region of matrix variate and significantly change the structure of the optimal solutions. The corresponding analytical derivations will change distinctly.
We also would like to point out that the matrix-monotonic optimization framework is applicable to more complicated communication systems. Recently, in [18] based on the matrixmonotonic optimization framework, a general framework on hybrid transceiver optimizations under sum power constraint is proposed. Different from the fully digital MIMO systems, in a typical hybrid MIMO system, at the source or the destination the precoder or the receiver consists of two parts, i.e., analog part and digital part. For the analog part, only the phase of the signal at each antenna is adjustable. After that, in [19] based on the matrix-monotonic optimization framework, a framework on the transceiver optimizations for multi-hop AF hybrid MIMO relaying systems is further proposed. In multihop communications, the forwarding matrix at each relay consists of three parts, the left analog part, the inner digital part and the right analog part.

A. Two-user MIMO Uplink
We first consider the MU-MIMO uplink, where a pair of 4-antenna mobile users communicate with an 8-antenna BS. We define P k σ 2 n as the SNR for the kth user, where P k is the sum transmit power of user k and σ 2 n is the noise power at each receive antenna of the BS. Without loss of generality, the same maximum transmit power is assumed for all the users, i.e., P 1 = P 2 . Based on the Kronecker correlation model [13]- [15], the spatial correlation matrix R Rx of the BS's receive antennas and the spatial correlation matrix R Tx,k of the kth user's transmit antennas, where k = 1, 2, are specified respectively by R Rx i,j = r |i−j| r and R Tx,k i,j = r |i−j| t,k . In the simulations, we further set r t,1 = r t,2 = r t . Three power constraints, namely, the shaping constraint, the joint power constraint and the per-antenna power constraints, are considered. For the shaping constraint, the widely used Kronecker correlation model of R s k i,j = 0.6 |i−j| is employed [28]. For the joint power constraint, the threshold is chosen as τ k = 1.4. For the per-antenna power constraints, the power limits for the four antennas of each user are set to 1.2, 1.2, 0.8 and 0.8, respectively. It is worth highlighting that the transceiver optimization under these three power constraints can be transferred into convex optimization problems, which can be solved numerically using the CVX tool [58]. This approach however suffers from high computational complexity, especially for high dimensional antenna arrays. By contrast, our approach presented in Section III provides the optimal closed-form solutions for the same transceiver optimization design problems. Fig. 4 compares the sum rate performance as the function of the SNR for the proposed closed-form solutions and for the numerical optimization solutions computed by the CVX tool. It can be seen that our closed-form solutions have an identical performance to the solutions computed by the CVX tool.

B. Signal Compression for Distributed Sensor Networks
In this subsection, we investigate the performance of the proposed algorithm employed for signal compression in distributed sensor networks. Specifically, the distributed sensor network considered consists of K sensors and a data fusion center. Each sensor is equipped with 4 antennas and the data fusion center is equipped with 8 antennas. The perantenna power constraints for the four antennas of each sensor are set to 1.2, 1.2, 0.8 and 0.8, respectively. For the signal correlations between different sensors, the distance-dependent correlation matrix model of [27] is adopted. Specifically, we have R xm,n = e −dm,n I for the mth sensor and the for the nth sensor, where d m,n is the correlation between these two sensors. In our simulations, d m,n is distributed uniformly between 0 and 1. In order to quantify the performance advantages attained, a benchmark algorithm based on CVX is used in this subsection. The algorithm based on CVX aims for minimizing the weighted sum MSE under per-antenna power constraints, which is termed as the linear minimum mean square error (LMMSE) algorithm. In the LMMSE algorithm, the signal compression matrices of the different sensors and the combiner matrix at the data fusion center are optimized iteratively. At each iteration, the optimization problem considered is a standard QCQP problem, which can be readily solved by CVX. Observe in Fig. 5 that the proposed algorithm always outperforms the CVX-based benchmarker.

C. Dual-hop AF MIMO Relaying Network
A dual-hop AF MIMO relaying network is simulated, which consists of one source, one relay and one destination. All the nodes are equipped with 4 antennas. At the source and relay, per-antenna power constraints are imposed. Specifically, the power limits for the four antennas are set as 1, 1, 1 and 1, respectively. The SNR in each hop is defined as the ratio between the transmit power and the noise variance, i.e., SNR k = P k σ 2 n k . Without loss of generality, the SNRs in the both hops are assumed to be the same, namely, SNR 1 = SNR 2 = SNR.
In contrast to the existing works [28], [31], which consider the transceiver optimization unrealistically with the perfect CSI, in this paper, we focus on the robust transceiver optimization, which takes into account the channel estimation error. In the simulations, the estimated channel matrix is generated according to H k = H W,k Ψ Without loss of generality, we assume σ 2 e1 = σ 2 e2 = σ 2 e . It can be seen from Fig. 6 that our robust design achieves better sum rate performance than the non-robust design of [31]. Furthermore, as expected, the performance gap between the robust and non-robust designs becomes larger as the channel estimation error increases.

VIII. CONCLUSIONS
In this paper, we investigated the application of the framework of matrix-monotonic optimization in the optimizations with multiple matrix-variates. It is shown that when several properties are satisfied, the framework of matrix-monotonic optimization still works, based on which the optimal structures of multiple matrix-variates can be derived. Then the multiple matrix-variable optimizations can be effectively solved in iterative manners. Three specific examples are also given in this paper to verify the validity of the proposed multi-variable matrix-monotonic optimization framework. Specifically, under various power constraints, i.e., sum power constraint, shaping constraints, joint power constraints and multiple weighted power constraints, the transceiver optimizations for uplink MIMO communications, the compression matrix optimizations for distributed sensor networks, and the robust transceiver optimizations for multi-hop AF MIMO relaying systems have been investigated. At the end of this paper, several numerical results demonstrated the accuracy and performance advantages of the proposed multi-variable matrix-monotonic optimization framework.
APPENDIX A COMPUTATION OF P k AND Ξ k Given the following block diagonal matrix the permutation matrix P k aims at changing the orders of the kth element X H k H H k R −1 n k H k X k and the first element X H 1 H H 1 R −1 n1 H 1 X 1 along the diagonal line. Before constructing P , we first give an identity matrix I that has the same dimensions as Φ. Moreover, I can be interpreted as a block diagonal matrix as where I k is an identity matrix of the same dimensions as Moreover, I is further divided into the following submatrices where I k and I k have the same row number for 1 ≤ k ≤ K. Based on the above definitions of I k 's, we have and Therefore, based on (98) P k is constructed by interchanging I 1 and I k , i.e., . .
It is obvious that P k is a unitary matrix, i.e., P k P H k = I and P H k P k = I.
Based on (101) and together with (99) and (100), we have where Ξ k is the following block diagonal matrix

APPENDIX B MSE MATRIX FOR MULTI-HOP COMMUNICATIONS
Based on the signal model given in (69), at the destination the received signal y equals After performing a linear equalizer G, the signal estimation MSE matrix at the destination can be written in the following formula [26], [28], [31] Φ MSE G, {X k } K k=1 , C = E{(Gy − Cx 0 )(Gy − Cx 0 ) H } where C = I + B and B is a strictly lower triangular matrix [17]. For the linear transceivers, B is a constant matrix, i.e., B = 0. On the other hand for the nonlinear transceivers with THP or DFE, B corresponds to the feedback operations and

APPENDIX C FUNDAMENTAL DEFINITIONS OF MAJORIZATION THEORY
A brief introduction of majorization theory is given in this appendix. Generally speaking, majorization theory is an important branch of matrix inequality theory [57]. Majorization theory is a very useful mathematical tool to prove the inequalities for the diagonal elements of matrices, the eigenvalues of matrices and the singular values of matrices. Majorization theory can reveal the relationships between diagonal elements and eigenvalues, based on which some extrema can be computed. Moreover, majorization theory can quantitatively analyze the relationships between the eigenvalues or singular values of matrix products and matrix additions and that of the involved individual matrices. Based on majorization theory, a rich body of useful matrix inequalities can be derived, based on which the extrema of the matrix variate functions can be derived. The definitions of additively Schur-convex, additively Schurconcave, multiplicatively Schur-convex and multiplciatively Schur-concave functions are given in the following. Meanwhile, we would like to point out that Schur-convex function is a kind of increasing function and Schur-concave function is a kind of decreasing function [28]. They actually have no relationship with the traditional convex or concave properties defined in the convex optimization theory [50].