{'fraction_non_alphanumeric': 0.03867006856960612, 'fraction_numerical': 0.01148142242066656, 'mean_word_length': 4.202405640812941, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Study on Estimating Quantum Discord by Neural Network with Prior Knowledge

Machine learning has achieved success in many areas because of its powerful fitting ability, so we hope it can help us to solve some significant physical quantitative problems, such as quantum correlation. In this research we will use neural networks to predict the value of quantum discord. Quantum discord is a measure of quantum correlation which is defined as the difference between quantum mutual information and classical correlation for a bipartite system. Since the definition contains an optimization term, it makes analytically solving hard. For some special cases and small systems, such as two-qubit systems and some X-states, the explicit solutions have been calculated. However, for general cases, we still know very little. Therefore, we study the feasibility of estimating quantum discord by machine learning method on two-qubit systems. In order to get an interpretable and high performance model, we modify the ordinary neural network by introducing some prior knowledge which come from the analysis about quantum discord. Our results show that prior knowledge actually improve the performance of neural network.


I. INTRODUCTION
In recent years, the field of machine learning (ML) has developed rapidly with deep learning, such as the famous Alpha GO [1,2] and various technologies based on computer vision. This method also be used to solve another problem beyond computer science, such as phase transition in condensed matter [3][4][5][6], effective representation of quantum multi-body state [7], molecular nature prediction [8][9][10] and so on. Thus, we hope this method will help us to deal with some quantum information problems. In quantum information area, distinguishing a quantum state entangled or separeable is an important problem. Recognizing a bipartite state is entangled or separable by machine learning method [11][12][13] has begun to study.
Thus, the problem how to measure entanglement is more worthy of attention, especially for quantum correlation of mixed states. Our work attempts to predict a measure of quantum correlation.
In 2000, Olliver and Zurek [14,15] produced quantum discord (QD) by analogy of classical mutual information to define this measure of quantum correlation. Because of the optimization problem in the definition of QD, it is very hard to get analytical results for general cases. So, we investigate how to apply ML technology to estimate this measure of quantum correlation. In ML, give a set of data D from a function f : x → y, ML algorithm A can get an approximate function h from a hypothesis set H, by training based on the observed data. The function f is called target function. We hope to find an universal method to estimate the QD well, and it will work for any bipartite system. In theory, deep neural network (DNN) can fit any target function, however, extracting specific information form such model is hard due to the black box property of DNN. Thus we must modify the model to make extracting information easy by intrducing prior knowledge which base on the analysis about QD.
In Sect. II, we will introduce the quantum discord of two-qubit systems. In Sect. III, we will describe the neural network models we used. In Sect. IV, we will show results in two different cases. Finally, we discuss some problems and future works in Sect. V.

II. QD OF TWO-QUBIT
QD is a measure of quantum correlation mainly for mixed states. In a bipartite quantum system, there are classical correlation and quantum correlation. Assume ρ AB is the density operator of the bipartite system, and ρ A , ρ B are density operators of its two part A(lice) and B(ob). By analogy classical mutual information, Olliver and Zurek gave the definition of quantum mutual information as [14] where S(ρ) = −tr(ρ log 2 ρ) = − i λ i log 2 λ i is von Neumann entropy, λ i is eigenvalue of corresponding density operator. Quantum mutual information constant total correlation of the system, this means that quantum mutual information is the sum of classical correlation C(ρ AB ) and quantum correlation Q(ρ AB ). So we can get quantum part by [14] Q(ρ AB ) = I(ρ AB ) − C(ρ AB ).
Since, classical correlation is depend on measurement, we used to view part A by measuring part B. Give a set of basis {Π k |Π k = I ⊗ B k } to measure B, the state of the system after measuring is here p k = tr B (Π k ρ AB ) is the probability of measurement [14]. Then, the conditional entropy of the final state is Now, the mutual information of this system is [16] Henderson and Vedral [17,18] gave the explicitly form of classical correlation From the above, we can know that the quantum correlation of a bipartite system, which named QD by Olliver and Zurek [14], is There are a few states that can be solved analytically. In 2008, Luo [19,20] got analytical solution of QD for a kind of qubit-qubit system state which is ρ = I + 3 j=i c j σ j ⊗σ j . In 2010, Ali with coworkers [16] got a more general solution of a kind of state which called X-state, which elements are all zeros except main-diagonal and antidiagonal by similar method. In 2015, Maldonado-Trapp et.al. gave some supplements to the results of two-qubit X-states QD [21]. However, it is still a hard task to solve the third term in Eq.(7) for large systems or general cases.
In order to estimate QD by neural network (NN) well, we should choose an appropriate target function to reduce model complexity, In other word, the simpler taget function is, the better performance we can get. From Eq. (7), we can see that the first two terms are easy to analytically solve, so we just pay attention to the third term of Eq. (7), called optimization term. For convenience, we write it as

III. MACHINE LEARNING MODEL
Since the original parameters of a two-qubit state are too few to reveal feature directly.
Therefore, before feed data to a network, we need a feature transformation as pre-processing to make feature more obvious. We apply a power series transformation by the following rule. Let n is the dimension of the original feature space, and L is the highest power after transforming. So, for a vector x ∈ R n , the result is Φ( here a i is the power of the i-th base of x. And, we list some size of Φ for some cases in Tab.I.

A. Estimating by ordinary NN
Firstly, we use an ordinary NN attempt to estimae QD. The NN model we used has only one hidden layer, see Fig.1. Assume we have a raw data set here M is the size of data set. The input of the NN can be written as a matrix, X, forming by feature vectors and The hidden layer h has 1. An ordinary NN with one hidden layer. X is input data set, h is a hidden layer with F nodes,ỹ is output of this model. Σ means summing the input for each node, and σ is sigmoid active function. W 1 and W 2 are the weight matrices between layers.
F nodes. The values of optimization term, or labels for ML, can be written as a vector y, here y i = c (i) , is the label of the i-th data. W 1 and W 2 are weight matrices of net connections, ỹ is the predictive value of the model, and σ is the active function , σ(x) = 1/(1 + exp(−x)).

The computations of this NN are
here,λ i,j are eigenvalues of density operatorρ i . thus we replace the active function by a new "active function" In this NN model, the input of this layer may be the approximate of eigenvalues, ideally.
What' more, the change of active function can further reduce the model complexity for previous layer.
We illustrate this model in Fig.2, we call it PKNN for convenience. The computations of this model are, The second layer realizes the sum in the definition in von Neumann entropy in some way.
And, we don't need offset term in this layer according to the definition of entropy.
We should note one thing that because of the optimizer, min {B i } , the result of QD is shattered. In other word, there are some, more than one, conditions g( x) ≥ 0, cause the target function c( x) or eigenvalueλ i,j have different analytical forms in different parameter ranges, The above can be written in one formula here, θ(·) is step function Unfortunately, how much conditions are there in the target function is unknown in advance, therefore, we should choose an appropriate value in an experiment.
The layers of h and h c are used to probe eigenvaluesλ i,j and condition functions g( x), then they are an approximation of eigenvalues and conditions.

IV. RESULT
We will compare above three models in this section. To evaluate the performance of these models we use mean quadratic loss function as index: At last, these models are trained by TensorFlow [22].

A. Result on X-state
First, we test our models on two-qubit X-states, we can use a 7-dimension real vector, x ∈ R 7 , to characterize it. Thus we have ρ 11 = According to the eigenvalues of ρ AB X [16], there are some extra restrictions for those parameters: So, for X-state, the optimization term is a function of a 7-D vector: In this experiment, we use a training set with M = 6000 to inhibit over-fitting. These data are numerical solutions come from randomly generated two-qubit X-states. In order to balance direct feature and the amount of calculation, we choose L = 6 as the highest power of transformation and the second layer has F = 16 nodes. Every model runs 5 times, and runs 3 × 10 5 steps each time. Then we compare their average performance. 3 × 10 5 steps can guarantee that loss function will not change obviously in every turns. Other training parameters are same, those are initial learning rate η = 0.2 and decaying with 0.98 every 3,000 steps.
Tab.II shows all results, the last line is the average value of loss function, . We can see that PKNN is better than an ordinary NN and DBNN has the best performanceon on average. This accords with our expectation.
Random initialization is hard to find best result each time. One of the reasons is that there are lots of local minima in those models. Moreover, the non-monotonic function E(·) makes the the loss function more complex then monotonic activation function. For the same reasons, those models are unstable. Even so, these results can prove our suppose that prior knowledge can improve the performance of NN.  Then, We show the performance of the best result in Tab.II, the result is plotted in Fig.4.
The horizontal axis is the theoretical value of optimization term or label, c, and the vertical axis is the predictive value,c from DBNN. We can see that the predictive value falls in a bit wide range.
Next, we test our model on a X-state It is easy to get the analytical solution by Ali's result [16], where and We plot the analytical solution and the predictive result of a DBNN in Fig.5. It show that we have a good predictive about QD including condition.

B. Result on real state
To show the universality of our method, we test it on a special two-qubit state which is researched little, we call it real state. Its elements are all real, ρ ij ∈ R. We can use 9 independent parameters to characterize it: x 4 x 2 x 7 x 8 x 5 x 7 x 3 x 9 The solution of this state is unknown, thus, we don't know the range of non-diagonal elements. So we generate this kind of state randomly with following empirical restrictions to improve sampling efficiency, and discard the one with negative eigenvalue. Well, for a real state, the optimization term is a function of a 9-dimensional vector: The truncated term is still at L = 6. From previous calculations in Tab.I, we know that the size of first layer is 5,005, it's huge. The size of training set is same as before, M = 6000.
The initial learning rate η = 0.2, and decay with 0.96 every 3,000 steps in this experiment.
As a transitional model, we don't test PKNN this time.
Tab.III show all results and Fig.6 display the best result of DBNN in Tab.III. We can see that DBNN has similar loss with X-state on real states training set and test set. The range of data points in Fig.6 is also similar to Fig.4. By comparing the results of ordinary NN  Since the first layer of our model is simple, it will increase exponentially with the number of parameters of a system. What'a more, for a big system, more parameters mean higher power is necessary, then it lead to a huge dimension of the transformed feature space.
Thearfore, we hope to find a more effective method to express mathematical expression which should be trainable. If possible, this method maybe extended to the more general physical problems, that, finding the law through "big data" of a phenomenon will provide a new way.