{'fraction_non_alphanumeric': 0.02706869967449032, 'fraction_numerical': 0.008360459139969162, 'mean_word_length': 4.5339400834281385, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Learning to Solve the AC Optimal Power Flow via a Lagrangian Approach

Using deep neural networks to predict the solutions of AC optimal power flow (ACOPF) problems has been an active direction of research. However, because the ACOPF is nonconvex, it is difficult to construct a good data set that contains mostly globally optimal solutions. To overcome the challenge that the training data may contain suboptimal solutions, we propose a Lagrangian-based approach. First, we use a neural network to learn the dual variables of the ACOPF problem. Then we use a second neural network to predict solutions of the partial Lagrangian from the predicted dual variables. Since the partial Lagrangian has a much better optimization landscape, we use the predicted solutions from the neural network as a warm start for the ACOPF problem. Using standard and modified IEEE 22-bus, 39-bus, and 118-bus networks, we show that our approach is able to obtain the globally optimal cost even when the training data is mostly comprised of suboptimal solutions.


I. INTRODUCTION
The AC optimal power flow (ACOPF) problems are fundamental to power system operations, but they are often computationally expensive to solve in real-time [1]. Recently, machine learning has emerged as a popular method to aid in solving ACOPFs [2]- [5]. By treating the optimization problem as a function that maps loads to the optimal solutions (voltage and angles), supervised learning techniques can be used to train a neural network (NN) that replaces a nonlinear programming solver [6]- [9]. Since training can be done offline using historical or simulated data, neural networks can be used in real-time to reduce computational burdens.
Because the solution of the ACOPF problem need to satisfy a set of nonlinear constraints, neural networks typically cannot guarantee that their outputs are feasible. Therefore, most needs an additional power flow or OPF step to obtain feasible solutions [7], [10]. Equivalently, we can think of these machine learning methods to be producing a good warm start for a solver. Since the starting point is of critical significance to a nonconvex problem, a good initialization would offer significant computational speedups in real-time [11].
A key, and often understated assumption, in using learning is that the training set if of high quality. In the context of ACOPF, the training set consists of pairs of active/reactive loads and their corresponding ACOPF solutions, and the assumption is that the solution are the optimal ones. However, the training set is typically constructed using existing noninear programming solvers, and there is no guarantee that the solutions are in fact globally optimal. Because ACOPF is not convex, there could exist multiple local solutions. These solutions can lead to large differences in the cost function, and there is no tractable way to judge the quality of solutions for a given problem. Therefore, the quality of the learned solutions are fundamentally limited by the quality of the training data sets.
In addition to the presence of suboptimal solutions, a more challenging setting for learning would be to have several solutions associated with loads that are close to each other. Because of the presence of multiple solutions, a small change in load may lead a solver to jump between distinct solutions. Therefore, the training set could include data that are close in the load, but quite different in the solutions. For a neural network trained using regression loss, it would output the average of the local solutions, leading to a initialization point that neither increase the computation speed or help with the quality of the solutions.
In this paper, we present a machine learning architecture that overcomes the challenge of multiple suboptimal local solutions in the training data set. Instead of focusing on the load/solution pairs, we seek to learn a neural network that maps load to the dual variable of the power balance constraints. These dual variables are the locational marginal prices, and would be readily available from any modern nonlinear solver. These dual variables are used to form a partial Lagrangian, whose solution we also learn via a neural network. All together, we use two neural networks, one mapping the load to the dual variables, another mapping the predicted dual variables and load to the solution of a partial Lagrangian.
We use the predicted solution of the partial Lagrangian as a warm start. Interestingly, this warm starting point tends to be closer to the globally optimal solution of the ACOPF, even if the training data set only has suboptimal solutions or a mixture of global and local solutions. Intuitively speaking, the optimization landscape of the partial Lagrangian is much "easier" than the original ACOPF problem, and its solution is robust to errors in the Lagrangian multiplier. Therefore, the quality of the learned warm start can also be potentially much better than directly learning based on load/solution pairs.
In this paper, we show that our duality-based approach outperforms existing approaches on modified IEEE 22,39 and 118 buses networks [12]. The difference is especially significant if the training data set contains some strictly suboptimal solutions. Our method maintains the advantage of using neural networks to emulate solvers: it provides a good warm start in real-time. It also has the added benefit of improving the solution quality in the training data set. Our paper is organized as follows. Section II states the problem definition, Section III gives the main algorithm, Section IV provides some geometric intuition behind the algorithm and Section V shows the simulation results.

A. ACOPF Formulation
Consider a power system network where n buses are connected by m edges. For bus i, let V i denote its voltage magnitude, θ i its angle, P G i and Q G i the active and reactive output of the generator and P D i and Q D i the active and reactive load. We use P f ij and Q f ij to denote the active and reactive power flowing from bus i to bus j. The admittance between buses i and j is g ij −b ij . We use θ ij as a shorthand for θ i −θ j .
The ACOPF problem is to minimize the cost of active power generations while satisfying a set of constraints [12]:

B. Learning Neural Networks
It is useful to view the optimization problem in (1) as a mapping from the demands P D and Q D to the solutions V and θ. Machine learning is typically used to find a proxy of this mapping by training (deep) neural networks. Suppose (1) is solved for a number of demands and the corresponding solutions are collected into a training data set. Several approaches exist for training, including direct regression [2], the emulation of a iterative solver [7], and using sensitivity information [6]. The constraints in (1f), (1g) and (1h) can be satisfied by using tanh or sigmoid activation functions in the output layer of the neural network. But the rest of the constraints would need to be satisfied by calling another power flow or optimal power flow step [7], [10].
However, all of the above methods face a common challenge, stemming from the fact that the ACOPF problem in (1) is nonconvex and may have multiple solutions [12]- [14]. A common assumption has been that during normal operations, the ACOPF would have only one solution with "practical" values. However, multiple recent works have shown that for reasonable operation conditions, there could be multiple solutions that differ significantly in cost, but all have practical values (e.g., with voltages being all close to 1 p.u.) [15]- [18].
Assuming (1) is feasible, it can have two class of solutions: local solutions and global solutions. Local solutions are all the solutions that satisfies local optimally conditions, for example, the KKT conditions or second order ones [11]. Out of this set, the solutions with the lowest cost are called the global ones. We sometimes refer to the local solutions that are not global as strict local solutions.
Over the years, many nonlinear programming (NLP) solvers have been developed for the ACOPF problem, and their speed and efficiency have improved dramatically (e.g., see [19] and the references within). However, NLP solvers are typically only able to return local solutions and there is generally no way to tell whether they are globally optimal or not. Therefore, a training data set created using NLP solvers may fundamentally limit the solution quality when a neural network is used. Importantly, the solution returned by a NLP solver is sensitive to a variety of factors, and small changes in demand can lead to a large change in the solution. For example, small changes in the demand can lead to the solution switching between global and local. Therefore, a data set may very well consist of a mixture of local and global solutions, which tend to be very confusing for the neural network to learn.

III. ALGORITHM
In this section, we describe our learning approach to find more optimal solutions to ACOPF problems using neural networks, even when the training data set contains a mixture of local and global solutions. We first give a partial Lagrangianbased approach, then we discuss the learning model and the training of neural networks. We also show how to obtain the optimal solution to the ACOPF problem using the trained neural networks.

A. Lagrangian-based Approach
In paper [20], we propose an iterative approach to improve the solution quality by alternatively solving (1) and its partial Lagrangian. The partial Lagrangian for (1) is formed by dualizing the active and reactive power balance constraints (1b) and (1c). Suppose the Lagrangian multipliers associated with (1b) and (1c) are µ P i and µ Q i , respectively, then the partial Lagrangian for (1) is: ) are the AC power flow equations.  The critical insight in [20] is that the solution of the partial Lagrangian in (2) tend to be close to the global optimal solution of the original ACOPF in (1). This is true even if the multipliers µ P and µ Q are the ones associated with the strict local solutions (see [20] for more details). Therefore, the solution of (2) is a good warm start point for an ACOPF solver. In [20], an iterative algorithm was proposed, where an ACOPF and its partial Lagrangian are repeatedly and iteratively solved to move from local solutions to global ones.
In this paper, we use learned neural networks to replace explicitly solving (1) and (2). Namely, we train two neural networks, the first to predict the dual variables of the active and reactive power balance constraints from the load, and the second to predict the solution of (2) from the load and the predicted multipliers from the first neural network. The output of the second neural network is used as a warm start point for an ACOPF solver. Since this starting point is close to the optimal solution, the ACOPF solver is solved much faster and may return a more optimal solution than a solver with a flat or random start.

B. Training of Neural Networks
Our algorithm treats (1) and its partial Lagrangian (2) as two operators and train two neural networks separately to emulate them and provide the solutions. The two models of neural network are shown in Fig. 1. For the first neural network to approximate the solution of (1), the input is the active and reactive load demands (P D , Q D ), and the output is the predicted multipliers, denoted by (μ P ,μ Q ). Let x be the collection of voltage magnitudes and angles, µ be the collection of (µ P , µ Q ), and (x i , µ i ) be the i-th pair of data in the training set, then the first neural network is trained by minimizing the following mean squared loss: where g acopf (·) represents the neural network mapping the load demand to the dual variables, and w denotes all trainable parameters.
Then we train a second neural network to emulate the solutions to the partial Lagrangian (2). The input for the second neural network is the active and reactive load demands (P D , Q D ), as well as the associated dual variable solutions (µ P , µ Q ) to (1). The output of the neural network is the solution to (2), denoted by (V,θ). Let z be the collection of inputs,x be the collection of outputs, and (z i ,x i ) be the i-th pair of data in the training set, then the second network is also trained by minimizing the mean squared loss: where g dual (·) represents the neural network mapping the load demand and dual variable solutions to the minimizer of (2).

C. Making Predictions in Real-time
After training the two neural networks, we use the process in Fig. 2 to make predictions in real-time. We use the first trained neural network to predict dual variable solutions (μ P ,μ Q ) from the load demand. Then from the predicted dual variable solutions, we use the second trained neural network to predict the solutions (V,θ) of the Lagrangian. In the end, we call the NLP solver to solve (1) using (V,θ) as the initialization. This learning algorithm is summarized below as Algorithm 1.
Algorithm 1: Solving ACOPF using learning Inputs: (P D , Q D ) Use the first trained neural network to predict multipliers: g acopf (P D , Q D ; w) −→ (μ P ,μ Q ) Use the second trained neural network to predict solutions to (2): In our algorithm, a data set that must contain a large number of globally optimal solutions is not necessary, since we are predicting high-quality warm starts instead of the ACOPF solutions. Even when the training data set contains suboptimal solutions, the solution of the Lagrangian would be a good warm start that makes the NLP solver get around being trapped at strictly local solutions. The reason is that the partial Lagrangian in (2) has "nice" geometric properties [20]: It has minimums that are near the globally optimal solution of the ACOPF problem, even for multipliers obtained at local optimal solutions of the ACOPF problem.
Also, our algorithm is robust in the sense that the solutions x to partial Lagrangian are not sensitive to the variations in µ. That is, even if the multipliersμ associated with different local solutions are different, the resulting solutions to partial Lagrangian do not change much. This also means we do not ask the predictions of the neural network to be rather accurate. Therefore, the training set for our algorithm is not necessarily to be very large. We sketch the geometric intuitions behind our algorithm in Section IV. We validate our algorithm in standard and modified IEEE benchmark systems, and report simulation results in Section V.

IV. GEOMETRY AND INTUITION
In this section, we use a 2-bus network as an example to shed some light on why Algorithm 1 might learn more globally optimal solutions, even when the training data consists of a lot of local solutions. The main reason is that Algorithm 1 predicts the solutions of the partial Lagrangian, which would be close to the global minimum of the ACOPF problem regardless of the quality of the training data. In comparison, a training set with a lot of local solutions would be harmful to the learning process of direct regression method.
In the considered 2-bus network, we ignore the reactive power and set both voltage magnitudes to 1 p.u. for simplicity. Suppose bus 1 is a generator and the reference (slack) bus with an increasing cost function c(·), and bus 2 is the load bus with angle −θ. The line admittance is g − jb. Given a load of l at bus 2 and ignoring all constraints except for the load balancing one, the ACOPF in (1) becomes This is an example of an OPF with a disconnected feasible space, since there are two discrete solutions to (5b) and we are asking for the lower cost one. To see how a NLP solver would approach this problem, we adopt the common practice in nonlinear programming and form a penalized version of (5) [11], [21]. The penalized unconstrained problem is given by where ρ is a penalty parameter. For large enough ρ, the solutions of (6) would coincide with those of (5) [11]. The function L ρ is plotted in Fig. 3. We can see that there are two local minimas, with the left one being global. The strict local minimum (the right one) satisfies both first and second order optimality conditions. Therefore, if we initialize a NLP solver with a poor starting point, it would be stuck at the strict local solution.
Since the sinusoidal functions are periodic with period 2π, let us consider the range θ ∈ [−π, π]. It is interesting now to compare the solution of L µ and the original problem in (5) (or equivalently, L ρ ). The red curve in Fig. 3 plots L µ at the local minimum and the black curve at the global minimum. We can observe an interesting fact that the minimum of L µ is close to the global minimum of L ρ , even when the multiplier at the strict local solution is used.
Let us construct a training data set with both local and global solutions for the same load, and compare the learned warm starts using direct regression and Algorithm 1. Suppose a regression method is used to minimize the distance between a predicted solution and the solutions in the training set. Since a mixture of local and global solutions are used in training, the learned neural network would make a prediction that is the average of the two solutions. This predicted solution could very well end up being at the "wrong" place, as shown in Fig. 4, where it would lead to the local solution rather than the global one. Note that, we could potentially preprocess the training data, but that is likely to be cumbersome and removes some of the appeal of using machine learning. Fig. 4: Use direct regression to learn a warm start point for the ACOPF solver. When the training set has both local and global solutions, the predicted warm start using direct regression is marked as red, which is close to the strict local minimum of the ACOPF problem. A solver initialized with this predicted warm start would converge to the local solution.
Now we show what would happen when Algorithm 1 is used on this mixed data set. For Algorithm 1, we first predict the multipliers from the load. Since the training set is a mix of local and global solutions, the predicted multiplier would be some point lying between the locally and globally optimal values. The predicted multiplier is plotted in Fig. 5a. Then we predict the solution of L µ from the predicted multiplier. For a given multiplierμ, the solution to L µ can be solved by writing out the optimality condition of (7): where c is a shorthand for c (g−g cos(θ)+b sin(θ)) and is the gradient of the cost function. By varying the multipliers, we can represent the mapping from the multipliers to the solutions of L µ as follows:θ The mapping function (9) is plotted in Fig. 5a. The set of solutions of L µ that are mapped from the multipliers varying between the locally and globally optimal values is denoted by set I. The predicted solution of L µ would lie in set I. We also plot set I in Fig. 5b. We can see that every point in set I is close to the global minimum of L ρ . More precisely, every point is in the basin of attraction of the global solution. This means if we use the predicted solution of L µ as a warm start, the solver would converge to the global minimum.
(a) Learn the mapping from the Lagrangian multipliers to the solutions of the partial Lagrangian. When the training data set is a mix of local and global solutions, the predicted multiplier and the associated solution of the partial Lagrangian are marked as red. The set I corresponds to the set of solutions of the partial Lagrangian when the multiplier varies between the locally and the globally optimal values.
(b) The learned warm start point using Algorithm 1 is marked as red, which is close to the global minimum of the ACOPF problem. If this predicted warm start is used, the solver would converge to the global solution. In fact, the set I is in the basin of attraction of the global minimum. Fig. 5: Use Algorithm 1 to learn a warm start point for the ACOPF solver. Fig. 6: Generation costs of the obtained solutions using Algorithm 1 on different training sets for 22-bus network. The predicted costs using the baseline method are also reported for comparison. All the generation costs are represented proportional to the globally optimal cost. We can see that Algorithm 1 is not sensitive to the quality of training data. It is able to obtain the global solution even when the training data is only consisted of local solutions.
In the next section, we test Algorithm 1 on IEEE benchmark systems, and show the intuition developed in this section is true for much larger and more complex problems. The simulation results show that Algorithm 1 is able to obtain globally optimal solutions even when the data set is only comprised of strictly local solutions.

V. SIMULATION RESULTS
In this section, we demonstrate the simulation results of using Algorithm 1 to predict solutions to the ACOPF problem. We test our algorithm on IEEE networks with 22, 39 and 118 buses. The full specifications for these networks can be found in [12] and [22]. The popular solver IPOPT [23] is used to generate training samples for each network. For the 22 and 118-bus networks, there exist more than one solution for a given load input. We construct data sets comprised of different proportions of local solutions, and show the performance of Algorithm 1. For a comparison baseline, we use the method in [7], where a deep neural network is trained to learn the mapping from load to optimal generation values by minimizing the loss between the learned and ground-truth values. Then power flow equations are solved to recover and ensure feasibility of the overall ACOPF solutions. We would see that our method can obtain globally optimal solutions even when the training data only contains local solutions. For the 39-bus network, we would show that using the warm starts learned by our algorithm can speed up the computation time of solving ACOPF problems using IPOPT.
We use fully-connected neural networks with 2 hidden layers for both Algorithm 1 and the baseline method. For the baseline method, the activation function of the neural network is sigmoid for all layers. For Algorithm 1, we use ReLU as activation function except for the output layer, where linear activation function is used. All neural network models are implemented using Tensorflow software library.

A. 22 bus
In the 22-bus network, there exist two solutions for a given load. The cost of the local solution is 30% higher than that of the global solution. We generate the training data by varying the load around the nominal value with 1% variations using uniform distribution. For each given load, we solve the ACOPF problem using IPOPT to obtain both solutions. Then we construct 5 different training sets by adjusting the proportion of strictly local solutions in the data. There are 4000 training samples. We use 90% of them for training and 10% for testing. We compare the obtained solution quality using Algorithm 1 to that of the baseline method.
The obtained solutions using Algorithm 1 are always feasible, since the ACOPF problem is solved in the last step. In the baseline method, the power flow equations are solved to ensure feasibility of the solution. However, the active power generation is directly predicted using the neural network in the baseline and will not be affected by the power flow step. For simplicity, we assume the obtained solutions using the baseline method are feasible, and compare the predicted generation costs to the globally optimal cost.
The generation costs of the obtained solutions using both methods on different training sets are reported in Fig. 6, where the generation costs are represented proportional to the globally optimal cost. In Fig. 6, as the proportion of strictly local solutions in the training set increases, the predicted cost using the baseline method also increases, and is larger than the globally optimal cost on every training set. In contrast, Algorithm 1 is able to obtain the global solution, even when the training set is comprised only of strictly local solutions. This implies that Algorithm 1 is not sensitive to the quality of the training set, and local solutions can also be useful. This observation carries over to larger networks and we would test our algorithm on 118-bus network in Section V-C.

B. 39 bus
For the 39-bus network that we use in this paper, there is only one solution. Since there is no difference in obtained solution quality when initialization is different, we use this network to demonstrate the computation time speed-up of calling IPOPT with the learned warm start points using Algorithm 1 as initialization. In comparison, we also initialize IPOPT with randomly generated data points using Gaussian distribution. We evaluate the computation time on Macbook Pro with Intel Core i5 8259U CPU @ 2.30GHz. We call IPOPT with both initialization for 200 instances and report the computation time for each instance in Fig. 7. The computation time of using the learned warm starts given by Algorithm 1 is plotted as the blue line, which is faster than the random initialization (red) almost for every instance. In many cases, the learned warm start is much faster than random initialization. Note that the neural networks used in Algorithm 1 are feed-forward functions, and their evaluation time (sub-milliseconds) is negligible for the comparison in Fig. 7.
To evaluate computational improvements, we also compute the average relative speedup of computation time between the two initialization methods. Using the learned warm start point from Algorithm 1 as initialization can provide an on average 12% speedup in computation time. This is because the minimizer of the Lagrangian would be close to the global Fig. 7: Computation time of calling IPOPT to solve the ACOPF problem in 39-bus network with different initialization. The blue curve is the computation time when the warm starts learned by Algorithm 1 are used as initial points, which is lower than the random initialization (red curve) almost for every instance.
solution of the ACOPF problem as we discussed in Section IV. Therefore, using the predicted solution of the Lagrangian as the initial point enables IPOPT to find the solution in a shorter time.

C. 118 bus
For the 118-bus network, there exist three solutions for a given load. The worst cost is 39% higher than the globally optimal cost. We generate the training data by varying the load around the nominal value with 0.5% variations using uniform distribution. For each given load, we solve the ACOPF problem using IPOPT to obtain the solution with the highest cost and the global solution. Then we construct 3 different training sets. Each of these training sets has half or more than half local solutions with the highest cost. There are 1900 training samples. We use 90% of them for training and 10% for testing. We compare the generation costs of the obtained solutions using Algorithm 1 to that predicted by the baseline method. All the generation costs are represented in proportional to the globally optimal cost.
The predicted generation costs using the two methods are plotted in Fig. 8. The predicted cost using the baseline method is larger than the globally optimal cost on every training set, and increases as the the quality of the training data declines (the proportion of local solutions increases). Since the neural network in the baseline method is trained by minimizing the loss between learned and ground-truth generation values, it is not a surprise that the predicted cost using the baseline method increases as the proportion of local solutions increases. In contrast, Algorithm 1 is able to obtain the globally optimal cost regardless of the quality of the training data. Even when the training data only contains the solutions with the highest cost, the predicted cost using Algorithm 1 is globally optimal. This is because Algorithm 1 does not predict solutions to the ACOPF problem, but predicts solutions to the partial Lagrangian. As we discussed in Section IV, the predicted solutions of the partial Lagrangian would be close to the global minimum of the ACOPF problem, and hence could be good warm starts for the solver to reach the global solution.

VI. CONCLUSION
In this paper, we propose a partial Lagrangian-based learning approach to predict solutions of the ACOPF problem. First, we use a neural network to learn dual variables of the ACOPF problem. Then we use a second neural network to predict solutions of the partial Lagrangian from the predicted dual variables. Using the predicted solutions of the partial Lagrangian as warm starts, the ACOPF solver can reach more globally optimal solutions. We illustrate the intuition behind our learning approach using a 2-bus network, which shows that the optimization landscape of the partial Lagrangian is much better than the original primal problem. We validate the effectiveness of our algorithm on standard 22-bus, 39-bus and 118-bus networks. The simulation results show that our algorithm is able to obtain the globally optimal cost even when the training data is only comprised of suboptimal solutions.