{'fraction_non_alphanumeric': 0.03391146084256731, 'fraction_numerical': 0.008029548739360848, 'mean_word_length': 4.456842412735504, 'pattern_counts': {'":': 0, '<': 4, '<?xml version=': 0, '>': 1, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 7, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Cooperation and Storage Tradeoffs in Power-Grids with Renewable Energy Resources

One of the most important challenges in smart grid systems is the integration of renewable energy resources into its design. In this work, two different techniques to mitigate the time varying and intermittent nature of renewable energy generation are considered. The first one is the use of storage, which smooths out the fluctuations in the renewable energy generation across time. The second technique is the concept of distributed generation combined with cooperation by exchanging energy among the distributed sources. This technique averages out the variation in energy production across space. This paper analyzes the trade-off between these two techniques. The problem is formulated as a stochastic optimization problem with the objective of minimizing the time average cost of energy exchange within the grid. First, an analytical model of the optimal cost is provided by investigating the steady state of the system for some specific scenarios. Then, an algorithm to solve the cost minimization problem using the technique of Lyapunov optimization is developed and results for the performance of the algorithm are provided. These results show that in the presence of limited storage devices, the grid can benefit greatly from cooperation, whereas in the presence of large storage capacity, cooperation does not yield much benefit. Further, it is observed that most of the gains from cooperation can be obtained by exchanging energy only among a few energy harvesting sources.


I. INTRODUCTION
Renewable energy provides a greener alternative to traditional fossil fuel based electric power generation. Thus, there has been significant emphasis on integration of renewable energy into smart grid design [1]. However, a significant challenge lies in the inherently stochastic and intermittent nature of renewable energy production. A popular technique to compensate for this is the use of expensive fast-ramping fuel-based generators as a back-up. However, with greater penetration of renewable energy this technique is no longer cost effective [2]. As a result, there is a need for more cost effective solutions such as the use of energy storage [3], and load scheduling by demand-response [4]. Prior work on design and analysis of renewable energy with storage includes [5], which formulates the finite horizon optimal power flow problem with storage as a convex optimization problem. This work shows that for the special case of a single generator and single load, the optimal policy is to charge the battery at the beginning and discharge towards the end of the time horizon. [6] and [7] formulate the problem as a dynamic programming problem and derive threshold based control policies for battery charging and discharging decisions. In [8], storage is used as a means to reduce the time average electricity bills in data center applications. Using a Lyapunov optimization based approach, this work shows that increasing the storage capacity results in a greater reduction in the electricity bills. Other relevant works include [9] and [10] (and references within). On the other hand, prior work on demand response includes [11], which formulates the problem of scheduling the power consumption as a Markov decision problem, where the scheduler has access to the past and the current prices, but only statistical knowledge about future prices. It is shown that incorporating the statistical knowledge into the scheduling policies can result in significant savings. [12] considers the problem of demand response in a finite time domain and solves the problem using convex optimization based techniques. The multi-period power procurement is handled in [13], where it is solved using stochastic gradient based techniques. The combination of storage and demandresponse has been examined in [14], and is solved using Lyapunov optimization based approach with the conclusion that storage combined with demand response can give greater cost savings.
One of the other techniques to combat the intermittent nature of renewable energy that has been explored relatively less, is the use cooperation in distributed power generation units [15]. The idea is to exploit the averaging effect produced by diversity in renewable energy production across different geographical areas. By enabling cooperation, areas that have excess production can transfer energy to areas that are deficient. For example, studies have been conducted by monitoring the renewable energy production across 5 different states in the United States (Arizona, Colorado, Nevada, New Mexico, and Wyoming). These studies have shown that while the variability of the load greatly increases with an increase in penetration of renewable energy in the individual states, aggregating the diverse renewable resources over these geographic areas leads to only a slight increase in the load variability [15]. This also leads to a very substantial reduction in the operating cost of the grid as well ($2 billion in this case). In terms of analytical results, the impact of aggregation of wind power has been considered in the framework of coalitional game theory in [16] and [17], where it is shown that independent wind producers can benefit by aggregating their harvested energy. Distributed energy production has also been studied within the framework of micro-grids (MGs) [18], with a focus on distributed storage and decentralized control of MG networks [19], [20]. Energy sharing among MGs has also been studied in [21] via simulations, and shown to reduce energy losses in the network.
While all of the above mentioned works address the issues of storage, demand response and aggregation individually, the combination of the dual averaging effect produced by storage and cooperation by energy sharing has not been explored. The objective of this work is to provide an analytical framework for studying the trade-off present between storage and cooperation. We consider a scenario consisting of MGs that are powered by harvesting renewable energy and are serving their respective loads. The MGs have finite capacity storage units and can cooperate by transferring energy among themselves. For any excess load, the MGs can borrow energy from the macro-grid. The objective is to minimize the time average cost of energy exchange within the entire grid. We first provide an analytical characterization of the optimal cost by examining the steady state behavior of the system for some particular settings. We then provide an online algorithm to solve the optimization problem using the technique of Lyapunov optimization [22]. The control decision to be taken at each time slot is how to divide the excess renewable energy optimally between storage and cooperation, and how much energy is to be borrowed from the macro-grid. We analyze the optimal cost as a function of the storage capacity and the number of cooperating MGs. We also investigate the following question: for a given number of cooperating MGs, what is the storage capacity needed in order to be self sufficient (i.e. to eliminate the need for energy transfer from the macro-grid).
Our result shows that when the storage capacity is low, cooperation among the MGs yields a significant reduction in the time average cost of energy exchange. However, when the MGs have a large storage capacity, cooperation does not yield much benefit. This is because each MGs can simply store all its excess harvested energy and use it during the time slots when it is deficient. Further, most of the gains are obtained by cooperation among only a few neighboring MGs.
The rest of the paper is organized as follows. We present the system model and provide the problem formulation of minimizing the time average cost of energy transfer across the grid in Section II. We provide the analytical modeling of the optimal cost for some special cases in Section III. Then, in Section IV, we present an algorithm to solve the time average cost minimization problem using the technique of Lyapunov optimization, and provide results for the algorithm performance. Numerical results are presented in Section V followed by conclusions in Section VI. Finally, the proofs of some results in the paper are presented in Appendices A, B and C.

II. SYSTEM MODEL
We consider an inter-connected power grid consisting of N MGs and a macro-grid as shown in Figure 1. The MGs are capable of harvesting renewable energy (e.g. wind, solar energy etc). In addition, the MGs are equipped with batteries in which they can store the harvested energy for future use.
A. Energy Supply, Demand and Distribution Models 1) Energy Generation: MG i harvests X i [t], i = 1, . . . , N units of energy during the time slot t, which is the only source of energy generation at the MG. We assume that the harvested energy X i [t] evolves according to an independent and identically distributed (i.i.d.) random process across time 1 . However, the energy harvesting process can be arbitrarily correlated across different MGs. The macro-grid generates energy from convention energy sources. We assume that the macro-grid has a very large supply of energy (and do not impose any constraint on its energy generation).
2) Load Serving: MG i serves a set of users whose aggregate energy 2 demand is L i [t] units of energy per time slot. The energy demand is bounded as L i [t] ≤ L max , for finite L max . The energy demand is met in the following manner.
Firstly, the harvested energy is used to serve the load L i [t]. We consider the two cases as follows: , then MG i uses all the harvested energy to serve its load. The unsatisfied load is denoted byL i.e., MG j has excess harvested energy (i.e. the harvested energy is greater than its demand).

Transfer energy from the macro-grid
In case the energy from the battery and the energy borrowed from neighboring MGs is insufficient to satisfy the demand, MG i can borrow G i [t] units of energy from the marco-grid. The sum of energy drawn from the battery, energy exchange with the neighboring MGs, and the energy borrowed from the macro-grid must satisfy the residual demand, i.e., ∀t, i = 1, . . . , N.
(1)  • Now we consider the second case, in which the harvested energy exceeds the energy demand, i.e., if , then the MG does the following: 1. As previously mentioned, it can donate an amount B i,j [t] to satisfy the load of MG j .

Store an amount
own battery to be used at a later time. Accordingly, at each time t, we have 3) Energy Storage: Next, we consider the energy model for the battery at the MGs. At MG i , the battery evolves according to the following rule: where the energy availability constrains the battery at each MG i to satisfy We also impose a battery discharge constraint during every time slot t, namely, where B s max is the maximum discharge of the battery per time slot. Furthermore, the energy storage device has finite capacity of E max units as follows: Additionally, we make the following practical assumption on the battery capacity: The constraints (4) and (5) 3 . Further, we assume that the costs are bounded as p i,j [t] ≤ p max ∀i, j, t for some finite p max , and q i [t] ≤ q max ∀i, t for some finite q max .
The total cost incurred for energy transfer to MG i during the time slot t (denoted by Cost i [t])is given by The objective of the controller is to design the system parameters in order to minimize the time average cost of energy transfer across the grid subject to the renewable energy generation and battery constraints, stated as follows: During each time slot t, the decision variables We denote the optimal value of the cost function Note that we have explicitly mentioned the subscript N to denote the optimal solution when N MGs cooperate.

B. Discussion of the System Model
We now provide some remarks on the system model.
• Note that in this work, we assume that the excess renewable energy from MG i can be exchanged with MG j only to satisfy the load of MG j during the same time slot. In other words, MG j cannot use the excess renewable energy of MG i to charge its battery. Further, we assume that the energy stored in the battery of MG i can only be used to serve its own load during a future time slot, and there is no energy exchange possible from the battery. These restrictions are imposed in order to clearly exhibit the trade-off between storing energy for future use and cooperation in the current time slot (by exchanging energy in the current time slot with neighboring MGs). The framework developed in this work can be easily extended to incorporate all these cases.
• Transferring energy between different elements of the grid incurs energy losses. The cost of energy transfer {p i,j [t], q i [t]} considered in this work can be interpreted as a price paid for these power losses. Typically, the distance between the MGs is much less compared to their distance from the macro-grid. Therefore, in practice, the cost of energy exchange between the MGs is lower, i.e., p i, . Moreover, the MGs can be connected by a short distance DC power line which incurs much less energy loss than the AC power line connection between the MGs and the macro-grid [23]. The objective function can be viewed as minimizing the cost associated with the energy losses.
• A more practical set-up in which the MGs are connected to different buses, and energy exchange performed via a constrained transmission network is not considered this work. This is done in order to clearly illustrate the trade-offs between cooperation and storage, without the complications arising from physical power flow constraints. The more practical case incorporating transmission network constraints is left for future investigation.

III. AN ANALYTICAL CHARACTERIZATION OF THE OPTIMAL COST
Before solving the optimization problem in (11), we first provide an analytical characterization of the time average cost for certain specific scenarios.

A. Single MG scenario
First, we consider the analysis only for a single MG which is equipped with a storage device. There is no energy to be exchanged, and any unsatisfied load must be fulfilled by using the energy stored in storage device, or by borrowing energy from the macro-grid. We make the following simplifications.
1) Modeling the energy arrival process: We assume that the energy arrivals per time slot are integer random variables. In addition, we consider the probability mass function (p.m.f.) of the excess energy arrival process 4 i.e., to be given by where Here, we have assumed that −M ≤X[t] ≤ M. Note that the random processX [t] can be used to model discrete random processes like the Poisson process.
2) Modeling the storage: As before, let us assume that the MG has storage capability represented as a virtual energy queue whose queue-length at time t is given by E[t], and the maximum battery capacity being E max . The evolution of the battery can be modeled as a random walk which evolves as follows: Assuming that the arrivals are i.i.d. across time, it can be verified that the random process ( The state diagram of the Markov chain corresponding to the random process E[t] is shown in Figure 2. We denote π t [i] = P(E[t] = i). Let us assume that the limiting state distribution function π(i) = lim t→∞ π t (i) exists for all 0 ≤ i ≤ E max and denote π = [π(1), π(2), . . . , π(E max )].
We denote the transition matrix of this Markov chain by P. The Markov chain corresponding to the random process E[t] has a limiting distribution π(i) = lim t→∞ π t (i) for all 0 ≤ i ≤ E max , independent of the initial distribution, if the system of equations has a strictly positive solution [24]. If the limiting distribution exists, then an analytical expression for the cost incurred in borrowing energy from the macrogrid (denoted by Cost) can be derived as follows: In the steady state, when the excess renewable energy arrival is negative, i.e., X = i for i = −1, . . . , −M (which happens with probability d i , i = 1, . . . , M, respectively), and when the battery is in the state j for j ≤ i, (which happens with probability π j ), the MG has to borrow i − j units of energy from the marco-grid at the price q max per unit. Mathematically we can write, Example 1. Consider the special case in which the the excess energy arrival process has the following distribution: The Markov chain associated with the random process E[t] in this case is illustrated in Figure 3. In this case, the stationary distribution of the Markov chain corresponding to the random walk has a simple form given by where r = a/d. Therefore, the cost of borrowing energy from the marco-grid is given by B. The case of multiple micro-grids The characterization of the optimal cost in the case of multiple MGs is complicated due to the fact the excess energy arrivals in the grids are correlated because of the ability to share energy among themselves. Therefore, we provide a closed form analytical characterization only in the special case case of a completely symmetric two MG set-up.
Consider a symmetric scenario consisting of 2 MGs (MG 1 and MG 2 respectively), such that the cost of energy exchanges p 1,2 = p 2,1 = p max and q 1 = p 2 = q max . Each MG has an excess energy arrival process whose p.m.f is given as in (19). Further, we consider a special case when the energy arrival process is independent across the two MGs. In order to model the energy transfer between the grid, we consider the following policy. Whenever, MG 1 produces excess energy and MG 2 has an energy deficit, MG 1 transfers its excess energy to MG 2 with probability α ∈ [0, 1]. Otherwise, MG 1 stores the excess energy into its battery with a probability 1 − α. Since the system is perfectly symmetric, MG 2 does the same in the case when it over produces and MG 1 has an energy deficit. In all other cases, there is no requirement to exchange energy between them. Let us denote the random variableZ 1 andZ 2 representing the effective excess energy arrival. It is related tõ X 1 andX 2 as follows: • IfX 1 = −1 andX 2 = 1 • In all other cases,Z 1 =X 1 andZ 2 =X 2 . Therefore, the unconditional p.m.f. ofZ 1 andZ 2 can be computed by integrating out the other variable. It is given as The evolution of the battery can now be modeled as a random walk which evolves as The steady state distribution of the Markov chain corresponding to this random walk is given by where Therefore, the cost of energy exchange within the grid has two components: energy exchanged among the MGs and the energy borrowed from the macro-grid. Mathematically, this is given as The minimum cost is then obtained by optimizing over the choice of α. Let us define and hence, the minimum cost of energy exchange is given by Cost(α * ). From (30), we can analyze the following two extreme cases, namely, the case with no storage and the case with infinite storage.
• E max = 0 -No storage In this case π(0) = 1. Therefore, and the cost is minimized when, Thus, in the absence of storage, the MGs must always share the excess available energy as long as p max < q max . The result is quite intuitive since in the absence of storage, one can always reduce the cost by exchanging energy locally between the MGs.
In this case, π(0) = d−a d(1−αa) . Therefore, Thus, the cost is minimized when α = 0. This implies that in the presence of infinite storage, any excess energy must always be stored rather than exchanging among them. In what follows, we provide a practical algorithm to solve the time average cost minimization problem across the grid using the technique of Lyapunov optimization technique.

IV. ALGORITHM DESIGN BASED ON LYAPUNOV OPTIMIZATION
The Lyapunov optimization method provides simple online solutions based only on the current knowledge of the system state as opposed to traditional approaches such as dynamic programming and Markov decision processes which suffer from very high complexity and require a-priori knowledge of the statistics of all the random processes in the system.
But first, we note that the technique of Lyapunov optimization is not directly applicable for solving (11). This is due to the presence of constraints (8) and (9), which have the effect of coupling the control decisions across time slots. In order to circumvent this issue, we consider an approach similar to [8], and formulate a slightly modified version of this problem, stated as follows: Note that in (35), all the constraints associated with the battery are relaxed, and a constraints of the form ∀i are added. Effectively, this constraint represents the condition for stability of the virtual energy queue associated with the battery. We will henceforth address this problem as the relaxed problem. Let us denote by g * N , the optimal value of the cost function of the relaxed problem over all possible control decisions. First, it is easy to see that g * N ≤ f * N , i.e. the solution to the relaxed problem acts as a lower bound on the original problem (since the relaxed problem has fewer constraints compared to the original problem and any feasible solution of the original problem is feasible for the relaxed problem as well).
We now focus on solving the relaxed problem. It can be shown that the optimal solution to the relaxed problem can be obtained by the method of stationary randomize policy, stated in the following theorem. Theorem 1. There exists a stationary and randomized policy Π that achieves and satisfies the constraints: The existence of such a policy can be proved by using the Caratheodory theorem, similar to the arguments in [22] and omitted here for brevity. Note that due to the high dimensionality, it is not practical to solve the problem using the method of stationary randomized policies.
In what follows, we apply Lyapunov optimization to solve the relaxed problem (35). Further, we will show that the solution developed for (35) by our method also satisfies all the constraints associated with battery, hence making it applicable for solving the original problem (11).
We proceed by considering the Lyapunov function associated with the virtual energy queues defined as follows: where θ is a perturbation which is given by θ = B s max + V q max . The exact rationale behind the choice of the value of θ will be specified later when we analyze the algorithm performance.
We will now examine the Lyapunov drift which represents the expected change in the Lyapunov function from one time slot to the other, which is defined as where the expectation is with respect to the random processes associated with the system, given the energy queue-length values ]. Using the equation for evolution of the virtual energy queue associated with the battery in (3), and some standard manipulations, it can be shown that the Lyapunov drift can be bounded as where C < ∞ is a constant. For completeness, we provide the proof of this step in Appendix A. We will henceforth denoteẼ (where V is another control parameter which will be specified later) to both the sides and denoting we have Using (1) Substituting for G i [t] in the right hand side (40), we have, From the theory of Lyapunov optimization (drift-plus penalty method), the control actions are chosen during each time slot to minimize the bound on the modified Lyapunov drift function (on the right hand side of (42)) [22]. Before we proceed, we provide the main intuition behind the solving the relaxed problem using this approach. Notice that the relaxed problem can viewed as minimizing the time average cost of energy exchange in the grid while maintaining the stability of the virtual energy queue (battery). The modified Lyapunov drift has two components, the Lyapunov drift term ∆[t], and V × Cost[t] term. Intuitively, minimizing the Lyapunov drift term alone pushes the queue-length of the virtual energy queue to a lower value. The second metric V × Cost[t] can be viewed as the penalty term, with the parameter V representing the trade-off between minimizing the queue-length drift and minimizing the penalty function. Greater value of V represents greater priority to minimizing the cost metric at the expense of greater size of the virtual energy queue and vice versa. This is indeed the rationale behind minimizing the modified Lyapunov drift ∆ V [t] during each time slot. The control algorithm using the aforementioned rule can be described as follows. During each time slot t, one must choose the control decisions as a solution to the following linear programming problem (obtained by minimzing the right hand side of (42)) : Let us denote the solution corresponding to (43) as where (x) + = max(x, 0).

A. Algorithm Performance Analysis
We will now analyze the performance of the algorithm described in the previous section. Lemma 1. By choosing the parameters V and θ as the following hold true: From the result of Lemma 1, it can be seen that the battery charging decisions are non-zero only when E i [t] < E max − Y max and the discharge decisions are non-zero only when E i [t] > B s max . As a consequence of this lemma, it can be verified that the algorithm developed satisfies all the constraints associated with the battery in the original problem, i.e., constraints (8) and (9). Therefore, the algorithm developed is a feasible algorithm for the original optimization problem, and it can be applied to the original system under consideration (with finite battery capacity constraints). This also justifies the choice of the perturbation parameter θ in our design.
Let us denote the cost function f We will now provide the main result related to the performance of our algorithm.
Theorem 2. For the algorithm developed in the previous subsection, the virtual energy queue-lengths can be bounded as follows: and the time average cost function achieved by this algorithm satisfies whereB < ∞ is a constant.
Proof: See Appendix C. Theorem 2 implies that performance of our algorithm can be made arbitrarily close to the optimal value by increasing the value of parameter V. However, this comes at the cost of increasing the battery capacity E max (due to the bound on the value of V in (45) , given E max ). Also, note that since g * N ≤ f * N , the performance bounds of (48) hold with respect f * N as well (and are infact tighter).

V. NUMERICAL RESULTS
In this section, we present some numerical results to examine the analytical modeling of the cost function and the Lyapunov optimization based algorithm described in the previous section.
First, we plot the optimal time average cost as a function of the storage size for the single MG case based on the analytical expression of (21), and by running the Lyapunov optimization based algorithm for T = 5000 iterations in Figure 4. The p.m.f of the excess energy arrival process considered in the simulations is provided in (19), with d = 0.5, a = 0.2. We observe that there is a good match between the two curves and the time average cost decreases with an increase in storage capacity.  Next, we consider the normalized time average cost for different combinations of the number of cooperating MGs and the storage capacity. We consider the following setting. We assume that the actual harvested renewable energy consists of two components, namely the predicted component which is deterministic, and a prediction error which is random i.e., . For the sake of simplicity, we assume that the predicted componentX i [t] perfectly matches the aggregate load L i [t]. We model the excess renewable energy pro- (also the prediction error W i [t] in this case) by the truncated normal distribution in accordance with some previous studies in this field [25]. Therefore, excess renewable energy production is a zero mean random process with its distribution given byX i [t] ∼ N (0, ν 2 ), where ν is the standard deviation. Note that following this model of energy production (i.e.,X i [t] = L i [t] and mean value of X i [t] = 0), we ensure that the aggregate energy production of each MG matches the aggregate load, and the only reason to store/exchange energy is to compensate for the fluctuations in renewable energy (thus capturing the most essential aspect of the problem considered in this work). In our numerical results, we choose the value of L i [t] = L i = 10 MW, ∀t, i and the variance ν = 3 MW.
In order to construct a wind farm of MGs, we choose a square grid of dimensions 10 × 10 km. The positions of the micro-grids are chosen by generating uniform random numbers within this grid. We consider that the macro-grid is located at the coordinate (20, 20) km. A random snapshot the power grid with N = 5 MGs is shown in Figure 5. We assume that the cost of transporting energy among different elements of the grid is directly proportional to the distance between them. Therefore, p i,j [t] = βd i,j , ∀t, units (where d i,j is the distance between MG i and MG j in km) and q i [t] = βD i , ∀t, (where D i is the distance between MG i and the macro-grid in km) and β being the proportionality constant. In our numerical results, we use β = 1. For each random snapshot the power grid, we run the simulation for T = 5000 time slots. Let us consider the time average cost incurred in transferring energy across the grid, where N is the number of cooperating MGs. In order to average out the position of the elements of the grid, we generate 100 random snapshots of the power grid and obtain an average of the normalized cost for these 100 positions. We plot the normalized cost incurredC N N as a function of N, for different values of storage capacity (2, 5, 10, 20 and 50 MWh) in Figure 6. In each of these cases, we choose B s max and Y max to satisfy the constraint (7). Specifically, we choose (B s max , Y max ) = (0.5, 0.5), (1, 1), (2, 2), (5,5), (10,10) MW in the five cases of storage capacity (2, 5, 10, 20 and 50 MWh) respectively. The value of B ex max chosen in 10 MW. The following observations can be made.
• For a given storage capacity, the cost of energy exchange decreases with an increase in the number of cooperating MGs. This is due to the fact that greater number of cooperating MGs leads to a greater diversity in energy production and hence greater possibility of sharing energy among the MGs (hence reducing the need for borrowing energy from the macro-grid). • The decrease in the normalized cost (as a function of the number of cooperating MGs) is greater for lower values of storage capacity. For higher values of storage capacity, the normalized cost does not reduce with increasing N . This is due to the fact that with greater storage capacity, the MGs are able to store any excess harvested energy (during the time slots when the harvested energy is greater than the aggregate load) and use it during the time slots when the harvested energy is deficient, thereby, eliminating the need for energy cooperation. • Further, most of the reduction in the time average cost is achieved by cooperation among only a few neighboring MGs. The incremental gain obtained by cooperation among large number of MGs is not significant. Next, we try to examine the following question: for a given number of cooperating MGs, what is the storage capacity needed per MG to eliminate the need for borrowing energy from the marco-grid? In order to do so, we consider a hypothetical scenario in which p i,j [t] = p i,j = β unit ∀i, j, t and q i [t] = q i = 3β units ∀i, t (both being independent of the distance between elements of the grid). Once again, we choose L i [t] = L i = 10 MW, ∀t, i and ν = 3 MW. We look for the combination of the storage and number of cooperating MGs that yields a normalized time average cost C N = β × 10 7 units as the bench mark (The choice of β×10 7 units comes from the fact that p i,j [t] = β units and 10 7 corresponds to the 10 MW demand per time slot). We plot the optimal value of N and E max required to make the normalized cost below 1 unit in Figure 7. In our numerical results, we choose β = 1. It can be seen that for a given number of cooperating MGs, there exists an optimal storage capacity requirement to eliminate the need for borrowing energy from the macro-grid. It is evident that as the number of cooperating MGs increases, the optimal storage capacity requirement reduces. A more practical case in which the algorithm is implemented on the renewable energy data provided by National Renewable Energy Laboratory (NREL) of the United States is presented in [26] and similar results have been obtained.

VI. CONCLUSIONS AND FUTURE WORK
In this work, we explored the benefits of energy storage and cooperation among interconnected MGs as a means to combat the uncertainty in harvesting renewable energy. We modeled the set-up as an optimization problem to minimize the cost of energy exchange among the grid for a given storage capacity at the MGs. First, we provided an analytical expression for the time average cost of energy exchange in the presence of storage by analyzing the steady state of the system for some special cases of interest. We then developed an algorithm based on Lyapunov optimization to solve this problem, and provided performance analysis of this algorithm. Our results show that in the presence of limited storage devices, the grid can benefit greatly by cooperating even among only a few distributed sources. However, in the presence of large storage, cooperation does not yield much benefit. Our solution can be useful for the power grid designer in terms of choosing the optimal combination of storage size and cooperation in order to meet a specific cost criterion. Since this work is a first step towards exploring the trade-offs between cooperation and storage, we ignored the transmission network constraints throughout. The analysis of how these constraints impact the trade-off between cooperation and storage is left for future investigation.
APPENDIX A : PROOF OF (39) First note that Squaring both sides of the equation, we obtain, Using this bound and rearranging (49), we obtain Summing it over i = 1, . . . , N and taking the conditional expectation on both sides given E[t], we obtain the bound in (39).
APPENDIX B : PROOF OF LEMMA 1 Let us first focus on statement 1. First, note that since the objective is to minimize (43), it can be easily inferred that Using the bound on the value of V from (45), in equation (51), Let us now turn to statement 2. Once again, from the objective function in (43), it is clear that where the last step follows since q i [t] ≤ q max . Therefore, APPENDIX C: PROOF OF THEOREM 2 We will first prove the bound on the battery size 0 ≤ E[t] ≤ E max . We use the analysis similar to [8] to obtain a bound on the battery size. We consider four cases. We next proceed to prove the result on time average performance of the algorithm. Consider the bound on the Lyapunov drift function of (42). It is clear that the control actions chosen according to the solution of (43) minimize the bound on the Lyapunov function over all possible control actions. Comparing it with the control action chosen according to any stationary and randomized policy (which we will denote with the superscript Π), we have, Rearranging, we obtain where the last step we have used the fact that G Π In particular, let us consider the stationary and randomized policy Π from Theorem 1, which achieves the cost g * N . Using this policy in (56), we obtain Taking the expectation on both the sides and summing from t = 0, . . . , T − 1 , normalizing by T and taking the limit, it can be shown that He has also been has been a student researcher at the Indian Institute of Science, Bangalore during 2007. He has served as a member of Technical Program Committee for IEEE PIMRC in 2014, IEEE VTC in 2014, and IEEE WCNC in 2014. His research interests broadly spans wireless communication and signal processing with emphasis on small cell networks (SCNs), cross-layer design wireless networks, MIMO systems, stochastic network optimization, energy harvesting and smart grid systems.
Tony Q.S. Quek (S'98-M'08-SM'12) received the B.E. and M.E. degrees in Electrical and Electronics Engineering from Tokyo Institute of Technology, Tokyo, Japan, respectively. At Massachusetts Institute of Technology (MIT), Cambridge, MA, he earned the Ph.D. in Electrical Engineering and Computer Science. Currently, he is an Assistant Professor with the Information Systems Technology and Design Pillar at Singapore University of Technology and Design (SUTD). He is also a Scientist with the Institute for Infocomm Research. His main research interests are the application of mathematical, optimization, and statistical theories to communication, networking, signal processing, and resource allocation problems. Specific current research topics include cooperative networks, heterogeneous networks, green communications, smart grid, wireless security, compressed sensing, big data processing, and cognitive radio.
Dr. Quek has been actively involved in organizing and chairing sessions, and has served as a member of the Technical Program Committee as well as symposium chairs in a number of international conferences. He is serving as the TPC co-chair for IEEE   Dr. Poor is a member of the National Academy of Engineering, the National Academy of Sciences, and Academia Europaea, and is a fellow of the American Academy of Arts and Sciences, the Royal Academy of Engineering