{'fraction_non_alphanumeric': 0.02402338456773394, 'fraction_numerical': 0.005517054473653645, 'mean_word_length': 4.596858071505959, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Revenue Maximization through Cell Switching and Spectrum Leasing in 5G HetNets

One of the ways of achieving improved capacity in mobile cellular networks is via network densification. Even though densification increases the capacity of the network, it also leads to increased energy consumption which can be curbed by dynamically switching off some base stations (BSs) during periods of low traffic. However, dynamic cell switching has the challenge of spectrum under-utilizationas the spectrum originally occupied by the BSs that are turned off remains dormant. This dormant spectrum can be leased by the primary network (PN) operators, who hold the license, to the secondary network (SN) operators who cannot afford to purchase the spectrum license. Thus enabling the PN to gain additional revenue from spectrum leasing as well as from electricity cost savings due to reduced energy consumption. Therefore, in this work, we propose a cell switching and spectrum leasing framework based on simulated annealing (SA) algorithm to maximize the revenue of the PN while respecting the quality-of-service constraints. The performance evaluation reveals that the proposed method is very close to optimal exhaustive search method with a significant reduction in the computation complexity.


I. INTRODUCTION
The demand for capacity improvement in order to achieve enhanced data transmission is a challenge that is constantly facing mobile network operators (MNOs). This is due to increase network operators (also known as primary network (PN) operators). This is because spectrum is normally auctioned by the telecommunication regulatory body in each country (e.g., Office of Communications (Ofcom) in the UK) at a very expensive rate. Spectrum leasing results in enhanced spectrum utilization and additional revenue to the PN operators, since it has been observed that the licensed spectrum is not always fully utilized most of the time [8]. The spectrum purchased by the SN from the PN can be used to provide data services which are delay tolerant (DT) such as meter readings, health information from wearables, etc., and do not require real-time data transmission. It can also be used to provide non-delay tolerant (NDT) services such as location and traffic update services, voice calls, etc., which require real-time data transmission for quick decision making. Therefore, the PN operator can gain revenue both from energy cost savings due to dynamic cell switching and from leasing the dormant spectrum of the BSs that are turned off to the SN.
Several approaches have been proposed in literature for implementing dynamic cell switching in mobile cellular networks [9]- [15]. These methods comprise analytical, heuristic and machine learning based approaches. Similar optimization techniques have also been proposed for spectrum leasing [16]- [22]. However, very few research works have considered both cell switching and spectrum leasing for maximizing the revenue of the PN [23] and [24], even though only a homogeneous network deployment scenario as well as a fixed electricity and spectrum pricing policy were considered, thereby making their work quite simplistic. Therefore, in this paper, we propose a cell switching and spectrum leasing framework for maximizing the revenue of the PN. The solution to this problem is non-trivial as it involves trying different options out of a large set of possibilities. The problem becomes very difficult to solve when the number of SBSs deployed in the network becomes very large. The optimal solution is the exhaustive search (ES) approach because it tries all the possible options before selecting the best one. However, due to the huge computational overhead involved in implementing ES, we develop a heuristic solution that can find a near optimal solution with much lesser complexity.
A cell switching and spectrum leasing scheme based on simulated annealing (SA) algorithm is developed to determine the optimal cell switching and spectrum leasing strategy that would result in maximum revenue for the PN by considering both the revenue obtained from energy savings as a result of switching off some SBSs and that obtained from leasing the spectrum to the SN.
The SA algorithm also ensures that the quality of service (QoS) of the PN is maintained by ensuring that the options which violate the QoS of the network are avoided when selecting the optimal solution. To the best of our knowledge, this is the first work to consider heterogeneous network (HetNet) scenario for the PN as the previous works [23] and [24] considered only homogeneous networks. In addition, both fixed and dynamic electricity and spectrum pricing policies are considered in this work, thus making our work novel and more realistic.
The remaining part of this paper is organized as follows: In Section II we review the related literature, while in Section III the system model is comprehensively presented. The proposed SA algorithm based framework for cell switching and spectrum leasing is discussed in Section IV, followed by the performance evaluation in Section V. Section VI concludes the work.

II. RELATED WORKS
Dynamic cell switching techniques are the most commonly employed methods for optimizing energy consumption in cellular networks because they are the cheapest to implement and require minimal changes to network architecture [4]. These techniques result in significant energy savings compared to other methods such as cell zooming, bandwidth adaptation, sectorization, etc [6], [7]. The authors in [9] proposed a reinforcement learning based cell switching approach to optimize the energy efficiency as well as the CO 2 emission in a HetNet. A cell switching and traffic offloading scheme for energy optimization in ultra-dense network using artificial neural network was proposed in [10]. The authors in [11] developed a scalable reinforcement learning based cell switching framework using state-action-reward-state-action (SARSA) algorithm with value function approximation to determine the optimal switching policy that would minimize the energy consumption in an ultra dense network while ensuring that the QoS of the network is maintained. The work in [12] proposed deep reinforcement learning model for minimizing the energy consumption in a RAN by dynamically switching off/on some of the BSs without compromising the QoS of the network.
The authors in [15] considered the problem of SBS power control and user association in HetNets and proposed an optimization scheme that enables SBSs to be switched off during periods of low traffic. A heuristic algorithm was developed to determine the switching pattern of redundant SBSs. The authors in [14] proposed a delay constrained multi-level SBS switching mechanism for energy optimization in HetNets. A distributed Q-learning algorithm was developed to determine the optimal sleep level of the SBSs while considering their traffic load, interference level and volume of data in the buffer of the SBSs. Even though dynamic cell switching results in significant energy savings, it also results in spectrum under-utilization as the spectrum that was originally allocated to the SBSs that are switched off remain dormant when they are inactive. These dormant spectrum can be exploited via spectrum leasing operations.
As regards spectrum leasing, three major reasons for spectrum leasing have been advanced in literature [25]: i) For monetary gains, ii) to maximize transmission rates, and iii) to reduce the energy consumption of primary users (PUs). In the first case, the PN leases some of its spectrum to the SN at a cost in order to generate additional revenue. In the second case, the PN shares some of its spectrum to the SN in exchange for assistance in data transmission, thereby enhancing the data rates of the PUs. In the third case, the secondary users (SUs) act as a relay to the PUs thereby reducing the transmission distance between the PUs and the BSs which leads to energy savings in the PUs. In this work, we are interested in the first case that is spectrum leasing for monetary gains because we want to maximize the revenue of the PN.
In this regard, various research works using techniques such as game theory, matching theory, and machine learning techniques, etc., have been proposed [16]- [22]. The authors in [16] proposed a traffic-adaptive spectrum leasing scheme whereby the SUs are able to negotiate the duration of channel leasing with the PUs in order to ensure their continual utilization of the leased channel for the complete transmission of the data in their buffer. To achieve this objective, the average utilities of both the PN and SN were first formulated, after which a spectrum leasing agreement that is beneficial to both parties was developed using Stackelberg game model. The work in [18] proposed a joint optimization scheme for spectrum leasing and spectrum allocation using both Stackelberg game and matching theory. The proposed approach is able to determine the best price for leasing the spectrum as well as the best PU-SU pair while enhancing the spectral efficiency of the PUs and SUs. In [17], the authors considered a spectrum leasing problem between MNOs and mobile virtual network operators (MVNOs) using matching theory in order to maximize the utilities of the both parties in terms of spectrum leasing cost and bandwidth allocation. Their goal is to find a suitable paring between the MNOs and MVNOs that would maximize the revenue of the MNOs as well as the bandwidth allocated to the MVNOs. The work in [22] considered the problem of spectrum leasing optimization for CRN transmission over TV white spaces. A neural network based solution was proposed to determine the optimal transmission policy that would result in minimal spectrum leasing cost while considering the QoS of the CRN. The works in [19] and [20] considered the problem of resource allocation and spectrum leasing in CRNs where the PUs lease part of their spectrum to the SUs in exchange for data transmission assistance from the SUs as well energy saving for the PUs. A resource optimization model for the CRN to minimize the power consumption of the PUs, while considering the uncertainty of the communication environment was proposed. In [21], an adaptive spectrum leasing with channel aggregation for CRN was considered where the amount of spectrum that the PUs can lease to SUs varies with the number of active transmissions as well as the amount of buffered data. A leasing algorithm was developed to adjust the amount of spectrum to be leased while satisfying the requirement of both PUs and SUs.
Joint cell switching and spectrum leasing has been considered in [23] and [24] to maximize the profit of both PN and SN as well as to minimize the energy consumption of PN. The authors in [23] considered a CRN comprising both PN and SN where the PN aims to reduce its energy consumption by turning off some BSs and transferring the users to the SN to maintain their QoS. In addition, the PN obtains revenue by leasing the free spectrum to the SN while the SN also gains revenue from the PN by charging a roaming price. A sub-optimal heuristic algorithm was developed to optimize the energy consumption of the PN by determining the set of BSs to switch off per time.
In this paper, we develop a cell switching and spectrum leasing framework for revenue maximization in a HetNet. Different from previous works [23] and [24] which considered only homogeneous network deployment scenario, a HetNet deployment scenario with different types of SBSs is considered in this work, which makes our work more realistic. In addition, we did not consider offloading the traffic load of the PN SBSs that are switched off to the SN as was the case in previous works, as this would lead to additional expenses on the part of the PN due to roaming charges. Rather, we consider vertical traffic offloading from the switched off SBSs to the MBS in order to maintain the QoS and maximize the profit of the PN. Furthermore, a fixed pricing policy was considered in the previous works while in this work, both fixed and dynamic electricity and spectrum pricing policies as well as DT and NDT spectrum demand scenarios are considered. This is because both pricing policies and spectrum leasing scenarios are a better representation of what is obtainable in real systems.

A. Contributions
In this paper, we propose a cell switching and spectrum leasing framework to maximize the revenue of the PN. The proposed algorithm is able to learn the optimal cell switching and spectrum leasing policy that would result in maximum revenue for the PN while ensuring that the QoS of the PN is maintained. The proposed framework is implemented locally at each MBS since they are responsible for controlling the SBSs under their coverage. The following are the contributions of this work: • We formulate the problem as a binary integer programming problem and develop a cell switching and spectrum leasing framework using SA algorithm to determine the optimal policy that maximizes the revenue of the PN while ensuring that the QoS is maintained.
• We consider a HetNet comprising four different types of SBSs, which makes the network scenario more complex and realistic compared to the previous works that considered only homogeneous scenario.
• We consider two electricity and spectrum pricing policies: 1) fixed, and 2) dynamic policy, in order to study the effects of constant and varying electricity and spectrum prices on the maximum revenue of the PN, as both could be the cases faced in real systems. For the dynamic pricing policy, both DT and NDT spectrum demand scenarios are also investigated.
• In addition to the ES algorithm, two benchmark solutions are also developed for comparison with the proposed framework.
• A complexity comparison of the proposed method with that of the ES is carried out to highlight the advantage of the proposed framework.
• Finally, in order to capture the realistic behaviour of the network, the performance of the proposed framework is evaluated using real data comprising call detail records (CDR) of Milan city via extensive simulations and the result obtained is compared with benchmarks.

A. Network Model
Two types of networks are considered: First, the PN is a HetNet with control and data separated architecture (CDSA) [26] consisting of MBSs and SBSs. The MBSs serve as control BSs, provide constant coverage and low data rate transmission. The SBSs are deployed within the coverage of the MBSs and serve as data BSs to provide high data rate transmissions in hot spot zones.
The communication between MBS and SBSs are carried out in the control channels, which are separated from the data channels. Four types of SBSs-RRH, micro, pico and femto-are considered. Second, a SN is also assumed to operate in the same coverage area and the HetNet allows this SN to lease some unused spectrum whenever PN SBSs are put to sleep.
In this context, it is assumed that each SBS continuously monitors the activity of the SN BSs

B. Power Consumption of HetNet
We adopt the BS power consumption model in [27], [28] for estimating the power consumption of the BSs in the HetNet. The total power consumption of the HetNet comprises sum of the power consumption of the MBS and that of all the SBSs under its coverage. The instantaneous power consumption of a BS, P BS , at time t can be expressed as: where P o is the constant circuit power consumption, τ t is the instantaneous traffic load at time t, ζ is the load dependent power consumption component and P tx is the transmission power of the BS. It should be noted that the value of P o , ζ, and P tx is different for each type of BS (i.e., MBS, RRH, micro, pico, and femto).
As such, the total power consumption of the HetNet, P HN , can be expressed as: where P BS i,j denotes the power consumption of the j th BS in the i th macro cell (MC), and P BS i,1 represents power consumption of the MBS in the i th MC.

C. Pricing Policy
Two kinds of pricing policies are considered for both the electricity and spectrum: 1) Fixed Pricing Policy: The unit cost of electricity as well as that of the spectrum remains constant throughout the day, irrespective of the fluctuations in energy or spectrum demand.
2) Dynamic Pricing Policy: The electricity and spectrum price varies according to the amount of electricity and spectrum demanded at different times of the day. The dynamic pricing model for electricity was adapted from [29], where the instantaneous electricity prices were obtained by multiplying the fixed price by a variable factor to indicate changes in the prices at different times of the day. For the dynamic spectrum price, we assumed that the spectrum prices follow the traffic demand pattern of the PN. However, these values are scaled with the fixed spectrum price such that: C RB,t = m . C RB,F , where m is a time variable function that changes with the instantaneous traffic load, τ t , i.e., m = f (τ t ), C RB,t and C RB,F are the dynamic and fixed spectrum price (i.e., cost per resource block (RB)). According to 3GPP [30], a RB is equivalent to 12 successive subcarriers, thus taking one subcarrier to be 15kHz, we consider one RB to be 180kHz. The dynamic electricity and spectrum pricing policies are presented in Fig. 2. Under the dynamic spectrum pricing policy, two types of spectrum demand scenarios are considered: • Non-Delay Tolerant (NDT): This scenario deals with applications such as location updates, voice calls, etc., that require real-time data transmission and cannot tolerate delay because of the sensitivity of the information and its requirement for quick decision making. For such applications, the SN has to demand for the spectrum as soon as the need for data transmission arises, irrespective of the spectrum price.
• Delay Tolerant (DT): There are some other applications such as meter readings, feedback from wearables, etc., whose information may not be needed for real-time decision making and hence can tolerate some level of delay in data transmission. In these scenarios, the SN can decide to accumulate their service demands until the periods of the day where the spectrum price is cheapest, before transmission to save cost. In this work, the cheapest period is statistically decided only once and then the traffic is adjusted accordingly.

IV. PROBLEM FORMULATION
A specific time period T (in mins) is considered and it is then divided into equal time slots (in mins) with a duration of d (in mins). Then, we define an index vector ν that stores the time slots in an order, such that ν = [1, 2, ..., L], where L is the number of time slots and is given by L = T /d. The BSs of the PN are represented by B P i,j while that of the SN by B S i,j . We view the problem from the PN perspective and formulate the revenue maximization problem by considering the revenue obtained from the combination of cell switching and spectrum leasing.
Since the PN obtains its power supply from the grid, it can decide to turn off some SBSs during periods of low traffic to reduce their energy cost (i.e., gain some revenue from energy saving) and also lease the dormant spectrum to the SN in order to gain additional revenue.
1) Revenue from cell switching: The power consumption when no cell switching is implemented (i.e., when all the BSs are on), P on , can be expressed as: The power consumption when cell switching is implemented (i.e., when some BSs are turned off), P cs , is given by: where P s BS i,j is the power consumption of the BS when it is switched off (i.e., sleep mode power consumption of the BSs) Γ denotes the on/off status of (i, j) th BS at time t i,e., Since the MBS is always on, Γ i,1 = 1, ∀t.
Then, the power saving due to cell switching, P sv can be expressed as: Therefore, the revenue due to energy saving, R E , can be expressed as: where C e,t is the cost of electricity at any given time, t.
2) Revenue from spectrum leasing: The revenue due to spectrum leasing, (R l ), can be expressed as: where Ψ S i,j denotes the amount of spectrum (number of RBs) supplied by B P i,j , Ψ D i,j denotes the amount of spectrum demanded by Ψ S i,j from B P i,j and C RB,t is the unit cost of spectrum (i.e., price per RB).
B P i,j and B S i,j are assumed to have the same capacity, which implies that Ψ D i,j ≤ Ψ S i,j . Therefore, (8) can be simplified as: 3) Total Revenue: The total revenue of the PN, R T can be expressed as: and substituting (7) and (9) in (10), we obtain: Replacing (7) with (3) and (4) and simplifying (11), we get the closed form expression for the total revenue and is expressed as: 4) Optimization objective: The revenue maximization objective function is the joint optimization of the revenue due to cell switching and spectrum leasing and can be expressed as: The constraints of (13) are explained in the following. The traffic demand, Υ i , when all the BSs in the MC i are on (i.e., before traffic offloading) is computed as, To ensure QoS, the traffic of any SBS that is switched off will be transferred to MBS and therefore the actual traffic of MBS during the offloading process, denoted byτ i,1 is equal to, The traffic demand of the MC after traffic offloading,T i can be expressed as, Therefore, (17) must be equal to (19) to satisfy the constraint in (14). We are assuming that MBS will be able to handle the traffic of all the switched off SBSs. On the other hand, if there is a maximum limit on the amount of traffic that the MBS can handle then we also have to introduce another constraint. For example, let τ m i,1 denote the maximum traffic that MBS can serve in any time slot t. Then, we have the additional constraint in (15).
The solution to the problem in (13) is non-trivial as it involves deciding the optimal set of SBSs to turn off out of all the possible options, and then leasing their spectrum to the SN BSs in order to maximize the revenue of the PN. The optimal solution can be obtained from ES algorithm, however, the number of search spaces increases exponentially with the number of SBSs in ES, thereby resulting in huge computational overhead. Hence, we resort to a less complex heuristic which considers a lesser search space and can give a sub-optimal solution with reduced computational complexity (lesser search spaces compared to ES).

V. PROPOSED FRAMEWORK
The aim of this paper is to determine the optimal cell switching and spectrum leasing strategy that would maximize the revenue of the PN without compromising the QoS of the network.
Although ES always finds the optimal policy, it is computationally complex to implement because it has to sequentially search through all the possible cell switching and spectrum leasing combinations before deciding the optimal solution. As a result, in this work, we employ the SA algorithm which has lesser complexity since it involves lesser search spaces in finding the optimal solution. However, this algorithm is not always guaranteed to produce the optimal result as is the case with ES. In this regard, albeit being sub-optimal, through extensive simulations, we prove that the developed SA algorithm based solution produces almost the same results as the ES algorithm-especially when the network sizes are reasonable-with much less computational complexity, providing a promising trade-off between the performance and complexity.

A. Simulated Annealing (SA) Algorithm
The SA algorithm is a probability-based heuristic that deals with the annealing process in solid materials. The working principle of SA algorithm involves mimicking the process during which a heated solid material cools down. It is used in the optimization of difficult problems such as machine scheduling, inventory control and vehicle routing problems in the literature [31]. One of the most important features of the SA algorithm is that it ensures the results that degrade the value of the objective function are included in the solution process under certain conditions in order not to be stuck at a local optimum. In other words, an improved objective function value (better than the current best solution) is always accepted, whereas non-improved solutions are accepted based on a probability value [32]. This mechanism is elaborated in the next section.
In basic SA algorithm, the criteria used to accept a worse objective function are the random numbers between 0 and 1, the improvement in the objective function, and the current temperature values. In the operation steps of the algorithm, as the temperature of the system decreases, the possibility of accepting worse results decreases because, as can be seen in (21), the decrease in T value also decreases the selection probability of worse solutions. Thus, while the diversification feature is high at the beginning of the algorithm, intensification feature becomes prominent towards the last iterations. In other words, the algorithm performs a wider search by taking into account the worse solutions in the initial stage. However, it focuses on specific regions in the search space in the final stages. The probability of increase at temperature, T (in Kelvin), of δE amplitude in energy is presented in (20), where K is Boltzman constant: Therefore, the starting temperature of the system is a hyper-parameter of the algorithm and has a significant impact on the overall performance, such that it must be high enough to allow any feasible solution to be accepted. However, if it is set too high, the search process will be random until the temperature decreases to a certain level. As such, a certain T value is determined as a stopping criterion in order not to prolong the search process excessively.

B. SA algorithm for cell switching and spectrum leasing
To control the switching off/on of SBSs, it is necessary to determine the parameters of the algorithm in the first place. Then the objective function value of randomly generated initial solution s is calculated with (13). In this way, the revenues are obtained according to the energy saved from turning off some SBSs in the PN (7) and spectrum leased to the SN (9). During the search process, the algorithm attempts to transform the current solution s into one of its randomly selected new solution s . However, in the developed algorithm, instead of randomly selecting a neighborhood structure, each neighborhood is applied in an order as in sequential variable neighborhood search (VNS) algorithm [33]. We also expanded the search area in each iteration due to the small number of neighborhood types.
Note that only feasible solutions which guarantee (14) and (15) After the local search process (after k iteration), the temperature is decreased according to the formula T = T − α, where α is the temperature reduction parameter. The pseudo code for the developed SA based cell switching and spectrum leasing framework is presented in Algorithm 1.
The step-by-step implementation procedure of the proposed SA based cell switching and spectrum leasing framework is discussed in the following: 2) Solution Representation: The proposed SA algorithm has a representation scheme specially designed for the cell switching and spectrum leasing problem. It has a binary representation depending on whether the SBSs are off or on.

3) Initial Solution:
In the SA algorithm, the initial solution, which is the first feasible solution at the beginning of the iterations in the SA algorithm, is generated randomly or with certain methodical approaches such as nearest neighbor heuristics [34]. Simple heuristic methods are considered to decrease the solution time and increase the quality of the solution in some NP-hard problems. However, in this work, the initial solution is generated randomly, and not with any constructive heuristic method because the optimized initial solution can be trapped in a particular local optimum within the search space.   and T F are the initial and final temperatures, respectively. The initial temperature must be high enough to allow the acceptance of any feasible solution. If the initial temperature is too high, the probability of generating random solutions among feasible solutions at the beginning of the algorithm is higher. On the other hand, if the initial temperature is too low, the probability of getting stuck at the local optimum of the algorithm increases. The final temperature of the algorithm is set to avoid spending too much time in reaching the optimum. k is defined as the number of iterations of the local search procedure at each temperature, while α is the temperature reduction parameter. It refers to the amount by which the temperature will be decayed at the end of each iteration. K is Boltzmann constant and is used in calculating the probability of accepting or rejecting worse solutions. If the new objective function value is worse than current best solution, it will generate u, which is a random variable between 0 and 1. Then, the obtained solution will be accepted if the criterion represented in (21) is satisfied. Except for this situation, an improved objective function value is always accepted. For more detailed information about the working mechanism of the SA algorithm, refer to [36]. We considered different SA algorithm design parameters that are frequently used in the literature [31], [35] and chose the ones that lead to the best results during the preliminary tests. The best SA parameter combination is T = 1, α = 0.01, T F = 0.01, k = 10N , where N indicates the total number of SBSs in the PN. potentially shifting from an optimal to a sub-optimal solution, in an attempt to reduce the complexity, escape local minima, and converge to a value closer to the global optimum.  Table I.

A. Data Set and Pre-processing
To compute the total revenue of the HetNet using (12), the traffic demand of each BS in the PN (τ ) and SN (Ψ) is required. We leveraged the call detail record (CDR) data set of the city of Milan, Italy that was made available by Telecom Italia [37]. In the data set, Milan city was divided into 10,000 square grids with each having an area of 235×235 square meters. In addition, the call, short-message and Internet activities that were carried out in each grid was recorded every 10 minutes over a period of two months (November-December 2013). Although the activity levels contained in the data set are without unit and no additional information was provided regarding how the data set was processed, we decided to interpret the CDR of each grid as the traffic loads as they signify the amount of interaction between the users and the mobile network within the grid in each time slot. However, during the data processing stage of this work, we considered only the Internet activity level as the traffic load for the PN since it was the most significant part of the data set and also considering the fact that 5G networks would be mainly Internet based. The Internet activity level of two grids were selected at random to represent the traffic load of the MBS while that of one grid was chosen for each SBS. Then, the traffic loads were normalized separately according to the capacity of each type of SBS. We assume that the traffic demand of each BS in the SN is a fraction of the traffic demand of the SBSs in the PN such that Ψ = βτ where β is a variable between 0 and 1 (β was chosen to be 0.7 in this work). The traffic demand of the SN is shifted so that its maximum traffic demand coincides with the period of the day when the spectrum leasing price is minimum in order to depict the DT case while for the NDT case, the traffic demand remains intact.

B. Benchmarks
We compare the performance of the proposed method with three benchmark methods namely: ES, A-type, and D-type algorithms, which are briefly described in the following paragraphs.

1) Exhaustive search (ES):
This method sequentially considers all the possible cell switching and spectrum leasing combinations in order to determine the optimal off/on switching policy that would result in maximum revenue to the PN while ensuring that the QoS of the network is maintained. Therefore, this methods is guaranteed to always find the optimal policy without violating the QoS of the network. However, the computational complexity involved in sequentially searching through all the possible combinations makes it unsuitable for online implementation.
The goal of any other algorithm is to closely approximate the policy obtained from this approach, hence, it is suitable as a benchmark for this problem.
2) Sorting-based Algorithms: Two additional benchmark algorithms are developed using the sorting approach which we have named A-type and and D-type heuristic respectively. In the D-type heuristic, we first evaluate a utility function, N , which is the difference between the traffic demand of the SN BSs and that of the PN BSs, i.e., N = Ψ − τ . This utility function N is important because we are not only interested in switching off the SBSs with low traffic demand, but also those whose associated SN BS has high spectrum demand. It is necessary to satisfy both conditions if the revenue of the PN is maximized because both of them affects the total revenue (10) that can be generated by the PN. In addition, since the total revenue of the PN is dependent on the amount of revenue that can be obtained from energy savings and spectrum leasing, thus, higher values of N would result in greater revenue generation due to higher contributions from both components. On the other hand, lower values of N might result in lesser revenue generation due to smaller contribution either from the energy savings or spectrum leasing. After evaluating N , the SBSs are arranged in descending order according to the value of N . Then, the traffic load of the SBSs are sequential offloaded to the MBS until the capacity of the MBS is reached. The procedure for implementing A-type heuristic is similar to that of D-type except that in A-type, the SBSs are sorted in ascending order according to N .

C. Performance Metrics
The metrics that would be used in evaluating the performance of the proposed and benchmark methods are briefly discussed in this section.

1) Total Revenue:
The goal of this work is to determine the maximum revenue that can be obtained by the PN over a given period of time, T . As described in Section IV, this is obtained by combining the revenue due to energy saving from cell switching and the revenue obtained from leasing the spectrum to the SN. The total revenue of the network can be obtained from (13).
2) Average Network Throughput: The effect of the proposed framework on the QoS of the network is evaluated using the network throughput metric. Here, we consider the network throughput to be the traffic demand that can be served by all the remaining BSs (both MBS and active SBSs) after cell switching and spectrum leasing operation has been executed. To estimate this throughput, we assume that the activity level contained in the employed data set are throughput demands (in Mbps) so that τ can be seen as the normalized throughput of each BS (i.e., MBS and SBSs). Therefore, the average network throughput Q Net (t) can be obtained by aggregating the throughput demands of all active SBSs and the MBS [11], such that: where Q(t) is the average received throughput from each BS and can be expressed as: where r m is the average throughput that is allocated to each user (assuming equal resource allocation) and U m is the total number of users served by each BS at a given time t.

D. Results and Discussions
Fig. 5a shows the hourly total revenue obtained by the PN following the fixed electricity and spectrum pricing policy with NDT spectrum demand using the proposed and benchmark methods. In addition, the traffic load of the PN MBS, τ i,1 , is also presented. The first thing we observe from Fig. 5a is that the revenue obtained from all methods follows a trend that is opposite of that of the traffic demanded of the PN. This is so because during the periods of the day where the PN traffic is low, more SBSs can be switched off which translates to more revenue generation from energy savings and spectrum leasing. The opposite is the case when the traffic of the PN is high. Second, the SA algorithm follows ES almost exactly, since it is able to employ its mechanisms such as feasibility check and neighbourhood structures to determine the optimal cell switching and spectrum leasing pattern, but with much lesser complexity.
Third, both the A-Type and D-type heuristic solutions never outperform ES and SA algorithms because they also respect the constraint of not exceeding the MBS capacity. Even though they both respect the MBS capacity in order to maintain the QoS of the network, they utility, N , used in determining which BSs to switch off only considers the difference in traffic demand between the PN and SN, but is not able to distinguish between the various types of BSs present.
In this work, the PN and SN BSs have different capacities and power consumption, as a result, switching off a SBS with higher capacity and power consumption and leasing its spectrum to the SN would result in higher revenue than switching off one with a lower capacity. This limitation accounts for the lesser revenue obtained from both the A-type and D-type heuristics. Another interesting point to discuss about the observations in Fig. 5a is that the D-type heuristic mostly outperforms the A-type heuristic because it switches off the SBS with highest utility, N , values first and this helps in the generation of more revenue compared to A-type which does the opposite. However, this performance difference is mostly observable during the times of low traffic as there are more options and the higher utility is able to find a better solution.
For the time when the network traffic is high, they start performing alike, since the number of cell switching and spectrum leasing options becomes very low. Overall, the performance difference between the D-type and A-type solutions is not large, as the former outperforms the later with a minimum of 1% and a maximum of 29%. The last observation worth discussing is that the SA solution mainly outperforms both A-type and D-type solutions (by about 90% and 65% respectively) during periods of high traffic. The reason for this is that the number of cell switching and spectrum leasing options becomes very few during this period, thereby making it very difficult for them to find the best solution while the SA solution is carefully designed to be able to perform excellently well even in such periods. Fig. 5b presents the total revenue obtained every hour by the PN when the dynamic pricing policy with NDT spectrum demand is considered using the proposed and benchmark methods.
In the dynamic pricing policy, the prices of both electricity and spectrum vary at different times of the day depending on the amount of spectrum or electricity demanded. Similar to what was observed in Fig. 5a, the pattern of the total revenue over the whole day is the inverse of the traffic profile of the PN. Moreover, the revenue is generally scaled down compared to Fig. 5a, and this is more noticeable during periods of low traffic. This is because a dynamic pricing policy is used, where the PN sometimes needs to lease the spectrum for less and at those times it also earns less from energy savings because the prices are lower. The D-type heuristic also slightly outperforms the A-type heuristics with almost the same percentage (1% to 29%) as in Fig. 5a, due to the fact that higher utility values are considered first during cell switching which helps in greater revenue generation in the former compared to the later. The aforementioned confirms our previous argument on why the performance of the two benchmark algorithms are similar. The proposed SA algorithm also greatly outperforms the A-type and D-type algorithms with a similar percentage (90% and 65% respectively) as in Fig. 5a mostly during the period of high traffic in the PN because there are lesser cell switching and spectrum leasing options which make it difficult for the benchmark solutions to make the optimum decisions. Fig. 5c presents the total revenue obtained by the PN when dynamic pricing policy with DT spectrum demand is considered. In this case, the SN decides to delay its data transmission to periods when the spectrum price is low (which also coincides with period of low traffic demand in the PN) so that they can access more spectrum at a cheaper rate. It can be observed that there is an overall increase in the total revenue obtained by the PN in Fig. 5c, compared to Fig. 5a and Fig. 5b: the total revenue obtained from the proposed SA framework is about 19% and 16% higher than that obtained in the Fig. 5a and Fig. 5b, also it is evidenced by the peak value of the revenue of Fig. 5c being about £183 and £123 higher than that in Fig. 5a and Fig. 5b respectively. This is because in dynamic pricing policy with DT spectrum demand, the SN can lease more spectrum as the periods of low traffic in the PN matches the period of high spectrum demand by the SN although the prices are lower. This statement is validated by comparing the traffic demand of the PN in Fig. 5a with the DT spectrum demand in Fig. 5c; i.e., periods of lowest traffic demand in the PN (e.g., in the first quarter of the day where the traffic load is 14%) coincides with periods of highest spectrum demand from the SN (about 28%) so that even though the spectrum prices are lower at these times as seen in Fig. 2, the large amount of spectrum demanded by the SN causes the total revenue in this scenario to be highest.
The performance difference between the D-type and A-type heuristics is more significant in this scenario compared to the fixed and dynamic pricing policy with NDT scenarios in Fig. 5a and Fig. 5b with values ranging from 5.3% to 86%. The reason for the wider performance gap is that the NDT spectrum demand is responsible for preventing the D-type heuristic from significantly outperforming A-type heuristic. This phenomenon originates from the fact that in the fixed and dynamic pricing policy with NDT spectrum demand, the trend of the SN traffic demand follows the PN traffic demand, hence the margin in the values of N is smaller in the both cases compared to the dynamic pricing policy with DT spectrum demand, thus accounting for the lesser total revenue results of A-type and D-type heuristics in the previous scenarios.
On the other hand, for the dynamic pricing policy with DT spectrum demand, since the traffic demand of the SN is the inverse of the traffic load of the PN, the difference in the values of N at different time slots is higher and since D-type gives preference to SBSs with higher N during cell switching and spectrum leasing, more revenue is generated by the D-type compared to Atype, hence the reason for the wider margin in the revenue generated in the former compared to the later. In addition, the SA algorithm greatly outperforms the A-type and D-type benchmarks in terms of revenue generation by 124% and 95% respectively. These values are 34% and 31% higher than its performance against the two benchmarks in both the fixed and dynamic pricing with NDT spectrum demand in Fig. 5a and Fig. 5b respectively. The reason is that the SA is able to take advantage of the more available options to switch off SBSs during period of low traffic which coincides with high spectrum demand by the SN in order to generate much higher revenue than in the two previous scenarios.
It can also be observed that the results of D-type and A-type methods are almost the same in a few instances with the A-type even slightly surpassing that of the D-type at some points.
For the NDT cases (with both fixed and dynamic pricing policies) this occurs when both the data traffic of the PN and the spectrum demand of the SN are high. This is due to the fact that the difference in the values of the utility in this period is very small, thus, there is very little revenue from spectrum leasing as the SN is not able to access spectrum due to lack of dormant spectrum from the PN. Also, very little revenue can be obtained from energy saving since only very few SBSs can be turned off due to very high traffic load in the PN. For the DT case with dynamic pricing policy, the similarity in the results of both the D-type and A-type heuristics occur when the traffic demand of the PN is high and the spectrum demand of the SN is low.
At these periods, both benchmarks begin to function alike because even though there is a large difference in the value of the utility function, there is very little opportunity to switch off the SBSs due to high traffic in the PN. Hence, there is an insignificant difference in the performance of both benchmarks as relatively less revenue can be obtained during this period.    6a shows the total amount spent by the SN for spectrum purchase as well as the total quantity of spectrum obtained for a 24 hours period using the proposed SA-based framework and ES while Fig. 6b shows the unit cost of the spectrum (i.e., price per RB) for both DT and NDT spectrum demand using both algorithms. From Fig. 6a we can see that the total amount expended by the SN on spectrum purchase as well as the quantity of spectrum purchased are significantly higher in DT than in the NDT scenario with a percentage difference of 19% and 21% respectively. The rationale behind this is that most of the periods when the electricity and spectrum prices are low are also the periods when the traffic loads of the MBS and SBSs are low. As such, more SBSs can be turned off in order to ensure that more spectrum is available for SN to purchase during these periods. Although more spectrum is available to the SN for both DT and NDT spectrum leasing scenarios with the dynamic pricing polices during periods of low traffic load in the PN, the difference in the volume of spectrum demanded in both cases is what accounts for the difference in the amount expended on spectrum purchase in Fig. 6a.
In the DT case, the data to be transmitted is delayed until when the spectrum and electricity prices are low, which means that the SN is able to take advantage of more spectrum available in order to offer more data services to it users. However, for the NDT case, even though more spectrum is available during periods of low prices, the spectrum demanded by the SN during this period is also low, so lesser revenue is generated and fewer data services can be offered in this scenario. For example, in the first quarter of the day where the traffic load of the PN is the lowest (about 14%), the revenue generated by the dynamic pricing policy with DT spectrum demand is 47% higher than that obtained from the dynamic spectrum demand with NDT spectrum demand because more spectrum is available for leasing as well as a corresponding high spectrum demand from the SN. However, the availability of more spectrum does not correspond with high spectrum demand in the NDT case thereby leading to a lesser revenue generation. Fig. 6a also reveals that the total expenditure and quantity of spectrum purchased using SA algorithm is almost the same as that of ES algorithm which validates the excellent performance of the SA algorithm earlier discussed under Fig. 5a, Fig. 5b and Fig. 5c.
The purchase of more spectrum by the SN in the DT case compared to the NDT case means that the SN incurs more expenses during DT data transmission compared to NDT data transmission.
Therefore, the DT case is more beneficial to the PN because it results in more total revenue. It is also beneficial to the SN because it pays less for a unit of spectrum even though its total expenditures increases. Hence, where possible (for suitable applications), we can conclude that the shift in the SN traffic demand would be recommended. However, the kind of shift in the data transmission time of the SN does not have to be implemented in exactly the same way as in this work, instead, depending on the type of application, the latency requirements are evaluated and the appropriate shifts in the traffic is implemented accordingly, making the DT spectrum demand quite flexible and dynamic. Although this traffic shift may not always coincide with the cheapest time but to a cheaper time. In summary, DT spectrum demand will make the business of both PNs and SNs more sustainable because it is more profitable for both parties.
A major constraint in this work is to ensure that the QoS of the network is maintained by ensuring that traffic served by the network remains constant even when some SBSs are switched off. The PN is supposed to respect the capacity constraints of the MBS before switching off any SBS. From the simulations, we observed that both the proposed and benchmark solutions are able to maintain the QoS of the PN. The SA algorithm uses the feasibility check in Algorithm 2 to ensure that only solutions that do not exceed the capacity of the MBS are considered. The ES algorithm follows similar procedure by guaranteeing that solutions that exceeds the MBS capacity are excluded when selecting the optimal cell switching and spectrum leasing strategy.
Both the A-type and D-type algorithms are implemented in such a way that the traffic load of the SBSs are offloaded sequentially (in ascending and descending order respectively) and once the offloading capacity of the MBS is attained, no further SBS is turned off. By so doing, they both guarantee that the throughput of the network is maintained. It is also worthy of note that irrespective of the pricing model used for electricity and spectrum (fixed or dynamic) and the type of spectrum demanded by the SN (DT or NDT), the average throughput of the PN remains the same. This is because both the proposed and benchmark algorithms take the traffic-QoS constraint (14) into consideration thereby ensuring that the QoS of the network is not violated.

VII. CONCLUSION
In this paper, we considered the problem of revenue maximization through cell switching and spectrum leasing in order to maximize the revenue of the PN, which comprises a HetNet with different types of SBSs while the SN comprises SN BSs. An SA algorithm based solution was proposed to determine the optimal cell switching and spectrum leasing strategy that would result in maximum revenue for the PN while ensuring that the QoS of the network is maintained.
We considered fixed and dynamic pricing policy for both electricity and spectrum. Under the dynamic pricing policy, both DT and NDT spectrum demand scenarios were considered in order to determine the effect of these policies on the revenue of the PN as well as the expenditure and amount of service demands that can be met by the SN.
The simulation results show that the PN is able to obtain more revenue using the dynamic pricing policy with DT spectrum demand. Moreover, in the DT spectrum demand scenario, the SN is able to lease more spectrum when the spectrum prices are low, which enables it to serve more data services at a reduced average unit price. Thus, making this scenario more profitable to the SN compared to the fixed or dynamic pricing policy with NDT spectrum demand scenarios.
Overall, the performance of the proposed method is almost the same as that of the ES algorithm with lesser time complexity. In future, we intend to investigate a scenario whereby more than one SN BSs would be competing to lease the spectrum of each PN BSs and the PN would have to decide which of the SN BSs to lease the spectrum as well as the possibility of leasing the spectrum to more than one SN-BSs at the same time.