{'fraction_non_alphanumeric': 0.04816964703679431, 'fraction_numerical': 0.011305121243329276, 'mean_word_length': 4.047849716446125, 'pattern_counts': {'":': 0, '<': 17, '<?xml version=': 0, '>': 13, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 21, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Optimal Cyber-Insurance Contract Design for Dynamic Risk Management and Mitigation

With the recent growing number of cyberattacks and the constant lack of effective defense methods, cyber risks become ubiquitous in enterprise networks, manufacturing plants, and government computer systems. Cyber-insurance provides a valuable approach to transfer the cyber risks to insurance companies and further improve the security status of the insured. The designation of effective cyber-insurance contracts requires the considerations from both the insurance market and the dynamic properties of the cyber risks. To capture the interactions between the users and the insurers, we present a dynamic moral-hazard type of principal-agent model incorporated with Markov decision processes, which are used to capture the dynamics and correlations of the cyber risks as well as the user's decisions on the protections. We study and fully analyze a case with a two-state two-action user under linear coverage insurance, and we further show the risk compensation, Peltzman effect, linear insurance contract principle, and zero-operating profit principle in this case. Numerical experiments are provided to verify our conclusions and further extend to cases of a four-state three-action user under linear coverage insurance and threshold coverage insurance.


I. INTRODUCTION
Cyber risks created by malicious attackers such as ransomware [1], data breaches [2], and denialof-service [3], have become severe threats to the security of important devices and private data in Internet of things (IoT) and cyber-physical systems (CPS) [4]. For example, the CryptoLocker R. Zhang  ransomware attack has caused an estimated loss of $3 million [5]. The 2016 Dyn cyberattack has resulted in the disruption of major Internet platforms and services to large swathes of users in Europe and North America [6].
Although various defense methods such as firewalls [7], intrusion detection systems [8], and moving-target defenses [9], have been deployed to detect the intrusion attempts and protect the networked devices, they cannot eliminate the cyber risks due to the complexities of cyberenvironments [10]. Moreover, cyber threats are becoming stealthier, more strategic and purposeful as exemplified by the advanced persistent threats such as Stuxnet attacks on Iranian nuclear power plant in 2009 and the Ukrainian power plant attack in 2015 [11], [12].
Recently emerged cyber-insurance provides an economically viable solution to further mitigate the cyber risks and improve network resiliency [13]- [17]. The insured network users could quickly recover from severe cyber-incidents since part of the losses have been covered by the insurers. However, like the classic insurance, the insurers may suffer from offering coverage to reckless users due to the information asymmetry that the insurers cannot directly observe the users' protections [18]- [20].
Moreover, as suggested by the theory of risk compensation in traditional insurance scenarios [21], the users may become less careful against cyberattacks knowing that insurers will cover their losses, for example, users may click more phishing emails, ignore the warnings of upgrading firewalls or systems, and reduce the frequency of scanning viruses or worms. As a result, the users may encounter more severe cyber-incidents and the insurers may bear extra cyber risks.
Thus, it is imperative to study cyber-insurance contracts and its impacts on the users' cyber-risk statuses. However, classic risk analysis and insurance frameworks cannot be directly applied to cyber risks and cyber-insurance as cyber risks are dynamically evolving and strongly correlated [22]- [25]. For example, an adversary can first launch a node capture attack to compromise the system [26], [27], and then gain the administration to the devices [28], steal private information [29], or inject Ransomware worms or viruses [30].
In this paper, we capture the correlations and dynamics of the cyber risks as well as the users' decisions on the protections with the Markov decision processes (MDP) [31], [32]. Different states of the MDP are used to capture the different cyber risks from various sources, such as service failures, attackers, or network connections. The transitions of states capture the connections of different cyber risks, and they are affected by the user's actions of protections at different times.
To further mitigate the cyber risks, the user has a choice of purchasing cyber-insurance. After paying a premium, the user could receive financial coverages from the insurer to reimburse his losses caused by various cyber risks as shown in Fig. 1. The objective of the user is to find an optimal deployment of protections and cyber-insurance that minimizes his cyber-losses. Fig. 1. Cyber-insurance example. The blue, red, and green icons represent user, attacker, and insurer, respectively. A user pays a premium to an insurer to purchase the cyber-insurance. Then, the user could receive financial coverages from the insurer to cover part of his losses caused by cyberattacks.
A rational user selects a cyber-insurance from which he could benefit more, i.e., contracts with a low premium and a high coverage. However, an insurer tends to offer an insurance contract that has a high premium and a low coverage, as the insurer aims to maximize his operating profit. Moreover, similar to the traditional insurance scenarios, the insurer is not aware of the local protections of the user, and an inappropriate insurance contract could largely damage the insurer's profit.
We address such conflicting interests and the information asymmetry between the user and the insurer with a moral hazard type of principal-agent problem [19], [20], [33], [34]. The analysis, as well as the solution of the problem, is important to study the impacts of cyber-insurance to the user and design effective insurance contracts. The major contributions of this work are summarized as follows: • We integrate Markov Decision Processes (MDP) into a moral hazard type of principal-agent model to investigate the impacts of cyber-insurance on the user's cyber risks and design effective cyber-insurance contracts for the insurer.
• We fully characterize a case between a two-state two-action user and a linear coverage insurer.
The results of this case indicate that the optimal insurance contracts follow linear insurance contract principle and zero-operating profit principle. The analysis also demonstrates the existence of risk compensation and Peltzman effect in cyber-insurance.
• We develop computational tools to solve problems involving multiple cyber-risk states, various protection choices, and complex insurance contracts. Our numerical experiments illustrate risk compensation, Peltzman effect, and zero-operating profit principle in cases of a four-state three-action user under linear coverage insurance and threshold coverage insurance.

A. Organization of the Paper
The rest of this paper is organized as follows. Section II presents the related works. Section III and Section IV discuss the user's problem and the insurer's problem, respectively. Section V presents a case study of a linear coverage insurance contract on a two-state two-action user.
Section VI and Section VII present numerical results and concluding remarks, respectively.
Appendices A, B, and C provide the proofs of the Proposition 2, and Theorem 1, and Proposition 4, respectively. We provide a summary of notations in the following table for convenience.

II. RELATED WORKS
Recently, with fast-growing types and amounts of the networked devices and shortages of effective and state-of-art defense methods, cyber-insurance has drawn huge attention as it can transfer the unexpected cyber risks to the insurance companies [13]- [17], [25], [35]- [39]. The existing insurance framework could bring useful insights on modeling the cyber-insurance [13], [40]. The moral hazard models in the economics literature are good tools to capture the information asymmetry between the insured and the insurers [18]- [20]. Various frameworks and methodologies have been brought up to investigate cyber-insurance contracts and their impacts to cyber risks.
Several works have studied cyber-insurance through market-based approaches by analyzing the supply and demand relations between insurers and insureds [14], [16], [25], [35]. In [16], Pal et al., have analyzed regulated monopolistic and competitive cyber-insurance markets, and showed that cyber-insurance can improve the network security but the insurer can make zero expected profits in monopoly markets. In [14], [25], [35], Böhme et al., have presented several market models of cyber-insurance with the consideration of interdependency between cyber risks and information asymmetries between insurers and insureds, and showed analytical results on the impacts of cyber-insurance to cyber-security and the vialibity of a market for cyber-insurance.  Game theory has been used to capture the interactions between insurers and insureds of cyber-insurance [17], [36], [39]. In [39], Laszka et al., have used a two-player signaling game to capture the information asymmetry between a potential client and an insurer, and further studied incentives for auditing potential clients before cyber-insurance premium calculations. In [36], Grossklags et al., have presented several security games to capture the decision-making of network users on protections and insurance. The equilibrium analysis shows that users may seek to self-protect themselves at just slightly above the lowest protection level in the weakest-target game. In [17], Zhang et al. have studied the interactions between insureds, attackers, and insurers with a bi-level game-theoretic framework in a networked environment and demonstrated the impacts of network connections to the three types of players.

Summary of Notations
Most previous works have focused on the information asymmetry and interdependencies of cyber risks, however, their models have not captured the dynamics and correlations of the cyber risks, which have been studied with different methodologies and models [41]- [45]. In [44], Poolsappasit et al. have used a bayesian attack graphs model to analyze the network security risk assessment and mitigation. In [45], the authors have used a differential epidemic model to capture the spreading of viruses and worms in computer networks. These works aim to reduce the impacts of cyber risks through local protections, such as firewalls [7], intrusion detection [28], or moving target defenses [9], which cannot fully mitigate the risks of cyberattacks.
In this work, we focus on studying the dynamics and correlations of the cyber risks and analyzing the impacts of the cyber-insurance to both the insureds and the insurers. We first capture the cyber risks as well as the user's deployments of local protections with Markov decision processes, which have been used variously to analyze cybersecurity [46], [47]. We then use the existing moral hazard type of principal-agent model to capture the interactions between the user and the insurer with incomplete information. The analysis of both the optimal insurance contract and the user's response to it provides useful insights on the designation of the cyber-insurance contracts in the real world.

III. USER'S OPTIMAL PROTECTION POLICIES
We use discrete Markov decision processes (MDP) to capture the evolvements of the user's cyber risks with time, and an illustration is provided in Fig. 2. Let s t ∈ S denote the user's cyber-risk state at time t ∈ Z ≥0 , where S ≡ {S n |1 ≤ n ≤ N} is the set of all possible cyber-risk states. Different cyber-risk states may incur various types of losses, e.g., data breaches, physical device damages, and compromised financial accounts. In this paper, we consider that all types of losses are measurable and can be quantified by monetary direct losses. We assume that each cyber-risk state S n ∈ S is associated with a fixed direct loss X n ∈ R ≥0 , and the user's direct loss at time t can be denoted by x t ∈ X , where X ≡ {X n |1 ≤ n ≤ N}. losses. The objective of the user is to find the optimal protection sequence {a t } t≥0 which minimizes his cumulative losses. The user can also purchase cyber-insurance to mitigate his losses. The insurer first announces the insurance contract {K, r}, where K and r indicate the premium and the coverage function, respectively. The user can decide whether to purchase the insurance or not. If the user chooses to purchase the insurance, he must pay a premium K, and when he faces a loss of x, the insurer should provide a coverage of r(x) to him. The objective of the insurer is to maximize his profit. Note that the insurer has no information of the user's protection sequences.
The user can adopt different protections, such as firewalls, intrusion detection systems, and moving-target defenses, to reduce the possibilities of entering cyber-risk states that can incur severe losses. Let a t ∈ A denote the protections at time t, where A ≡ {A m |1 ≤ m ≤ M} is the set of all available protections. The transition probability p(s t , a t , s t+1 ) denotes the probability that the user goes to state s t+1 at time t + 1 when he is currently in state s t and adopts protection a t , which naturally captures the correlations among different cyber-risk states under different protections. Note that ∑ N n=1 p(s t , a t , S n ) = 1 as the user can only enter states within S at time t + 1.
We further provide two examples to illustrate the states S and protections A of the user. Example 1. Suppose a customer whose computer faces threats of Ransomware. In this example, the customer has S = {S 1 , S 2 } and A = {A 1 , A 2 }. States S 1 and S 2 denote that the computer is secure and compromised, respectively. The customer can choose to do nothing A 1 or add firewalls A 2 . The computer has a lower probability of facing Ransomware, i.e., entering state S 2 , if the customer deploys firewalls. When the computer is compromised, the customer needs to either pay the money or replace the computer, which can be covered if he has purchased cyber-insurance.
Example 2. Consider a cloud center who aims to protect itself from the damages caused by potential attackers. In this example, the cloud center has State S 1 denotes the situation when it is safe and faces no cyberattacks. However, the cloud center may encounter data breaches and denial-of-services, which are represented by states S 2 and S 3 , respectively. Each state S n ∈ S is associated with a direct loss X n . For example, at time t, s t = S 2 indicates that the cloud center faces data breaches which inflict X 2 direct losses to it. Specially, the direct loss X 1 = 0 at state S 1 , which indicates that the cloud center has no loss when it faces no cyberattacks. To defend against these cyberattacks, the cloud center may deploy firewalls, intrusion detection systems, and moving-target defense, which are represented by protections A 2 , A 3 , and A 4 , respectively. Specially, the cloud center can also choose to do nothing, which is denoted as A 1 . The cloud center has smaller probabilities of entering states with high losses if he deploys protections, however, these protections are also costly. The cloud center can also purchase cyber-insurance to cover part of its losses and help it recover from cyber-incidents. The objective of the cloud center is to find an optimal deployment of protections and cyber-insurance such that its future cumulative losses are minimized. Cases involve other cyber risks or protections can be extended through increasing the size of S and A .
Besides the protections, the user can also mitigate his losses through purchasing cyber-insurance.
After paying a premium to an insurer, the user could receive a coverage of r(x t ) from the insurer when he faces a direct loss of x t , where r : R ≥0 → R ≥0 is the coverage function of the insurance.
The objective of the user is to find an optimal sequence of protections {a t } t∈Z ≥0 that minimizes the expected cumulative effective losses given the initial state s 0 ∈ S , which can be captured as where function c(a t ) returns the cost of protection a t , and δ ∈ (0, 1) is the discount factor which indicates that future losses are valued less at time 0.
In this paper, we consider that the user decides his protections contingent on his current states.
Such feedback strategy allows the user to maintain his security level by adopting the necessary protections that can reduce the losses from cyberattacks and save the costs of protections at the same time. The strategy is usually denoted by a stationary protection policy π : S → A , e.g., π(S n ) = A m indicates that the user always takes protection A m at state S n . As a result, the expected cumulative effective losses of the user under a stationary protection policy can be captured as The user aims to find an optimal stationary protection policy π * r ∈ Ω that minimizes his expected cumulative effective losses given the coverage function r, and such objective can be captured as where Ω denotes the set of all possible stationary policies.
A rational user purchases the insurance only when the expected cumulative effective losses plus the premium under the insurance is lower than the losses without insurance, which can be captured as where K ∈ R ≥0 is the premium of the insurance and r 0 indicates a zero coverage function, i.e., r 0 (X) = 0 for all X ∈ R ≥0 , which corresponds to the case when there is no insurance. The fact that the user purchases the insurance only when inequality (4) is satisfied must be considered by the insurer while designing effective insurance contracts.
The optimal protection policy π * r could be obtained by solving (3) with either dynamic programming or linear programming [48], and we summarize both approaches in the following subsections.

A. Dynamic Programming Approach
Recall equation (2), let us define the loss function l(s t , a t , r) = x t − r (x t ) + c (a t ) which indicates the effective loss at time t under the coverage function r. Note that l(s t , a t , r) does not take x t as variable since the direct loss x t is uniquely determined by the user's cyber-risk state s t . Thus, we can express the expected cumulative effective losses as where l(s 0 , π(s 0 ), r) and δ ∑ s∈S p(s 0 , π(s 0 ), s)V (s, π, r) capture the effective loss at time 0 and the future expected cumulative effective losses, respectively. As a result, given a coverage function r, the optimal protection policy π * r can be found by the following dynamic programming operators [48].
By iterating (6) and (7) for all states s ∈ S until no further changes take place, we can achieve π * r and V (s, π * r , r), and the convergence to the optimum is guaranteed [48].

B. Linear Programming Approach
Besides the dynamic programming, we can also use linear programming to solve the user's problem (3). Problem (3) can be reformulated into a linear programming problem in the standard form as [48] min where η ∈ R NM×1 and θ ∈ R N×1 are denoted as the prime variable and the dual variable, respectively. Note that N and M are the sizes of S and A , respectively. Vector b ∈ R N×1 is a column vector of size N with all the elements equal to 1. Vector d ∈ R NM×1 is a column vector of size NM which captures the per-state and per-action losses, and the (N(n − 1) + m)-th element E ∈ R N×NM has that E n,N(n−1)+m = 1 for 1 ≤ n ≤ N and 1 ≤ m ≤ M and all the other elements are 0, and matrix P ∈ R N×NM is the transition probability matrix where P n ,N(n−1)+m = p(n, a m , n ), The optimal primal variable η * represents the optimal state-action frequencies; the optimal dual variable θ * represents the expected cost-to-go values of the states for the given coverage function r, i.e., θ * n = V (S n , π * r , r) for 1 ≤ n ≤ N. After solving the dual problem, we can find the optimal protection policy π * r by plugging V (S n , π * r , r) into (6).

IV. INSURER'S OPTIMAL INSURANCE CONTRACTS
In this section, we present and analyze the insurer's problem of designing cyber-insurance contracts. An illustration of the interactions between the user and the insurer has been provided in Fig. 2. Note that the insurer first announces the insurance contract {K, r}, and the user then makes the decision of purchasing the insurance based on the expected cumulative effective losses under that insurance contract. If the user chooses to purchase the insurance, the insurer instantly earns a profit of K at time 0, but the insurer is required to pay the coverage of r(x t ) when the user faces a loss of x t at time t. As a result, the insurer's operating profit can be captured as s 0 } denotes the expected cumulative coverage provided by the insurer to the user. The objective of the insurer is to find an optimal insurance contract {K * , r * } that maximizes his operating profit. As a result, the insurer's problem can be captured as Constraint (8a) captures the insurer's individual rationality that he chooses not to provide the insurance if he has a negative profit. Constraint (8b) captures the user's individual rationality on purchasing the insurance and it comes from inequality (4).
By solving problem (8), the insurer can find an optimal insurance contract which maximizes his operating profit and is acceptable by the user. After combing the user's problem and the insurer's problem, the interactions of the user and the insurer can be captured by the following Problem (9) is an optimization problem nested with various sub-optimization problems. The solution of Problem (9) captures both the user's objective of minimizing his expected cumulative effective losses and the insurer's objective of maximizing his own profit with the consideration of the user's rational choice of purchasing the insurance. To find the solution of problem (9), we can first solve the user's problem (3) and obtain the optimal protection policies π * r and the corresponding losses V (s 0 , π * r , r) to the coverage function r, and then achieve {K * , r * } by solving the insurer's problem (8).
We can simplify the insurer's problem (8) by exploring the expected cumulative effective losses and the optimal protection policies as discussed in the following subsection.

A. Insurer's Problem: Simplifications and Direct Conclusions
We first notice that the expected cumulative coverage is equal to the expected cumulative direct losses minus the expected cumulative effective losses, i.e., can be interpreted as the expected cumulative effective losses given the optimal protection policy π * r and the zero coverage function r 0 . Thus, Problem (8) can be rewritten as follows. max Constraint (10b) indicates that the maximum premium that can be charged by the insurer for a coverage function r is The user chooses not to purchase the insurance with a premium K > K max because the losses and the premium under the insurance are higher than the losses without insurance.
As a result, problem (8) is equivalent to the following problem after letting K be equal to K max and plugging (11) into its objective function and constraint.
where R denotes the set of all possible coverage functions, and the constraint indicates that the profit of the insurer cannot be negative. After solving (12), we can find the optimal coverage function r * , and then the optimal premium can be computed through (11). Similarly, the principalagent problem (9) can also be rewritten as Comparing to (8) and (9), we only need to find the optimal protection policies π * r to obtain the optimal insurance contract {K * , r * } through (12) and (13).
One useful insight regarding the operating profit could be obtained without solving (12) or (13), which is summarized in the following remark and proposition.
Remark 1. Any coverage function r that yields π * r = π * r 0 , i.e., the user has the same optimal protection policy between the case under the coverage function r and the case under no insurance, is a feasible solution with the corresponding premium K , and the insurer has a zero operating profit under that insurance contract as V (s 0 , π * r 0 , r 0 ) −V (s 0 , π * r , r 0 ) = V (s 0 , π * r 0 , r 0 ) −V (s 0 , π * r 0 , r 0 ) = 0.
Proposition 1. Any insurance contract {K, r} that yields π * r = π * r 0 and meets (11) is optimal for the insurer, and the insurer has a zero operating profit under that contract.
Proof. The operating profit of the insurer has that V (s, π * r 0 , r 0 ) −V (s, π * r , r 0 ) ≤ 0 from (13b), i.e., π * r 0 ∈ arg min π∈Ω V (s, π, r 0 ). Thus, the maximum profit that the insurer can achieve is 0. As a result, if the user has π * r = π * r 0 under an insurance contract {K, r}, that contract is feasible from Remark 1 and it is also optimal.
Remark 1 indicates that the insurer has a zero operating profit when the user has the same protection policies with or without insurance, which explains market neutrality. Proposition 1 indicates that the insurance contract in Remark 1 is optimal for the insurer. We denote this conclusion as the zero operating profit principle.

V. CASE STUDY: TWO-STATE TWO-ACTION USER AND LINEAR COVERAGE INSURER
In this section, we present a representative case where the user has two states and two actions and the insurer provides the linear coverage. Analysis of this case provides structural insights of the insurance contracts. Recall Section III, the user in this case has the set of states S GB ≡ {S G , S B }, where S G and S B indicate good state and bad state, respectively. The losses that associated with the states can be further identified as X G and X B . The difference between the good state and the bad state is that the user has lower losses at the good state than that at the bad state, i.e., To reduce the losses, the user can choose to take a strong protection A H or a weak protection A L , in other words, the user has the action set A HL = {A H , A L }. We further use shorthand notations C H and C L to represent the costs of protections A H and A L , respectively, i.e, c(A H ) = C H and c(A L ) = C L . The differences between a strong protection and a weak protection can be identified in detail as follows: • p(s, A H , S B ) < p(s, A L , S B ), ∀s ∈ S GB , which indicates that the user has a higher probability of going to the bad state when he has a weak protection.
• p(s, A L , S G ) < p(s, A H , S G ), ∀s ∈ S GB , which indicates that the user has a higher probability of going to the good state when he has a strong protection.
• 0 ≤ C L < C H , which indicates that the cost of a strong protection is higher than the cost of a weak protection.
These differences capture the fact that a strong protection can make the user more secure but its cost is also higher. An optimal protection policy π * ∈ Ω can be achieved by solving Problem (3) which minimizes the user's expected cumulative effective losses.
Besides protections, the user can also purchase the insurance to further mitigate his losses. We consider that the insurer offers a linear coverage with R ∈ [0, 1] denoting the coverage level of the insurance, i.e., r(x) = Rx. Specially, R = 0 and R = 1 indicate no coverage and full coverage, respectively.
Methods in Sections III and IV can be used to find the optimal protection policy of the user and the optimal insurance contract for the insurer. Since there are only two states and two actions for the user, we can find them analytically.

A. User's Optimal Protection Policy
We first introduce several notations to simplify representations. Since the user has only two states S G and S B , we use s c = s to denote the other state for a given state s ∈ S GB . Since the user adopts a stationary protection policy, i.e., he has fixed protections at each state, we identify his state protections as α G and α B for the good state and the bad state, respectively. We further define the action dependent expected cumulative effective loss function as follows Remark 2. For a protection policy π that has π(S G ) = α G and π(S B ) = α B , the expected cumulative effective loss function (5) is equivalent to the action dependent expected cumulative effective loss function (14), i.e., As a result, the dynamic programming operators (6) and (7) can be written as where Both (17) and (18) V (S B , α B ; α G , R) = δ p(S B ,α B ,S G )l(S G ,α G ,R)+(1−δ p(S G ,α G ,S G ))l(S B ,α B ,R) where As a result, we can find π * R by solving (15) and (16) with (19) and (20), respectively. Since there are only two protection choices A H and A L , we can find the optimal protection policy by comparing the action dependent expected cumulative effective losses under A H and A L . Lemma 1. The optimal protection policy π * R given the coverage level R can be summarized as follows: Proof. The user chooses a protection policy with lower expected cumulative effective losses in both good state and bad state.
We consider that the user always takes A L when V (s, A H ; α s c , R) = V (s, A L ; α s c , R). We can further simplify the comparisons in Lemma 1 as shown in the following proposition. Proof. See Appendix A.
Thus, we could obtain the optimal protection policy of the user by analyzing h(s, α s c , R), and we further have the following observation on it. Before we obtain the optimal protection policy π * R , we note the following proposition regarding the uniqueness of π * R .
Theorem 1. The optimal protection policy π * R is unique.
Proof. See Appendix B.
With Lemma 1, Proposition 3, and Theorem 1, we can obtain the optimal protection policies of the user with respect to the coverage level as stated in the following proposition.

Proposition 4. Let us define the value of transition probabilities as
The user's optimal protection policies with respect to the insurer's coverage level can be summarized with the following cases as also shown in Fig. 3. .
. , Proof. See Appendix C. We can see from Proposition 4 that the user tends to take weak protections with the increase of the coverage level in all cases, and this reckless behavior is often referred as the risk compensation [21]. One critical impact of the risk compensation is the Peltzman effect as shown in the following theorem.
Theorem 2. Peltzman effect: The user faces higher cyber risks under cyber-insurance. Such phenomena exists in the following cases: 1] in Case 2 and Case 4(b); 1] in Case 3 and Case 4(a); 1] in Case 4(c).
Proof. We only need to prove that the user has higher expected cumulative direct losses in these cases. Let V d (s, π) and V c (π) denote the expected cumulative direct losses and the expected cumulative costs given the initial state s and the protection policy π. We have that V d (s, π) + V c (π) = V (s, π, 0). Recall that the optimal protection policy π * 0 without insurance has V (s, π * 0 , 0) ≤ V (s, π, 0) for π ∈ Ω, thus, when the user has a different optimal protection policy π * R = π * 0 given the coverage level R, we have V (s, π * 0 , 0) ≤ V (s, π * R , 0). As a result, we can achieve R ) and the user faces higher cyber risks.

B. Optimal Insurance Contract
Recall Section IV, the insurer's problem (12) in this case can be written as follows: where τ(R) denotes the operating profit of the insurer if he provides a coverage level of R. Note that S G in τ(R) indicates that the initial state of the user is the good state. .
Proof. We can obtain the optimal insurance contract for all cases using the results from Proposition 1. In Case 1, any coverage level R * ∈ [0, 1] is optimal, and the associated pre- where k(S G , A L ; A L ) comes from (27) in Appendix A. Similarly, we can obtain the optimal insurance contracts in Cases 2, 3, and 4.
We can see from the optimal insurance contracts that the premium is linear on the coverage level, which can be summarized as the linear insurance contract principle. Moreover, all the optimal insurance contracts lead to a zero operating profit for the insurer, which indicates a zero-operating profit principle. The optimal insurance contracts usually provide limited coverage levels. When the coverage level is high, the user tends to act recklessly, which induces high risks and high direct losses of the user as shown in Theorem 2, and the insurer is required to cover the extra losses caused by that, which induces a negative profit of him. As a result, the insurer chooses not to provide the insurance to the user in that case.

VI. NUMERICAL EXAMPLES
In this section, we first present numerical experiments on a two-state two-action user and a linear coverage insurer to verify our previous analytical results. We then present numerical experiments on a four-state three-action user with a linear coverage insurer and a threshold coverage insurer.

A. Two-State Two-Action User and Linear Coverage Insurer
In this subsection, we aim to verify our analysis on the two-state two-action user and the linear coverage insurer with numerical experiments. We assume that the user has δ = 0.9, X G = 0, With the dynamic programming approach or linear programming approach in Section III, we can compute the optimal protection policies and the expected cumulative effective losses of the user as shown in Figs.4(a,b). We can further calculate the premium and the operating profit of the insurer as shown in Figs.4(c,d). We can see that the numerical results coincide with our analytical results.

B. Four-State Three-Action User and Linear Coverage Insurer
In this subsection, we consider a more complicated example where the user has four states and three actions, and the insurer provides linear coverage. We show that our model can be used to analyze the interactions between the user and the insurer in a numerical way.
We assume that the user's states can be identified as S G , S B,1 , S B,2 , and S B,3 with the state losses The user has a larger probability of going to the good state and a smaller probability of going to the bad state with better protections. We then take the following transition probabilities in this example: Let δ = 0.9, the optimal protection policies and the expected cumulative effective losses of the user are shown in Figs.5(a,b). We can see from (a,b) that the user decreases his protections with the increase of the coverage level, and the user also has lower expected cumulative effective losses with higher coverage levels. The premium and the operating profit of the insurer are shown in Figs.5(c,d). We can see that with the increase of the coverage level, the premium is linearly increasing. Moreover, the maximum operating profit that can be achieved by the insurer is 0. We can also observe that the optimal insurance contract tends to provide limited coverage levels, and higher coverage levels can lead to negative operating profits of the insurer. Thus, the risk compensation, the zero-operating profit principle, and the linear insurance contract principle still hold in this example.

C. Four-State Three-Action User and Threshold Coverage Insurer
In this subsection, we consider a threshold coverage insurance and show its impact on the four-state three-action user and the insurer.
We use the same settings for the user as in the previous subsection. The threshold insurance contract has two coverage levels R 0 = 0 and R 1 = 0.9, which are distinguished by a threshold 20]. When the loss of the user x ≤ X R , the insurer provides no coverage R 0 x, otherwise, the insurer provides a coverage R 1 x. A lower X R indicates that the insurance has a higher coverage for smaller losses. The objective of the insurer is to maximize his operating profit by finding the optimal threshold X * R and the associated premium K * . The optimal protection policies and the expected cumulative effective losses of the user are shown in Fig.6(a,b), and we can see from them that the user decreases his protections with the decrease of the threshold X R , which indicates that the user tends to act recklessly knowing that the insurer provides a high coverage even he has a small loss from cyber risks. Moreover, we can see that the premium is a staircase function on the threshold X R , and it decreases with the increase of X R , which shows that the insurer charges a higher premium to provide a higher coverage level. The maximum operating profit that can be achieved by the insurer is 0. As a result, this example shows the similar risk compensation and zero-operating profit principle as in the previous examples. Note that the gray area has X R > X B,3 , i.e., the insurer provides no coverage for the user at any states, which is equivalent to the case when there is no insurance.

VII. CONCLUSION
In this paper, we have presented a dynamic moral-hazard type of principal-agent model to study the cyber-insurance and its impacts on the cyber-security. The dynamics and correlations of the cyber risks have been modeled by Markov decision processes where the user aims to find the optimal protection policy to mitigate the impacts of cyberattacks. Both the computational and analytical tools have been presented to design the optimal cyber-insurance contracts. We have studied and fully analyzed a case where the user has two states and two actions, and the insurer provides linear coverage insurance. We have further demonstrated the Peltzman effect that the user has higher cyber risks under insurance due to risk compensation, i.e., the user tends to act more recklessly knowing he is protected. We have presented the linear insurance contract principle and the zero-operating profit principle of the optimal cyber-insurance contract.
Numerical experiments have been used to corroborate our results and further demonstrate the case study with a four-state three-action user and his interactions with linear coverage insurance and threshold coverage insurance. The risk compensation and the zero-operating profit principle have been shown to hold in these cases. One direction of future research is the investigation of cyber-insurance contracts over complex networks such as scale-free and small-world networks with dynamic cyber risks. the threshold X R . The green area denotes the region of optimal insurance contracts.
APPENDIX A. PROOF OF PROPOSITION 2 To simplify the notation in this proof, we define the discounted transition probabilities as Remark 3. The following facts hold for p: (ii) If δ = 1, we have p(S G , α G , S G ) = p(S G , α G , S B ) and p(S B , α B , S B ) = p(S B , α B , S G ).
Thus, (21) can be written as Remark 4. The following facts hold for I p : The action dependent expected cumulative effective losses (19) and (20) can be rewritten as follows: where Note that where the fourth equality is achieved by plugging Remark 4(i)(ii). Similarly, we can achieve that As a result, we have where h(s, α s c , R) has been defined in Proposition 2. Since 1 − δ > 0, p(s, α s , s) > 0, and If π * R = Π HL , we have h(S G , A L , R) < 0 and h(S B , A H , R) ≥ 0 from Proposition 2. Thus, π * R = Π LL and π * R = Π HH . Similarly, if π * R = Π LH , we have π * R = Π LL and π * R = Π HH ; if π * R = Π LL , we have π * R = Π LH and π * R = Π HL ; if π * R = Π HH , we have π * R = Π LH and π * R = Π HL . Thus, to prove the uniqueness of π * R , we only need to prove that (i). π * R = Π LH and π * R = Π HL cannot exist at the same time and (ii). π * R = Π LL and π * R = Π HH cannot exist at the same time.
which violates h(S G , A L , R) < 0. As a result, π * R = Π LH and π * R = Π HL cannot exist at the same time.
If π * R = Π LL and π * R = Π HH at the same time, we have h(S G , A L , R) ≥ 0, h(S B , A L , R) ≥ 0, h(S G , A H , R) < 0, and h(S B , A H , R) < 0, which indicates that ρ < 0 from (35)  There are only four possible protection policies Π LL , Π HL , Π LH , and Π HH . Thus, the optimal protection policy π * 0 without insurance has only four cases: Case 1, Case 2, Case 3, and Case 4 as presented in Proposition 4, which are determined by h(s, α s c , 0) in Proposition 2. As a result, we only need to prove the trends of π * R with respect to R in different cases. We first note that when R = 1, we have h(S G , α B , 1) > 0 and h(S B , α G , 1) > 0, which indicates that π * R=1 = Π LL . Moreover, if the user has π * R = Π LL for a coverage level of R ∈ [0, 1], we have h(S G , A L , R) ≥ 0 and h(S B , A L , R) ≥ 0 from Proposition 2. Since h(s, α s c , R) is linearly increasing on R as shown in Proposition 3, we have h(S G , A L , R) ≥ 0 and h(S B , A L , R) ≥ 0 for R ≥ R, which indicates that π * R = Π LL for R ≥ R. Thus, we can conclude that π * R = Π LL when R is sufficiently large and the user chooses not to change his policy with the increase of R once he achieves π * R = Π LL for all cases.
To prove Cases 2, 3, and 4, recall the value of transition probabilities ρ from (22). If ρ < 0, we have h(S G , A H , R) < h(S B , A L , R) from (37). However, when π * R = Π LH , we have h(S G , A H , R) ≥ 0 and h(S B , A L , R) < 0 from Proposition 2, which violates h(S G , A H , R) < h(S B , A L , R). Thus, we have π * R = Π LH if ρ < 0. As a result, Case 2 and Case 4(a) has ρ < 0 and the user have π * R = Π LH in these cases. Since h(S B , A H , 0) ≥ 0 when π * 0 = Π HL in Case 2 and h(S B , A H , R) is linearly increasing on R, h(S B , A H , R) ≥ 0 for R ∈ [0, 1]. Thus, the user has π * R = π HH in Case 2 and Case 2 holds. The threshold R G is achieved by solving h(S G , A L , R) = 0. Case 4(a) holds from Case 2, and the thresholds R G and R B are achieved by solving h(S G , A L , R) = 0 and h(S B , A H , R) = 0, respectively.
If ρ > 0 and π * R = Π HL , we have h(S G , A L , R) < 0 and h(S B , A H , R) ≥ 0 from Proposition 2 and thus h(S B , A H , R) − h(S G , A L , R) = ρδ ((C H −C L ) − (1 − R)(X B − X G )) > 0 from (38), which indicates that (C H −C L ) − (1 − R)(X B − X G ) > 0. However, we could obtain that h(S G , A L , R) > 0 following similar arguments as in (39), which violates h(S G , A L , R) < 0. Thus, we have π * R = Π HL if ρ > 0. As a result, Case 3 and Case 4(b) have ρ > 0 and the user has π * R = Π HL in these cases. Since h(S G , A H , R) ≥ 0 when π * 0 = Π LH in Case 3 and h(S G , A H , R) is linearly increasing on R, h(S G , A H , R) ≥ 0 for R ∈ [0, 1]. Thus, the user has π * R = π HH in Case 3 and Case 3 holds. However, Π LH and Π HL indicate that h(S G , A H , R) ≥ 0 > h(S B , A L , R) and h(S G , A L , R) < 0 ≤ h(S B , A H , R), respectively. Thus, we have π * R = Π HH and π * R = Π LL if ρ = 0. As a result, Case 4(c) has ρ = 0 and the user has π * R = Π HL and π * R = Π LH . Thus, Case 4(c) holds, and the thresholds R G and R B are achieved by solving h(S G , A H , R) = 0 and h(S B , A H , R) = 0, respectively.