{'fraction_non_alphanumeric': 0.03774447727592514, 'fraction_numerical': 0.005065428450823132, 'mean_word_length': 4.187773722627737, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 6, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 15, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Flip the Cloud: Cyber-Physical Signaling Games in the Presence of Advanced Persistent Threats

Cyber-physical systems (CPS) have the potential to provide scalable, rapidly deployable, cost effective, and on-demand enhancements of physical systems through the use of advanced computational processes run on infrastructure such as a cloud. On the other hand, CPS are subject to numerous design challenges; among them is that of security. Secure CPS must appropriately manage the interdependence between cyber and physical components of a system. In particular, recent advances in adversary technology pose Advanced Persistent Threats (APT) which may stealthily and completely compromise a cyber system. In this paper, we design a framework for a cloud-enabled CPS that specifies when a device should trust commands from the cloud which may be compromised. This interaction can be considered as a game between three players: a cloud defender/administrator, an attacker, and a device. We use traditional signaling games to model the interaction between the cloud and the device, and we use the recently proposed FlipIt game to model the struggle between the defender and attacker for control of the cloud. Because attacks upon the cloud can occur without knowledge of the defender, we assume that strategies in both games are picked according to prior commitment. Because of this, the game requires a new equilibrium concept, which we call Gestalt Equilibrium, a fixed-point of mappings that express the interdependence of the signaling and FlipIt games. We present the solution to this fixed-point problem under certain parameter cases, and illustrate an example to the cloud control of an unmanned vehicle. Our results contribute to the growing understanding of CPS and cloud controlled systems.

. The cloud has the potential to use in cyber physical system (CPS). In CPS, there are different kinds of sensors. Each sensor measures specific physical processes. The cloud infrastructure gathers all sensors information. The cloud process the information and send the corresponding task to each cyber physical system. Cloud computing has garnered significant attention in both industry and academia, and it is still an evolving topic. Cloud computing owners specifically design infrastructure to provide low cost and on-demand services to users. Users can build their own virtual machines in the cloud and buy the capacity that they require with precision. Thus, cloud computing can provide cost-effective solutions for both parties. In addition, cloud computing has the potential to provide scalability, rapid deployment, and ubiquitous accessibility.
Despite the advantages that cloud computing provides, it also has some drawbacks. They include -but are not limited to -accountability, virtualization, and security and privacy concerns. In this paper, we focus especially on providing accurate signals to a cloud-connected device and deciding whether to accept those signals in the face of security challenges.
Recently, system designers face security challenges in the form of Advanced Persistent Threats (APT ) [17]. APT arise from sophisticated attackers who can infer a user's cryptographic key or leverage zero-day vulnerabilities in order to completely compromise a system without detection by the system administrator [14]. This type of stealthy and complete compromise has demanded new types of models [6] [18] for prediction and design.
In this paper, we consider a cloud-enabled CPS, such as the one shown in Fig. 1. Specifically, we propose a model in which the cyber component of the system -i.e., the cloud -is vulnerable to APT and may fall under adversarial control. We synthesize a mathematical framework that enables devices controlled by the cloud to intelligently decide whether to obey commands from the possiblycompromised cloud or to rely on their own lower-level control. This requires us to carefully study the interaction between the physical and cyber components of the system.
We model the physical aspect of the cloud-enabled CPS using a signaling game. Signaling games provide a proper framework for modeling dynamic interactions in which one of the players operates based on a belief about the private information of the other. A significant body of research has utilized this frame-work for security [7] [19] [13] [8]. We model the physical aspect of the cloud-enabled CPS, using the recently proposed FlipIt game [6] [18]. This game is especially suited for studying systems under APT. The signaling and FlipIt games are coupled, because the outcome of the FlipIt game determines the likelihood of benign and malicious attackers in the robotic signaling game. Because the attacker is able to compromise the cloud without detection by the defender, we consider the strategies of the attacker and defender to be chosen with prior commitment. The circular dependence in our game requires a new equilibrium concept which we call a Gestault equilibrium 1 . We specify the parameter cases under which the Gestault equilibrium varies, and solve a case study of the game to give an idea of how the Gestault equilibrium can be found in general. Our proposed framework has versatile applications to different cloud-enabled CPS such as urban traffic control, drone delivery, design of smart homes, etc. We study one particular application in this paper -control of an unmanned vehicle under the threat of a compromised cloud.
Our contributions are as follows: • We model the interaction of the physical and cyber aspects of cloud-enabled CPS by introducing a novel game consisting of two coupled games: a traditional signaling game and the recently proposed FlipIt game.
• We provide a general framework by which a physical device connected to a cloud can decide whether to follow its own limited control ability or two trust the signal of a possibly-malicious cloud.
• We propose a new equilibrium definition for this combined game: Gestault equilibrium, which involves a fixed-point in the mappings between the two component games.
• Finally, we apply our framework to the problem of unmanned vehicle control.
In the sections that follow, we first outline the system model, then describe the equilibrium concept. Next, we use this concept to find the equilibria of the game under selected parameter regimes. Finally, we apply our results to the control of an unmanned vehicle. In each of these sections, we first consider the signaling game, then consider the FlipIt game, and last discuss the synthesis of the two games. Finally, we conclude the paper and suggest areas for future research.

System Model
We model a cloud robotics challenge in which a cloud is subject to APTs. In this model, an attacker, denoted by A, (she) capable of APTs can pay an attack cost to completely compromise the cloud without knowledge of the cloud defender. The defender, or cloud administrator, denoted by D, (he), does not observe these attacks, but has the capability to pay a cost to reclaim control of the cloud. The cloud transmits nearly continuous messages to a robotic resource, denoted by R, (it). The robot resource may follow these commands, but it is also equipped with an on-board control system for autonomous navigation. It may elect to use its authonomous navigation system rather than obey commands from the cloud.
This cloud robotics challenge involves two games: the FlipIt game introduced in [18], and the well-known signaling game. The FlipIt game takes place between the attacker and cloud defender, while the signaling game takes place between the possibly-compromized cloud and the robotic resource. For brevity, denote the FlipIt game by G F , the signaling game by G S , and the combined game -call it CloudControl -by G CC as shown in Fig. 2. In the next subsections, we formalize this game model.

Cloud-Robot Signaling Game
Let θ denote the type of the cloud. Denote compromized and safe types of clouds by θ A and θ D in the set Θ. Denote the probabilities that θ = θ A and that θ = θ D by p and 1 − p. Signaling games typically give these probabilities apriori, but in CloudControl they are determined by the equilibrium of the FlipIt game G F .
Let m H and m L denote messages of high and low risk, respectively, and let m ∈ M = {m H , m L } represent a message in general. After R receives the message, it chooses an action, a ∈ A = {a T , a N }, where a T represents trusting the cloud and a N represents not trusting the cloud.
For the robotic resource R, let u S ⊂ R for all possible θ, m, a. u S R is a utility function such that u S R (θ, m, a) gives the robot's utility when the type is θ, the message is m, and the action is a. Let u S A : M × A → U A and u S D : M × A → U D be utility functions for the defender and receiver. We have U X = [min u S X (m, a), max u S X (m, a)] ⊂ R, where X ∈ {A, D} for all possible m, a. Note that these players only receive utility in G S for the proportion of time that their type controls the cloud in G F , so that type is not longer a necessary argument for u S A and u S D . Denote the strategy of R by σ S R : A → [0, 1], such that σ S R (a | m) gives the mixed-strategy probability that R plays action a when the message is m. The role of the sender may be played by A or D depending on the state of the cloud, determined by G F . Let σ S A : M → [0, 1] denote the strategy that A plays when she controls the cloud, so that σ S A (m) gives the probability that A sends message m. (The superscript S specifies that this strategy concerns the signaling game.) Similarly, let σ S D : M → [0, 1] denote the strategy played by D when he controls the cloud. Then σ S D (m) gives the probability that D sends message m. Let Γ S R , Γ S A , and Γ S D denote the sets of mixed strategies for each player.
gives the expected utility to sender X when he or she plays mixed-strategy σ S X and the receiver plays mixed-strategy σ S The CloudControl game. The FlipIt game models the cyber aspect of CPS. Due to possibility of compromising cyber aspect, the physical aspect is modeled by signaling game. These two games play at the same time, i.e., concurrently.
Next, let µ : Θ → [0, 1] represent the belief of R, such that µ (θ | m) gives the likelihood with which R believes that a sender who issues message m is of type θ.
gives the expected utility for R when it has belief µ, the message is m, and it plays strategy σ S R . u S R is given bȳ The expected utilities to the sender and receiver will determine their incentives to control the cloud in the game G F describing in the next subsection.

FlipIt Game for Cloud Control
The basic version of FlipIt [18] 2 is played in continuous time. Assume that the defender controls the resource -here, the cloud -at t = 0. Moves for both players obtain control of the cloud if it is under the other player's control. In this paper, we limit our analysis to periodic strategies, in which the moves of the attacker and the moves of the defender are both spaced equally apart, and their phases are chosen randomly from a uniform distribution. Let f A ∈ R + and f D ∈ R + (note that R + represents non-negative real numbers) denote the attack and renewal frequencies, respectively.
Players benefit from controlling the cloud, and incur costs from moving. Let w X (t) denote the average proportion of the time that player X ∈ {D, A} has controlled the cloud up to time t. Denote the number of moves up to t per unit time of player X by z X (t). Let α D and α A represent the costs of each defender and attacker move. van Dijk et al. consider a fixed benefit for controlling the cloud. In our formulation, the benefit depends on the equilibrium outcomes of the signaling game G S . Denote the equilibrium utilities of D and A byū S * D and u S * A . These give the expected benefit of controlling the cloud. Finally, let u F D (t) and u F A (t) denote the time-averaged benefit of D and A up to time t. Then and as time continues to evolve, the average benefits over all time becomẽ We next express these expected utilities over all times as a function of periodic strategies that D and A employ.
give the average utility to D and A, respectively, when they play with frequencies f D and f A . If and if f A = 0, we havē Note that f A = 0 means that the attacker does not have any incentive to attack the cloud. In other words, what the attacker receives from controlling the robot via cloud is lower than the cost of its control.
Equations (5)-(9) with Equation (1) forū S X , X ∈ {D, A} and Equation (2) forū S R will be main ingredients in our equilibrium concept in the next section.

Solution Concept
In this section, we develop a new equilibrium concept for our CloudControl game G CC . We study the equilibria of the FlipIt and signaling games individually, and then show how they can be related through a fixed-point equation in order to obtain an overall equilibrium for G CC .

Signaling Game Equilibrium
Signaling games are a class of dynamic Bayesian games. Applying the PBE concept (see [9]) to G S , we have Definition 1.
where the functionsū S X σ S R , σ S X , X ∈ {D, A} andū S R σ S R are formulated according to Equation (1) and Equation (2), respectively.
Letū S * D ,ū S * A , andū S * R be the utilities for the defender, attacker, and device, respectively, when they play according to a strategy profile σ S * D , σ S * A , σ S * We will employ T S as part of the definition of an overall equilibrium for G CC after examining the equilibrium of the FlipIt game.

FlipIt Game Equilibrium
For the FlipIt game, definition 2 applies the concept of NE to G F .
whereū F D andū F A are computed by Equation (5) and Equation (6) if f D ≥ f A and Equation (7) and Equation (8)  To find an overall equilibrium of G CC , we are interested in the proportion of time that A and D control the cloud. As before, denote these proportions by p and 1 − p, respectively. These proportions can be found from the equilibrium frequencies by Now, we can define a mapping T F : U D × U A → [0, 1] which maps the values of controlling the cloud for the defender and attacker (ū S * D andū S * A , which come from the equilibrium of the signaling game) to the proportion of time that the attacker controls the cloud in equilibrium. This mapping gives In addition to interpreting p as the proportion of time that the attacker controls the cloud, we can view it as the likelihood that, at any random time, the cloud will be controlled by the attacker. Of course, this is precisely the value p of interest in G S . Clearly, G F and G S are coupled by Equations (14) and (18). These two equations specify the overall equilibrium for the CloudControl game G CC through a fixed-point equation, which we describe next.

Gestalt Equilibrium of G CC
When the CloudControl game G CC is in equilibrium the mapping from the parameters of G S to the game's equilibrium and the mapping from the parameters of G F to that game's equilibrium are simultaneously satisfied as shown in Fig.  3. Definition 3 formalizes Gestalt equilibrium.

Definition 3 (Gestalt equilibrium)
The cloud control ratio p † ∈ [0, 1] and equilibrium signaling game utilitiesū S † D andū S † A constitute a flip-fixed equilibrium of the game G CC composed of coupled games G S and G F if Equation (19) is simultaneously satisfied.
In short, the control ratio p † must satisfy the fixed-point equation In this equilibrium, A receivesū F A according to Equation (6) or Equation (8), D receivesū F D according to Equation (5) or Equation (7), and R receives u S R according to Equation (2).
Solving for the equilibrium of G CC requires a fixed-point equation essentially because the games G F and G S are played according to prior committment. Prior committment specifies that players in G S do not know the outcome of G F . This structure prohibits us from using a sequencial concept such as subgame perfection and suggests instead a fixed-point equation.

Analysis
In this section, we analyze our proposed game in Section 2 based on our solution concept in Section 3. First, we analyze the signaling game and calculate the corresponding equilibrium. Then, we solve the FlipIt game for different value of expected payoff resulting from signaling game. Finally, we propose our theorem showing under which condition our equilibrium concept exists.

Signaling Game Analysis
The premise of G CC allows us to make some basic assumptions about the utility parameters that simplifies the search for equilibria. We expect these assumptions to be true across many different contexts.
• A1: u R (θ D , m L , a T ) > u R (θ D , m L , a N ): It is more beneficial for receiver to trust low risk action of defender type of sender.
• A2: u R (θ A , m H , a T ) < u R (θ A , m H , a N ): It is more beneficial for receiver to not trust high risk action of attacker type of sender.
• A3: u X (m , a T ) > u X (m, a N ), X ∈ {A, D}, m , m ∈ {m L , m H } : Both sender's type prefer that the receiver trusts their actions whether it is high or low risk.
• A4: u A (m H , a T ) > u A (m L , a T ): High risk action is more valuable for attacker than low risk when the receiver trusts the sender's action.
Pooling equilibria differ in regions defined by quantities in Equations (21) and (22) which we call trust benefit. T B H (p) and T B L (p) give the benefit of trusting (compared to not trusting) high and low messages, respectively, when the prior probability is p. These quantities specify whether R will trust a message that it receives in a pooling equilibrium.
We illustrate the different possible combinations of T B H (p) and T B L (p) in the quadrants of Fig. 4.1. The labeled messages and actions for the sender and receiver, respectively, in each quadrant denote these pooling equilibria. For example, in Quadrant I, there are pooling equilibria in which A and D pool on both m H and m L , and R plays a T . These pooling equilibria apply throughout each entire quadrant. Note that we have not listed the requirements on belief µ here. These are addressed in the appendix, and become especially important for various equilibrium refinement procedures.
The colored regions of Fig. 4.1 denote additional special equilibria which only occur under the additional parameter constraints listed within the regions. and Blue are pooling equilibria similar to those already denoted in the equilibria for each quadrant, except that they do not require restrictions on µ. Yellow is a separating equilibrium, although it is a rather unproductive one for D and A, since their messages are not trusted. The equilibria depicted in Fig. 4.1 will become the basis of analyzing the mapping T S (p) which will be crucial for forming our fixed point equation that defines the flip-fix equilibrium. Before studying this mapping, however, we first analyze the equilibria of the FlipIt game on its own.

FlipIt Analysis
In this subsection, we calculate both A and D players' Nash equilibrium in FlipIt game. Equations (5)-(8) represents both players utilities in FlipIt game. Note that, solution of this game is similar to what has presented in [18], [6] except that the reward of controlling the resource is not equal to one. To calculate Nash equilibrium, we normalize both players' benefit with respect to the reward of controlling the resource. different cases the frequency of move at Nash equilibrium are: In the above equation, 0 + means that attacker attacks the cyber aspect only once. Since, the average utility of defender is negative, he does not have any incentive to defend against his resource. Therefore, if attacker attacks (moves) only one time, she control the resource all the times.
Next, we put together the analysis of G S and G F in order to study the flip-fix equilibria of the entire game.

G CC Analysis
From assumptions A1-A4, it is possible to verify that (T B L (0) , T B H (0)) must fall in Quadrant I or Quadrant IV and that (T B L (1) , T B H (1)) must lie in Quadrant III or Quadrant IV. Clearly, there are numerous ways in which the set (T B L (p) , T B H (p)) , p ∈ [0, 1] can transverse different parameter regions. Rather than enumerating all of them, we consider one here. . This leads to an L that will traverse from Quadrant I to Quadrant IV. Let us also assume that u D (m L , a T ) < u D (m H , a N ), so that Equilibrium 5 is not be feasible. These parameters lead to the equilibria specified in Table 1. The equilibrium numbers refer to the derivations in the appendix. Table 1. Signaling game equilibria by region for a game that traverses between Quadrant I and Quadrant IV. Some of the equilibria are feasible only for constrained beliefs µ, specified in Appendix. We argue that the equilibria in each region marked by (*) will be selected.

Region Equilibria
Quadrant I Equilibrium 3: Pool on mL; µ constrained; R plays aT *Equilibrium 8: Pool on mH ; µ unconstrained; R plays aT In Quadrant I, both senders prefer pooling on m H . By the first mover advantage, they will select Equilibrium 8. On the border between Quadrant I and Quadrant IV, A and D both prefer an equilibrium in which R plays a T . If they pool on m L , this is guaranteed. If they pool on m H , however, R receives equal utility for playing a T and a N ; thus, the senders cannot guarantee that the receiver will play a T . Here, we assume that the senders maximize their worst-case utility, and thus pool on m L . This is Equilibrium 3. Finally, in Quadrant IV, 4 These parameters must satisfy uR (θD, mH , aT ) > uR (θD, mH , aN ), uR (θD, mL, aT ) > uR (θD, mL, aN ), uR (θA, mH , aT ) < uR (θA, mH , aN ), and uR (θA, mL, aT ) > uR (θA, mL, aN ). Here, we give them specific values in order to plot the data.
both senders prefer to be trusted, and so select Equilibrium 3. From the table, we can see that a plot of the ratioū S * A /ū S * D will have a jump at the border. This jump is illustrated by the blue, diamond points in Fig. 6.
Next, consider the mapping p = T F ū S * D ,ū S * A . we have noted, p depends only on the ratioū S * A /ū S * D 5 . Indeed, it is continuous in that ratio when the outcome at the endpoints is appropriately defined. p = T F ū S * D ,ū S * A is represented by the red boxes in Fig. 6, with the independent variable on the vertical axis.
We seek a fixed-point, in which p = T F ū S * D ,ū S *

Cloud Control Application
In this section, we study a specific application of CloudControl games: automated vehicle control through access to the cloud. Access to the cloud offers robots such as automated vehicles several benefits [11]. First, it allows access to massive computation resources -i.e., infrastructure as a service (IaaS ) (See [5]). Second, it allows access to large datasets. These datasets can offer superaddative benefits to the sensing capabilities of the vehicle itself, as in the case of the detailed road and terrain maps that automated cars such as those created by Google and Delphi combine with data collected by lidar, radar and vision-based cameras [1], [10]. Third, interfacing with the cloud allows access to data collected or processed by humans through crowd-sourcing applications; consider, for instance, location-based services [15], [16] that feature recommendations from other users. Finally, the cloud can allow vehicles to collectively learn through experience [11].
Attackers may attempt to influence cloud control of the vehicle through several means. In one type of attack, adversaries may be able to steal or infer cyptographic keys that allow them authorization into the network. These attacks are of the complete compromise and stealth types that are studied in the FlipIt framework [18], [6] and thus are appropriate for a CloudControl game. FlipIt also provides the ability to model zero-day exploits, vulnerabilities for which a patch is not currently available. Each of these types of attacks on the cloud pose threats to unmanned vehicle security and involve the complete compromise and steathiness that motivate the FlipIt framework.

Dynamic Model for Cloud Controlled Unmanned Vehicles
In this subsection, we use a dynamic model of an autonomous car to illustrate the interaction between the cyber and physical components of the CloudControl game for unmanned cars. We consider a car moving in two-dimensional space with a fixed velocity v but with steering that can be controlled. (See Fig. 7, which illustrates the "bicycle model" of steering control from [3].) For simplicity, assume that we are interested in the car's deviation from a straight line. (This line might, e.g., run along the center of the proper driving lane.) Let z (t) denote the car's vertical distance from the horizontal line, and let θ (t) denote the heading of the car at time t. The state of the car can be represented by a two-dimensional vector w (t) z (t) θ (t) T . Let δ (t) denote the angle between the orientation of the front wheel -which implements steering -and the orientation of the length of the car. We can consider δ (t) to be the input to the system. Finally, let y (t) represent a vector of outputs available to the car's control system. The self-driving cars of both Google and Delphi employ radar, Lidar, and vision-based cameras for localization. Assume that these allow accurate measurement of both states, such that y 1 (t) = z (t) and y 2 (t) = θ (t). If the car stays near w (t) = 0 0 T , then we can approximate the system with a linear model. Let a and b denote the distances from the rear wheel to the center of gravity and the rear wheel to the front wheel of the car, respectively. Additionally, define the variable Let v 0 = v cos κ (δ) denote the speed of the rear wheel. Then the linearized system is given in [3] by the equations:

Control of Unmanned Vehicle
Assume that the unmanned car has some navigation capability without the help of the cloud, but that the cloud typically provides more advanced navigation.
Self-Control Specifically, consider a control system onboard the unmanned vehicle designed to return it to the equilibrium w (t) = 0 0 T . Because the car has access to both of the states, it can implement a state-feedback control. Consider a linear, time-invarient control of the form More advanced controls are possible, but we use a proportional control in order to provide a simple, clear illustration. This proportional control results in the closed-loop system d dt The unmanned car R may also elect to obtain data or computational resources. Typically, this additional access would improve the control of the car. The cloud administrator (defender D), however, may issue faulty commands or there may be a breakdown in communication of the desired signals. In addition, the cloud may be compromised by A in a way that is stealthy. Because of these factors, R sometimes benefits from rejecting the cloud's command and relying on its own navigational abilities. Denote the command issued by the cloud at time t by δ cloud (t). With this command, the system is given by d dt

Filter for High Risk Cloud Commands
In cloud control of an unmanned vehicle, the self-navigation state feedback input given by δ car (t) in Equation (31) represents the control that is expected by the vehicle given its state. If the signal from the cloud differs significantly from the signal given by the self-navigation system, then the vehicle may classify the message as "high-risk." Specifically, define a difference threshold τ , and let Equation (34) translates the actual command from the cloud (controlled by D or A) into a message in the cloud signaling game. Equations (32) and (33) give the dynamics of the unmanned car electing to trust and not trust the cloud. Based on these equations, Fig. 8 illustrates the combined self-navigating and cloud controlled system for vehicle control.

Conclusion and Future Works
In this paper, we have proposed a general framework for cloud-enabled cyberphysical system. We have modeled the cyber aspect of the game using the FlipIt game and the physical aspect using a traditional signaling game. Because these two games are played by prior commitment, they are coupled. We have defined a new equilibrium concept -i.e., Gestalt equilibrium which defines a solution to the combined game using a fixed-point equation. After illustrating various parameter regions under which the game may be played, we solved the game in a sample parameter region. Finally, we showed how the framework may be applied to unmanned vehicle control.