{'fraction_non_alphanumeric': 0.03934799398398439, 'fraction_numerical': 0.01170684118531767, 'mean_word_length': 4.625886119368855, 'pattern_counts': {'":': 0, '<': 10, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 5, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Piecewise linear switched dynamical systems: A review

This review paper introduces piecewise linear switched dynamical systems in three topics. In the first topic of autonomous chaotic circuits, we introduce the manifold piecewise linear system and chaotic spiking oscillator. Using piecewise exact solutions and mapping procedure, we obtain rigorous proof of chaos generation. In the second topic of recurrent neural networks, we introduce the hysteresis neural network and its application to associative memories. Performing theoretical analysis based on the piecewise exact solutions, we obtain parameter conditions for guaranteed storage of any desired memories. In the third topic of multiobjective optimization problems, we introduce a two-objective problem in a piecewise linear model of switching power converter with photovoltaic input. Applying a simple multiobjective evolutionary algorithm, we clarify existence of a trade-off between the maximum input power and circuit stability.


Introduction
A switched dynamical system is a continuous-time nonlinear system defined by multiple subsystems and nonlinear switching rules [1][2][3][4]. If the state variable fulfills a condition, a switching event occurs and the future dynamics is determined. Depending on the switching rules and nonlinear characteristics, the system exhibits nonlinear phenomena such as periodic orbits, synchronization, chaos, and bifurcation. Mathematical models of the switched dynamical system are used to describe/analyze various engineering systems including artificial neural networks, analog-to-digital converters, analog signal processing systems, and switching power converters. Analysis and synthesis of the switched dynamical systems are important not only as basic study of nonlinear dynamics but also for engineering applications.
This review paper introduces our previous works on piecewise linear switched dynamical systems. The dynamics is described by multiple linear subsystems connected by nonlinear switching rules. The advantages/disadvantages include the following.
where x denotes the dimensionless state variable, τ denotes the dimensionless time, andẋ ≡ dx dτ . Let the right hand side of Eq. (1) be either (+) or (−) at τ = 0. In order to define the switching rule, we introduce notations The right hand side of Eq. (1) is switched from (+) to (−) if a trajectory hits L − as shown in Fig. 1. The right hand side of Eq. (1) is switched from (−) to (+) if a trajectory hits L + .
The MPL is characterized by three parameters: the damping δ, the equilibrium point p, and the switching threshold T h . For simplicity, we assume 0 < δ < 1, 0 < p < 1, T h = 0.
In this case, Eq. (1) has unstable complex characteristic roots δ ±jω, where ω ≡ √ 1 − δ 2 . As shown in Fig. 1, the trajectories rotate divergently around equilibrium points ±p. If the trajectory hits negative x-axis (L − ) then the equilibrium point is switched from p to −p. If the trajectory hits positive x-axis (L + ) then the equilibrium point is switched from −p to p. Note that the switching occurs only on x-axis (L). Repeating the rotation and switching, the MPL can exhibit chaotic trajectories as shown in Fig. 2(a) and (b). The trajectories can be calculated by the piecewise exact solution where (x(0),ẋ(0)) is an initial condition at τ = 0. In order to define the return map, we consider a trajectory started from a point x 0 ∈ L at τ = 0 where a point on L is represented by its x coordinate. The trajectory intersects L at τ = π/ω and let x 1 be the intersection. Since x 0 determines x 1 , we can define the one-dimensional return map F from L to itself. The map is piecewise linear and is described exactly: where β ≡ e δπ ω . Now the dynamics is integrated into the iteration x n+1 = F (x n ). The chaos generation is guaranteed if 1 < β < 2. In this case, there exists an invariant interval I 1 on which the map is expanding:  where DF (x) is the slope of F at x. In this case, chaos generation is guaranteed theoretically in the sense of ergodic and positive Lyapunov exponent [17]. Figure 3 shows return maps corresponding to chaotic attractors in Fig. 2. Note that trajectories diverge for β > 2. As stated earlier, the MPL can be implemented by a simple hardware and chaos generation can be confirmed experimentally. Simple hardware examples can be found in [14].

Chaotic spiking oscillator
In order to realize chaotic behavior in autonomous circuits, an effective nonlinear element is required. The integrate-and-fire switch is used as a key element in the chaotic spiking oscillators (CSO). Note that the integrate-and-fire switching rule is basic to design a variety of spiking neuron models [27][28][29]. In Fig. 4, S denotes the integrate-and-fire switch and N denotes a linear sub-circuit. If the capacitor voltage v 1 reaches a threshold voltage V T , v 1 is reset to the constant base voltage E, instantaneously. If the sub-circuit N consists of resistors and dependent sources, it can be replaced with the Thevenin equivalent sub-circuit and the circuit can exhibit periodic waveforms as shown in Fig. 4(b). If the sub-circuit N includes one memory element (inductor or capacitor) as shown in Fig. 4(c), the integrateand-fire switch causes vibrate-and-fire dynamics. If N includes 1 or more memory elements, v 1 can vibrate below the threshold and the integrate-and-fire switch can cause chaotic behavior.
The CSO is defined in the case where the sub-circuit consists of one inductor and one negative linear resistor as shown in Fig. 4(c). If the capacitor voltage v is below the threshold V T , the switch S is opened and the circuit dynamics is described by v 1 is assumed to vibrate divergently. As v 1 reaches V T , the integrate-and-fire switch S is closed and v 1 is reset to the base E instantaneously holding the continuity property of i: Repeating the integrate-and-fire switching, the circuit generates chaos. Using discrete elements, this circuit can be fabricated easily and chaotic behavior can be confirmed experimentally. A simple hardware example can be found in [16]. We assume that Eq. (5) has unstable complex characteristic root In this case v 1 can vibrate divergently below the threshold V T . The divergent vibration and the integrate-and-fire switching correspond to stretching and folding mechanisms, respectively, which are fundamental for chaos generation. Using the following dimensionless variables and parameters: Equations (5) and (6) are transformed into the following.
whereẋ ≡ dx/dτ and p = δ. This dimensionless equation is characterized by two parameters δ and q which can be controlled by −R and E, respectively. This equation exhibits chaotic attractor as shown in Fig. 5. The circuit dynamics can be analyzed using one-dimensional return map. Some objects of the map are shown in   Let q < 0 and let a trajectory start from a point y 0 on L d at τ = 0. If 0 < y 0 < D, the trajectory return to L d at τ = 2π without reaching L T . If D ≤ y 0 , the trajectory hits the threshold L T and is reset to the base L q . Then the trajectory re-starts from L q and returns to L d . Since any trajectory started from y 0 on L must return to L d , a one-dimensional return map can be defined: where y 1 is the return point on L d . That is, the circuit dynamics can be integrated into the iteration y n+1 = f (y n ) as shown in Fig. 6. Using piecewise exact solution of Eq. (9), the return map can be described and chaos generation is guaranteed theoretically in the sense of ergodic and positive Lyapunov exponent [16,17].

Related systems
Chaotic phenomena in the MPL and CSO are analyzed theoretically using one-dimensional return maps. Such theoretical analysis is possible for other piecewise linear switched dynamical systems including hysteresis chaos generators [30][31][32][33] and diode chaos generators [34,35]. These systems are implemented by simple analog hardware and the chaotic phenomena are confirmed experimentally. Especially, we have presented two simple designs of hysteresis chaos generators: an RC-OTA hysteresis chaos generators [32] and a 2-port VCCS hysteresis chaos generators [33]. They are convenient to consider physical mechanism of chaos generation and to consider simple integrated circuits of chaotic circuits. If the number of memory elements increases, we obtain higher-order chaotic circuits that generate hyperchaos and related rich bifurcation phenomena [36][37][38][39][40][41]. If the pulse-coupling is applied to multiple CSOs, we obtain pulse-coupled network of CSOs [39]. This network exhibits a variety of synchronization phenomena of periodic/chaotic spike-trains. The real/prospective applications of the network include flexible image processing [42], associative memories [43], and spike-based communication [44].

Recurrent neural networks
Recurrent neural networks are continuous-time dynamical systems referring to inhibition/excitation and parallel distributed dynamics of biological neural networks. The network is constructed by connecting simple elements with nonlinear activation function. As a pioneer mathematical model of the network, the Amari-Hopfield network is known [45,46]. The dynamics is described bẏ where x is the state variable and f (x) is a smooth activation function such as the sigmoidal function. This network can exhibit a variety of nonlinear phenomena, coexisting equilibrium points, coexisting periodic orbits, synchronization, bifurcation, and chaos. In the classic studies, networks with stable equilibrium points are applied to various systems including associative memories and combinatorial optimization problem solvers [45][46][47][48]. These applications are based on Lyapunov-function stability analysis. The study of the recurrent neural networks has continued intensively. In recent years, the dynamic behavior (periodic/chaotic orbits) are used in reservoir computation systems [49]. As a piecewise linear switched dynamical system in recurrent neural networks, we introduce hysteresis neural networks (HNNs [19,20]).

Hysteresis neural networks
The HNN is characterized by binary hysteresis activation function as shown in Fig. 7: h(x) is switched from −1 to 1 (respectively, 1 to −1) if x reaches the threshold T h > 0 (respectively, −T h). The HNN dynamics is described bẏ where x ≡ (x 1 , · · · , x N ) is the state variable vector and y ≡ (y 1 , · · · , y N ), y i ∈ {−1, 1}, is the binary output vector. w ij denotes connection parameters and d i denote offset parameters. This piecewise linear switched dynamical system is governed by N pieces of linear subsystems connected by the hysteresis switching of h(x i ). If the output vector is fixed, the system is described by linear equatioṅ  cases of N = 2 and N = 3, the HNN exhibits various periodic orbits, synchronization, torus, chaos, and various bifurcation phenomena [50][51][52]. As N increases, the HNN can exhibits various nonlinear phenomena, however, theoretical analysis of them is not easy.
Here, we introduce application to self-associative memories in the case where the steady state is stable equilibrium points. In the associative memories, key problems are storage of desired memories (e.g., desired images) and suppression of spurious memories (e.g., unnecessary images). The HNN have several advantages: trajectories can be calculated precisely by piecewise exact solutions and test circuits are implemented by simple analog circuits. In recurrent neural networks with smooth activation function, the trajectory cannot be calculated exactly and theoretical analysis of the dynamics is not easy. For simplicity, we focus on the following case of the parameters: Theoretical results in more general case can be found in [19]. An equilibrium point of the HNN is given by An equilibrium point p(y) is said to be stable if An equilibrium point p(y) is said to be unstable if We can say that any initial state converges to either equilibrium point if The desired memories are a set of multiple binary vectors: If we can determine parameters such that the equilibrium point of s l is stable then the desired memory s l is said to be stored into the network. If an equilibrium point of some output vector y is stable and the y is not a desired memory, then the y is said to be a spurious memory. Storage of all the desired memories is guaranteed if Note that if a desired memory s k is stored then its inverse vector −s k is also stored because of the system symmetricity in Eq. (14). Hence half of the desired memories should be inverse vectors of the other half desired memories. It should be noted that this storage condition must be satisfied as the self-feedback parameter w ii is adjusted to satisfy the condition. That is, in this network, storage of any desired memories is possible. It should be noted that even if all the desired memories are stored, there usually exist spurious memories. Suppression of the spurious memories is not easy in various associative memories. These desired memories can be stored into 8-dimensional HNN (N = 8) if the connection of parameters (connection matrix W b ) are given by This full connection matrix is determined by correlation based learning and binalization presented in [53]. However, the example has 88 spurious memories. In order to suppress the spurious memories, we have applied the a simple sparsification algorithm [53] and have obtained a sparse connection matrix W s in Eq. (21). In this case, the network has only 4 spurious memories.

Related works
The HNNs are large scale systems and detailed analysis of the nonlinear dynamics is not easy. However, we have analyzed the dynamics almost completely in the case where the connection parameters are uniform [54]. Using a simple potential function, we have clarified theoretically that the HNN exhibits bifurcation from co-existing stable equilibrium points to multi-phase synchronization of stable periodic orbits. The potential function is developed into combinatorial optimization problems solvers such as an N-Queens problem [55], a dynamical N-Queens problems [56], and a Box puzzling problems [57]. In the application to the N-Queen problems (P vs NP problems), computation cost is calculated precisely up to N = 150. The calculation is executed by piecewise exact solutions of the HNN. The results suggest that the parallel processing of the HNN can reduce the computation cost that increases at most polynomial of N .

Multiobjective optimization problems
Multi-objective optimization problems (MOPs) have been studies in various fields of science [58][59][60][61][62][63]. Although uniobjective optimization problems require the optimization of only one objective function, the MOPs require the simultaneous optimization of multiple objective functions, e.g., cost and performance in various engineering problems. If the simultaneous optimization is achieved, we can obtain the optimal solution; however, usually the optimal solution is not feasible. In the MOPs, we encounter various difficulties. One typical problem is the presence of conflicting objectives, where an improvement in one objective may cause a deterioration in another objective. The task is to find solutions (Pareto set) which balance a trade-off. The solutions in a design space correspond to the Pareto front in the objective space. In order to find the Pareto front effectively, various evolutionary algorithms have been presented [60][61][62][63]. The multiobjective evolutionary algorithm based on decomposition (MOEA/D [60]) is known as one of the most efficient algorithms. The algorithms performance has been investigated in benchmarks such as multiobjective 0-1 knapsack problems.
In this section, we introduce two-objective problem in a piecewise linear switched dynamical system: a piecewise linear circuit model of switching power converter with photovoltaic input [64,65]. The first objective corresponds to extraction of the maximum input power. The second objective corresponds to robust/stable circuit operation. These objectives are important to realize efficient low carbon and renewable energy supply [66][67][68][69][70][71]. The two objectives are described by functions of control parameters of the converter. Using a piecewise linear modeling of the power converter, the two objective functions are described exactly. In order to analyze the two-objective problem, we apply a simple version of the MOEA/D. Figure 8(a) shows a boost converter with photovoltaic input. In order to analyze the dynamics precisely, we apply two simplifications used in [64]. First, the photovoltaic input of smooth characteristics is simplified into two-segment piecewise linear current-controlled voltage source as shown in Fig. 8(b):

The circuit model and objective problem
where (I P , V P ) denotes the breakpoint. In the circuit, the switch S and diode D can be either of the 2 states: State 1: S conducting and D blocking State 2: S blocking and D conducting As illustrated in Fig. 8(c), the switching rule is defined by where T is the clock period and J − is the lower threshold of input current. Second, we assume that the time constant of the load is much larger than the clock period RC T . In this case, the load is simplified into a constant voltage source V out . The circuit dynamics is described by Using the dimensionless variables and parameters Equations (23) and (22) are transformed into Since Eq. (25) is piecewise linear, we can use piecewise exact solutions in the analysis. Note that the five dimensionless parameters are classified into two categories. The three parameters (α, β, q) characterize photovoltaic input and output. The two parameters (γ, X − ) characterize switching control. We select (γ, X − ) as control parameters and other parameters are fixed: As the control parameters (γ, X − ) vary, the circuit exhibits various periodic/chaotic behavior [64]. For simplicity, we consider a periodic orbit with period 1. As shown in Fig. 9, an orbit started from x 0 reaches the lower threshold X − and is switched to increase. The orbit increases and reaches a point x 1 at time τ = 1. Since x 1 is determined by x 0 , we can define a map Using the piecewise exact solutions, the map F can be calculated exactly. The periodic orbit with period 1 corresponds to a fixed point of the map Using p as the initial point, the periodic orbit can be calculated exactly. This periodic orbit corresponds to input current i in with period T . For the periodic orbit, we calculate instantaneous input power p in with period 1 and the average input power P A : Fig. 9. Periodic orbit and stability.
p in (τ ) = x(τ )y(x(τ )), p in (τ + 1) = p in (τ ), P A = 1 0 p in (τ )dτ (28) Figure 10 shows typical examples of the periodic orbits. In order to consider stability of the periodic orbit, we define contraction rate as shown in Fig. 9: If |DF (p)| < 1 then the periodic orbit of x(τ ) and periodic instantaneous input power p in (τ ) are stable. If |DF (p)| > 1 then they are unstable. Using the piecewise exact solutions, the contraction rate can be calculated exactly [21]. The problem is optimal setting of the control parameters (γ, X − ) for (I) extraction of the maximum average input power (characterized by P A ) and (II) stabilization of the instantaneous input power (characterized by |DF (p)|). Figure 10

Two-objective problem and MOEA/D
In order to consider the trade-off between the maximum power point and stability, we present two objective functions where the normalized factors 1 are set to be P max = 1.1 and DF max = 2. As f 1 approaches to 0, the input power approaches to the maximum power point. As f 2 approaches to 0, the periodic orbit approaches to be super-stable. These two objective functions can be calculated precisely using the piecewise exact solutions. The two-objective problem is defined by In the real-valued objective functions f : S D → S O , S D and S O are referred to as the design space and the objective space, respectively. Let d ≡ (γ, X − ) ∈ S D be a decision value. A decision value d = d a ∈ S D is said to dominate another decision value d b ∈ S D if either of the following is satisfied: A decision value d p is referred to as a Pareto optimal solution if d p is not dominated by any other decision value. The set of all the Pareto optimal solutions is named a Pareto set while its image in the objective space is named Pareto front. A point z * is referred to as a reference point if the z * gives the minimum of all the objectives. A Pareto front and a reference point are illustrated in Fig. 11. If the Pareto front can be found, we can clarify existence of a trade-off. In order to solve the two-objective problem efficiently, we use a simplified version of the MOEA/D [21]. In the algorithm, the Tchebycheff decomposition approach is used [60,62]. This approach uses a reference point where i = 1, 2. In the algorithm, the two-objective problem is decomposed into uniobjective subproblems. The j-th subproblem is represented by the j-th adaptability function.
Step 2 (Update): For j = 1, · · · , N, do • Randomly select two solutions d k and d l from d j , j ∈ B(j) and then generate a new solution δ j by using a crossover operator [21].
• Using δ j and Eq. (36), the closest neighbor set B(j) is updated.
• Update of EP: Remove from EP all elements dominated by f (δ j ). Add f (δ j ) to EP if no elements in EP dominate f (δ j ).
Step 3 (Termination): g ← g + 1, go to Step 2, and repeat until the maximum generation g max . After the algorithm is terminated, the external population EP gives the Pareto front.
Applying the algorithm to the two-objective problem in Eq. (31), we have obtained the Pareto front as shown in Fig. 12. These results guarantee existence of a trade-off between the maximum power point of the photovoltaic input (in average input power) and stability of periodic orbit (in instantaneous input power). The Pareto set gives important information to set suitable parameter values in design of efficient and stable renewable energy supply systems.

Conclusions
Piecewise linear switched dynamical systems in three topics have been reviewed in this paper. In autonomous chaotic circuits, the MPL and CSO have been introduced. The dynamics is described by simple linear subsystems and switching rules with hysteresis. Using the piecewise exact solutions, the embedded one-dimensional return maps can be described exactly and rigorous proof of chaos generation is achieved. The MPL and CSO are implemented by simple analog hardware and chaos generation is confirmed experimentally.
In recurrent neural networks, the HNN has been introduced. The HNN is constructed by connection of artificial neurons with hysteresis activation function. The dynamics of described by multiple piecewise linear subsystems and the hysteresis switching. In application to self-associative memories, storage of any desired memories has been guaranteed theoretically. The HNN is applicable also to combinatorial optimization problems solvers.
In multiobjective optimization problems, a two-objective problem in a piecewise linear model of boost converter with photovoltaic input has been introduced. The first objective represents extraction of the maximum input power. The second objective represents stability of periodic orbit. Using the piecewise exact solutions, the two objective functions are calculated exactly. Applying a simplified version of the MOEA/D, the Pareto front is obtained effectively. The Pareto front guarantees existence of a trade-off between the maximum power point and stability.
Referring to these reviews, one research direction can be considered. That is, analysis of switched dynamical systems in two kinds of perspectives: nonlinear dynamics and multiobjective optimization. In a nonlinear dynamics perspective, a key target is bifurcation sets in the design space. In a multiobjective optimization perspective, a key target is Pareto front in the objective space. An important point is relationship between the two kinds of perspectives. Although bifurcation analysis is important in nonlinear dynamics, the bifurcation sets are not sufficient information to realize desired system operation. Although the Pareto front analysis is important in multiobjective optimization, the Pareto front is not sufficient information to clarify dynamics that causes the trade-off. If we consider both the bifurcation sets and Pareto front, more deep understanding of the objective systems is possible. Such consideration seems to be developed into excellent analysis and synthesis methods of nonlinear circuits and systems.