{'fraction_non_alphanumeric': 0.03415249832949736, 'fraction_numerical': 0.016667904076026432, 'mean_word_length': 4.713467656415695, 'pattern_counts': {'":': 0, '<': 6, '<?xml version=': 0, '>': 2, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Designing a Pseudo-Random Bit Generator with a Novel 5D-Hyperchaotic System

Dynamic and non-linear systems are emerging as potential candidates for random bit generation. In this context, chaotic systems, which are both dynamic and stochastic, are particularly suitable. This paper introduces a new continuous chaotic system along with its corresponding implementation, which targets field-programmable gate array (FPGA). This chaotic system has five dimensions, which exhibit complex chaotic dynamics, thus enabling the utilization of chaotic signals in cryptography. A mathematical analysis is presented to demonstrate the dynamic characteristics of the proposed hyperchaotic system. A novel digital implementation of the proposed system is presented. Moreover, a data scrambling circuit is implemented to eliminate the bias effect and increase the randomness of the bitstream generated from the chaotic signals. We show that the proposed random bit generator has high randomness. The generated bits successfully pass well-known statistical randomness test-suites, i.e., NIST SP800-22, Diehard, and TestU01. The ready-to-use random bit generator is deployed on a Xilinx Zynq-7000 SoC ZC702 Evaluation Kit. Experimental results show that the proposed random bit generator can achieve a maximum throughput of 6.78 Gbps, which is over 3.6 times greater than state-of-the-art designs while requiring under 4% of the resources available on the targeted FPGA.


I. INTRODUCTION
R ANDOM number generators are critical components that are responsible for generating public keys, private keys, and other kinds of random numbers that are utilized in cryptographic applications and security [1]- [3]. Differential chaotic systems, which present high dynamic characteristics and multi-dimensional signals, are superior in terms of generating random bits due to their ability to achieve a high level of randomness [4]. Therefore, in this work, we focus on the design of such systems, targeted at field-programmable gate arrays (FPGAs).
A hyperchaotic system exhibits rich dynamics since the system states it hosts are expanded exponentially in several directions simultaneously. This property makes the hyperchaotic system an interesting candidate for the generation of random keys used in miscellaneous applications in engineering, such as secure communications, cryptosystems, and encryptions [5], [6]. Therefore, in the present work, we develop a 5D hyperchaotic system with three positive Lyapunov exponents to provide better dynamic characteristics than the state of the art. Moreover, high-dimensional chaotic systems provide multiple outputs which improve the throughput of the overall random bit generator.
The implementation methodology adopted has a great impact on the digitalization of differential chaotic systems. The research in [7] presented an implementation of numerical techniques, including the Euler, mid-point, and fourth-order Runge-Kutta (RK) methods on FPGAs. Among these, the Euler method has the shortest data path, but the least accuracy. The fourth-order RK method shows the highest accuracy, but also has the longest time frame. In state-of-the-art RK algorithm implementations, mapping functions are used four times and implemented separately [8], [9]. We propose the fourth-folding Runge-Kutta (FFRK) method to allow reusing the mapping functions (the latter are implemented only once and reused four times), and as a result, the device resources required to implement the mapping functions are reduced by 75%. The iterations are controlled by adding MUXs and control signals, which use insignificant amounts of device resources. Moreover, taking advantage of the multiple dimensional signals in hyperchaotic systems, we implement an effective post-processing, in which five chaotic outputs are used to generate ready-to-use random bitstreams. In summary, the contributions of this work include i) a new 5D hyperchaotic system to generate stochastic signals, ii) a novel implementation of the fourth-order RK algorithm, and iii) a simple and effective data post-processing circuit.
The remainder of this paper is organized as follows. Related works are reviewed in Section II. The mathematical model of the hyperchaotic system is clarified in Section III. The implementation of the proposed hyperchaotic system in a FPGA hardware device using the new FFRK method is detailed in Section IV. The experimental resuts are presented and discussed in Section V. Then, the randomness evaluation of the random bit generator is presented in Section VI. Finally, Section VII summarizes the work in this paper.

II. RELATED WORKS
In this section, we review the differential chaotic systems and their implementations in the digital world to emphasize our research contribution. There are several commonly used differential chaotic systems in secure communications, including the Lorenz system, Chua's system, Liu's system and Lu's system [10], [11]. The Lorenz system has been used in different applications in many research works [10]. The research in [10] presents the implementation of a Lorenz system in FPGA hardware devices and co-simulation with Matlab. Liu's system is implemented in [11] using the Grunward-Letniknov algorithm. However, the above chaotic systems utilize multiple multiplexers, which are resource-demanding in hardware implementations. Moreover, the authors in [12] provide both analog and digital implementations of the Bruke-Shaw chaotic system in a Virtex-6 FPGA chip, which consumes a lot of device resources. The research in [13] presents an FPGA realization of Chua's system with multiple scrolls by using different saturated functions. The work in [8] implements a 3D chaotic system with no equilibrium points. A high-speed FPGA implementation of a 3D continuous chaotic system, which achieves a maximum operating frequency of 293 MHz, is presented in [9].
The authors in [14] present a new hybrid PRNG using Sprott 94 G chaotic system which improves security of the advanced encryption standard (AES) in cryptographic systems. Although instructive, these chaotic systems are 3-dimensional systems which has only one positive Lyapunov exponent. This means that thay have limited dynamic characteristics as the signal expands exponentially in only one direction. Speed optimization is applied to a 3D chaotic system and then expanded to a 4D chaotic system in [15], achieving a maximum throughput of 1882 Mbps for the random bit generator.
Nowadays, differential chaotic systems achieve higher complexity by conveying integer-order systems into the fractionalorder domain [16]. However, fractional-order systems are complicated to implement in hardware design due to their memory dependency. The hardware implementation of fractional-order differentiators and integrators requires careful consideration [11].
Although chaotic systems are unpredictable, and have random-like state trajectories, they can be studied and recovered by using computational tools. A cyber-attack has a high possibility of success if the target system uses a wellknown and self-excited oscillator for its random bit generator. After the transient process, a trajectory, starting from the point of an unstable manifold in a small neighborhood of unstable equilibrium, can be revealed. Therefore, the system's parameters can be computed to recover the target system. From this point of perspective, the development of modern computers enables the numerical simulation of complex nonlinear dynamical systems, and therefore the structure of their trajectories can be deduced. However, this approach shows very limited success when it comes to hidden attractor chaotic systems [17], [18]. The hyperchaotic systems with hidden attractors in [19], [20] were proposed to overcome such attacks. As compared to previous systems, a mathematical analysis of our hyperchaotic system with hidden attractors shows rich dynamic characteristics that are suitable for use in security and cryptographic applications.

III. 5D DIFFERENTIAL CHAOTIC SYSTEM MODEL
The following section presents the proposed hidden attractor hyperchaotic system, which is expressed by five differential equations as given in (1). The theoretical analysis is divided into three parts. The first part presents the proposed chaotic system and a theoretical study of chaotic characteristics, while the second part addresses the stability of the equilibrium. The simulation of chaotic transitions is investigated in the third part.

A. Mathematical Analysis
The proposed 5D dynamic system, which is developed from the 4D hyperjerk chaotic system presented in [21], is expressed as S = F (S), S = (x, y, z, u, v) ∈ R 5 , with (1) The ordinary differential equations (ODEs) are solved and simulated in MATLAB, based on the fourth-order RK integration algorithm with a step size of 10 −2 . Here, a small step size is chosen to provide a higher resolution and better accuracy. The equilibrium points of the proposed system are located on the line E(x, 0, 0, 0, 0). Therefore, the proposed system is a dynamical system with hidden attractors. According to [20], it is difficult to expose a chaotic attractor, and then reveal the chaotic system architecture by choosing an arbitrary initial condition. In other words, the chaotic attractors are invisible to attackers and their basin localization does not dissolve the chaotic oscillator. In what follows, we present a mathematical analysis of the novel chaotic system in terms of Lyapunov exponents and bifurcation diagrams.
Lyapunov exponents: The Lyapunov exponents of the system are defined as: ∂u(0) , and L 5 = lim t→∞  (1) is evaluated based on the following conditions: Lyapunov dimension: The Lyapunov dimension (D L ), which is closely related to the correlation dimension, is commonly used to evaluate the chaotic complexity [22]. A higher Lyapunov dimension value is associated with a chaotic system with a higher level of complexity. where j is the largest index of the positive Lyapunov exponent. In the proposed system, j = 3, and the Lyapunov dimension is therefore:

B. Stability of Equilibria
The stability evaluation of the equilibria is an important step to find the chaotic region of a dynamic system. To evaluate the stability of equilibria, the Jacobian matrix at the equilibria E = [c, 0, 0, 0] of the proposed hyperchaotic system is calculated. The eigenvalues of the Jacobian matrix satisfy the condition J E − λI = 0.
Assuming a small perturbation from the fixed points x(t) = x(0) + ∆x. If ∆x ≈ e λ t, the characteristic polynomial equation is derived as: This equation indicates that the Jacobian matrix at equilibrium points has one zero value and three non-zero values. According to the Routh-Hurwitz criterion, apart from the zero eigenvalues, the real parts of the roots of (6) are negative if and only if (c − 0.5) > 0. To make the equilibrium set E unstable, thereby enabling the possibility of chaos occurrence, the initial condition x(0) = c must satisfy c < 0.5. In other words, depending on the initial value, the proposed system has stable or unstable saddle-focus points. Thus, the dynamical behaviors of the line equilibrium chaotic system are heavily dependent on the initial state of the variable x(t).

C. Transitions to Chaotic Region
The bifurcation diagram and the Lyapunov spectrum of the state variable x(t) of the proposed system are shown in Fig. 2 and Fig. 3, respectively. Both figures illustrate that the value of the initial condition c has an impact on the characteristics of the system. Moreover, a stable region is observed for 0.5 < c < 0.92 in Fig. 2, where the data space converges. This corresponds to the negative Lyapunov exponents in Fig. 3. A dynamic and unbounded region is observed for c > 0.92, a region that corresponds to positive high-value Lyapunov exponents in Fig. 3. Finally, in the 0 < c < 0.5 range, the system is dynamic and bounded in a limited data space, as illustrated in the bifurcation diagram. This corresponds to the presence of positive Lyapunov exponents L 1 and/or L 2 , as shown by the Lyapunov spectrum. However, the system only exhibits rich and chaotic dynamics for c < 0.05, where a high positive value of L 1 is presented, as shown in Fig. 3. The gradual transition from periodic orbits to chaotic regions based on various values of the initial condition c is observed in Fig. 4. The chaotic region with rich dynamics illustrated under Fig. 4-d) is obtained with c = 0.05 which is compatible to the stability analysis of equilibria in the previous section.

IV. PSEUDO-RANDOM NUMBER GENERATOR SCHEME
In this section, we present a pseudo-random number generator scheme based on the chaotic output signals. We use fixedpoint 32-bits to represent data, with 1-bit for the sign and 27 bits for the fractions. Fig. 5 shows the scheme of the proposed random bit generator which includes the implementation of the 5D chaotic system block using FFRK with 32-bit fixedpoint data. The truncation block that follows keeps the 12 least-significant bits of the stream. Then, the data is upsampled and converted from parallel to serial before entering the post-processing block. In the post-processing block, a data scrambler is proposed using a simple LFSR of one of the chaotic outputs, while the random bits are obtained by XORing the output of the data scrambler with the other chaotic signals.
Due to the long data path of the RK algorithm and low selfoscillator frequency of the chaotic system, signal truncation is applied in order to have better randomness qualification in the following block. Therefore, 32-bit chaotic outputs are truncated, where only 12 least significant bits (LSB) are used in the following post-processing. 12-LSBs are up-sampled and serialized before the post-processing step. In what follows, we present a novel FPGA-based implementation of the fourthorder RK algorithm in a new manner to solve the differential chaotic equations. Moreover, we present the post-processing process with a data scrambler to increase the randomness of the generated bitstreams and eliminate bias effects.

A. Implementation of the 5D Hyperchaotic System
To solve differential equations, there are three well-known numerical methods, with different complexity and accuracy. These are, the Euler method, the Midpoint method, and the RK method. Compared to the other two candidates, the RK method has the longest calculation path and requires larger areas [7]. However, it produces the most accurate results. The proposed FFRK implementation optimizes the resources of the 5D hyperchaotic system. Compared to previous RK implementations, the proposed FFRK provides the most accurate solution for differential equations and consumes fewer device resources. The system state at the n-th time step is denoted by the vector S n = [x n , y n , z n , u n , v n ]. The proposed hyperchaotic system discrete-time implementation is calculated as follows: The fourth-order RK algorithm is obtained by defining the primitive parameters k 1 , k 2 , k 3 , and k 4 which are calculated as: where h is the discrete step size. The next system state vector S n+1 is then evaluated using the previous system state S n and  these above mentioned primitive parameters. Fig. 6 presents the block diagram implementation of F (S n ) in (7), which is reused to implement the parameters k i with different input variables. The high-level tool (XSG) automatically maps those functions to FPGA resources. The z −i notation indicates the insertion of i registers. We can see that the longest path amounts to 5 registers, which results in 5 clock cycles of output latency for the function F (S n ). The block diagram presented in Fig. 7 shows the implementation of the FFRK algorithm. The discrete step size h = 0.01 provides enough resolution for chaotic signals. First, the parameter k 1 is calculated while the register enable control signal E 1 is high.
The output k 1 from the block F (S n ) will be temporary stored in register R 1 . Then, the parameters k 2 , k 3 , and k 4 are produced at the high level of the control signals E 2 , E 3 and E 4 , respectively. Therefore, four primitive parameters k 1 , k 2 , k 3 , and k 4 are stored in four registers. Finally, the next state of the system is produced when all these parameters are ready at the high period of the enable signal E 0 of register R 0 . The enable signals are generated by the control unit which uses delay blocks. The phase spaces of the chaotic outputs of the proposed implementation shown in Fig. 8 are compatible with the simulation results and mathematical analysis.

B. Data Post-Processing Implementation
Before we address the post-processing process, we truncate 32-bit chaotic signals as indicated in Fig. 5. In previous works, the truncation was mentioned and applied [23], [24]. In [23], random bits are generated from the 33 rd bit to the 40 th bit in each 52-bit data. The authors in [24] used 16-bits out of a 32-bit signal for the post-processing. It is obvious that the full data-width in chaotic signal is not used to generate random bits due to the low self-oscillators. Nevertheless, to obtain sufficient randomness quality, no clear reason is presented for choosing the truncation position. The least significant bits (LSBs) are preferred due to their higher fluctuation. Therefore, it is understood that the bits going from the LSB to the truncation position will be used in the post-processing. Our research shows that the signal entropy is strongly related to the truncation position. To evaluate the truncated chaotic signal's entropy, Shannon's entropy of a bitstream is calculated. For a given random variableX n , with a possible outcome x i , i ⊂ [0, N − 1], each with probability p i , the signal entropy is: where N is the number of symbols. The average entropy per bit is calculated as:Ē = H(X n ) N b (11) in which,X n denotes the truncated chaotic signal and N b = log 2 N is the truncation position. Fig. 9 provides the average entropy per bit with different lengths of truncated signals. The average entropy per bit decreases significantly when the number of remaining bits is larger than 12. Therefore, in this work, a 12-bit truncated chaotic vectorS n = [X n ,Ỹ n ,Z n ,Ũ n ,Ṽ n ] is used in the data post-processing. The proposed data post-processing hardware implementation based on shift registers and XOR operations is presented in Fig. 10. The data scrambler is composed of four shift registers (m-shift registers) and XOR operations [25], [26]. The working principle is to evaluate the incoming bits from the truncated chaotic signals and to then reuse them. One of the chaotic signal outputs of the proposed 5D hyper-chaotic system (Ṽ n ) is used to generate the seed for the data scrambler. Then, the other serialized truncated chaotic signals are XORed with the output bit from the data scrambler. Based on the experimental result, m = 6 shift registers, which are implemented by delay units (flip-flops (FFs)) in the Xilinx System Generator, are utilized to obtain high success rates in the statistical tests, . Five random bitstreams (B 1 → B 5 ) are collected for the evaluation phase, which will be presented below.

V. EXPERIMENTAL RESULTS
In this section, we present the deployment of the random bit generator on the Xilinx Zynq-7000 SoC ZC702 Evaluation Kit. The implementation is performed using the Xilinx System Generator tool. The random number generator is generated as an IP-core and evaluated on Xilinx Vivado tools with the default configuration.

A. Comparison and Discussion
We start this section with a comparison with the state-ofthe-art chaotic systems, i.e., a Lorenz system as well as both Elwaki's systems of [27]. To illustrate the advantage of our FFRK implementation, we carried out our own implementation of the systems, in which we produced versions that either integrated the traditional RK implementation or our proposed FFRK method. The implementation is produced on the evaluation kit mentioned above. Four iterations of the mapping functions are used to generate k i in the traditional fourth-order RK method. Fig. 11 shows the amount of FPGA resources required for the various chaotic systems. The Lorenz systems are denoted LRZ-RK and LRZ-FFRK, while the Elwaki's systems are denoted EWK1-RK, EWK2-RK, EWK1-FFRK, and EWK2-FFRK; the RK and FFRK suffixes respectively indicate whether the system uses either the traditional RK implementation or our proposed FFRK implementation. From Fig. 11, it can be seen that compared to traditional RK methods [7], [28], the proposed FFRK requires 4 times less DSPs, under half the number of look-up tables (LUTs), and approximately 15% less FFs. Fig. 11 also includes results for the proposed 5D chaotic system, both with the traditional RK method (denoted 5D-RK) and the proposed FFRK method (denoted 5D-FFRK).  Using the proposed FFRK leads to a significant reduction in the amount of LUTs and DSPs required, approximately the same as that seen for the other systems. There is a small increase, (∼ 25%), in the amount of FFs required. In addition to the above, Table I compares our proposed hyperchaos-based PRNG implementation against the implementations of chaos-based PRNGs from the literature. In this context, we implement our system using the FFRK method using the default configurations for synthesis and implementation strategies. It is noted that changing the synthesis tool optimization policy to favor speed will lie a small improvement. However, the greatest gain will come from duplicating the design along with some straightforward circuitry to merge their results, effectively doubling the throughput at the cost of doubling the amount of required resources without any effect on latency.
In terms of resource requirements, it can be observed that the 3D chaotic system using the fourth-order RK algorithm in [9], which utilized up to 32% of the available resources from the Virtex-6 XC6VLX240T-1-FF1156 FPGA chip, requires more resources than the other systems. Compared to our work, the Grünwald-Letnikov (GL)-based algorithms of [11] require a similar number of FFs and from 3× to 6.4× the number of slices. While the implementation of the 3D-fractional-order chaotic system using the Adomian decomposition method (ADM) from [29] requires a smaller number of slices, the random number generator is only performed in software design. The 3D-chaos based PRNG, which is itself based on Lorenz and Lu chaotic systems from [30] using the Euler method, uses the least number of resources among all systems, but offers an achievable throughput of 1872 Mbps, which is much lower than that of our proposed system. The 4D-chaos based system from [15], which is optimized for throughput, is the one that comes the closest to our system in terms of the number of LUTs, where our implementation achieves 3.6× greater throughput. The system in [31] uses the highest number of resources among all the 3D-chaos based PRNG implementations using the Euler method. We recall that the hardware implementation of the Euler method requires fewer device resources, but achieves less accuracy as compared to the fourth-order Runge-Kutta algorithm [7].
Our design seeks to reach a high throughput, and a high level of randomness while requiring a modest amount of resources. The Euler method implementation provides less accuracy, and therefore the average entropy per bit is decreased and so is the randomness level for the output random bits. We also note that for the targeted FPGA, the proposed system requires under 4% of the available LUTs, FFs, or DSP blocks. The design works at a maximum frequency of 113 MHz, which enables a maximum throughput of 6.78 Gbps for the generator. The estimated power consumption of the random number generator is 73mW@6.78Gbps. Therefore, the estimated energy efficiency is 10.8 pJ/b. The FFRK implementation for the chaotic system has a delay of 65 clock cycles while there is only one latency at the random outputs.

VI. RANDOMNESS PERFORMANCE EVALUATION
In this section, we present multiple tests performed on the harvested binary bitstreams before and after applying post processing. These bitstreams are collected by hardware co-simulation of Vivado System Generator and Matlab. The PRNG core runs on the FPGA board connected to the computer and the output data are collected through JTAG. The experimental setup is illustrated in Fig. 13. The random binary outputs are observed using an oscilloscope.

A. Histogram
First, 100K length of 12b-truncated chaotic signals are collected and used in distribution and histogram analysis. As shown in Fig. 12-a), since there are no obvious patterns in their distribution, 12b-truncated chaotic signals have good randomness characteristics. Fig. 12-b) shows the histogram of the signalX n before post-processing where several peaks that can be observed. The histogram of the output bitstream B 1 after post-processing is illustrated in Fig. 12-c), which indicates a random distribution, where no apparent pattern can be discerned. Similar results are observed for B 2 to B 5 .

B. Standard statistical random testsuites
In order to be used in cryptography applications, the random binary bits should be tested using statistical tests which require long bitstreams. The NIST SP800-22 test-suite, which is developed and introduced by The National Institute of Standards and Technology (NIST), is commonly used [32]. The NIST test results are presented in Table II Slices  12430  5688  2659  -119  ---920  LUTs  43732  --1220  494  1169  1882  1355  2017  FFs  42092  4962  3360 192  118  416  480  1318  3458  DSPs  -99  59  8  8  ---8 Table IV. Most of the tests are passed, except for the multinomial test of bitstream B 3 , with a P-value = 0.00097, which is very close to the threshold (0.001). It should not be understood that if a RNG fails in some statistical tests, it cannot be used in practical problems; even the best commercial RNGs experience failures in very complicated tests [33].

VII. CONCLUSION
We have presented a novel 5D hyperchaotic system along with its implementation on FPGA. In this work, we proposed a novel method, which reduces device resource usage, to implement the fourth-order Runge-Kutta numerical technique to solve differential equations in chaotic systems. The PRNG, which is designed to take advantage of the high dimension and the high accuracy of the hyperchaotic system's hardware implementation, reaches a maximum throughput of 6.78 Gbps.
The design, which is deployed in the Xilinx FPGA chip, occupies under 4% of the available FPGA hardware resources. The random generated bits have been verified in modern statistical tests confirming that our proposed system is suitable for industrial applications. Future work includes integrating the proposed PRNG in real-time communication applications such as the secure streaming of images and videos.