{'fraction_non_alphanumeric': 0.03959742927684276, 'fraction_numerical': 0.015209483782204715, 'mean_word_length': 4.725693320384159, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 1, 'https://': 1, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Development and Validation of an AI-Driven Model for the La Rance Tidal Barrage: A Generalisable Case Study

In this work, an AI-Driven (autonomous) model representation of the La Rance tidal barrage was developed using novel parametrisation and Deep Reinforcement Learning (DRL) techniques. Our model results were validated with experimental measurements, yielding the first Tidal Range Structure (TRS) model validated against a constructed tidal barrage and made available to academics. In order to proper model La Rance, parametrisation methodologies were developed for simulating (i) turbines (in pumping and power generation modes), (ii) transition ramp functions (for opening and closing hydraulic structures) and (iii) equivalent lagoon wetted area. Furthermore, an updated DRL method was implemented for optimising the operation of the hydraulic structures that compose La Rance. The achieved objective of this work was to verify the capabilities of an AI-Driven TRS model to appropriately predict (i) turbine power and (ii) lagoon water level variations. In addition, the observed operational strategy and yearly energy output of our AI-Driven model appeared to be comparable with those reported for the La Rance tidal barrage. The outcomes of this work (developed methodologies and DRL implementations) are generalisable and can be applied to other TRS projects. Furthermore, this work provided insights which allow for more realistic simulation of TRS operation, enabled through our AI-Driven model.


Introduction
The La Rance tidal barrage is considered one of the greatest achievements of tidal power generation since its debut operation in 1966 [1,2,3,4,5,6]. Still in full operation today, this Tidal Range Structure (TRS) showcases an installed capacity of 240 , from its 24 × 10 bulb turbines, generating around 500 ℎ∕ [7], at competitive cost of nuclear or offshore wind sources [8,9]. Following the successful operation of La Rance, other TRS projects have also been constructed, such as the 20 Annapolis Royal Generating Station and the 254 Sihwa Lake Tidal Power Station, in Canada and South Korea, respectively [6,10]. Nowadays, La Rance inspires the conceptualisation of other TRS projects around the globe [11,6,12]. Among these, the UK stands out with several in depth studies of tidal barrages and lagoons in the following bay and estuaries: Colwyn Bay, Solway, Mersey, Loughor, Duddon, Wyre, Thames [13,14,15,16,17,18] and, most notably, the Severn Estuary (e.g. Cardiff, Newport, Fleming and Swansea Bay tidal lagoons, and also the Severn, Cardiff-Weston and Shoots tidal barrages) [19,14,20,21].
While La Rance is a significant example of how TRS can be economically viable, given enough time of operation, novel projects, such as the Swansea Bay tidal lagoon (SBL), have been questioned if they represent good value for money. This concern is exacerbated by the high initial investment costs that TRS require for construction (≈ £1.3 billion for SBL) and the recent reduction of solar and wind payback periods to ≈ 2 − 20 (year range) and ≈ 13 years, respectively (considering tax incentives and a maximum lifespan of 30 years) [22]. These current issues led the UK Government to halt support for the proposed SBL project in 2018 with the argument of high cost of electricity [23]. Nevertheless, in a recent turn of events, the Welsh Government launched a tidal lagoon challenge in Wales to ascertain market engagement in the construction of several TRS projects. A total of 55 companies registered for the challenge, with the winner possibly receiving Welsh Government financial support for designing a pathfinder TRS project [24,25]. These initiatives demonstrate that TRS development in the near future is still very likely.
For estimating the potential energy of TRS, analytical or numerical models (0D through 3D) have been be considered for simulating such systems. While analytical solutions are useful in estimating an upper bound for the available power [26,27], numerical models can aid in optimising the operational time-sequence of the hydraulic structures present in a TRS (e.g. turbine and sluices), while accounting for effects not considered in analytical models (e.g. variable bathymetry, pump operational modes). Each of these approaches have different strengths, weaknesses and unique physical simplifications. For instance, while 0D models can be computationally inexpensive, they are solely based on the simple statement of mass conservation for the impounded lagoon. In contrast, 1D through 3D models utilise the shallow water equations to obtain (in an increasing order of detail and computational time) the water profile and velocity components in and out of the lagoon [28,29,30]. Shallow water models are helpful in assessing environmental impacts of TRS [31,32], while also detecting the sudden high flow-rates developed when turbines start operating -a situation that can abruptly perturb water velocity and elevation (therefore power generation) in the vicinity of hydraulic structures [33]. Due to the characteristic features of the described approaches, 0D models are usually implemented into optimisation routines of TRS, since these often require a huge number of iterations for convergence [34,35,36]. As a next step, 1D to 3D models are employed to perform verification steps of the operational strategies devised from utilising the 0D model. 2D models are typically picked, among 1D and 3D options, given their satisfactory tradeoff between accounting for detailed bathymetry, coastal geography and accurately predicting complex shallow water effects (e.g. funnelling, resonance) [37,29], while not being computationally expensive as 3D models.
While the literature has advanced in developing the aforementioned TRS simulation models, these studies have focused on cross verification only (e.g. 0D against 2D) to attest for their validity [28,38,39,34,36,35,40,41]. This has been the case, since experimental data from constructed TRS are scarce and under the intellectual protection of few companies around the globe. To fulfil this gap in the literature, we develop in this paper an artificial intelligence (AI) driven model representation of the La Rance tidal barrage, utilising an updated Deep Reinforcement Learning (DRL) approach [42] and experimental data from the literature [1,43,44,45]. The development of such model occurs in two stages. First, a series of generalisable methodologies for simulating turbines (in power generation and pumping modes), transition ramp functions (for opening and closing the TRS hydraulic structures) and equivalent lagoon wetted area are developed and assembled into a parametrised 0D model of the La Rance tidal barrage. The 0D La Rance model is then validated in its capabilities of predicting (i) power output and (ii) lagoon water level variations, using measurements from [1] as reference. In a second stage, a novel DRL implementation for optimising the operation of La Rance's hydraulic structures utilising the Unity ML-Agents package, based on [42], is presented and discussed. The operational strategy learned by the DRL agent for controlling the sequence and timing of operation of the hydraulic structures (i.e. the AI-Driven La Rance model) is shown to be closer to the real strategy utilised in La Rance, in comparison over stateof-art approaches proposed by the literature for other TRS [36,34]. Furthermore, annual energy predictions from the AI-Driven model are also compared against measured data from La Rance, showcasing good agreement of results.

Deep Reinforcement Learning
In recent years, DRL techniques have represented a breakthrough for AI methods, performing optimally in several real-time problems that previously could only be solved by experienced human operators. The potential of DRL techniques for solving such tasks has received a significant attention in applications with games, such as Atari, Chess, Shogi, Game of Go and StarCraft II [46,47,48], where smart DRL agents were able to showcase a performance superior to human players. Amid the advancements of DRL algorithms, complex real-life optimisation problems also started to be tackled, such as trading and finance, self-driving cars, healthcare and energy efficiency (e.g. heating, ventilation, air-conditioning and datacentre cooling systems) [49,50,51,52,53]. In the context of renewable energy, reinforcement learning has also been used for optimising the operation of smart grids, wind turbines, solar panels, stream turbines and more recently, TRS [54,55,56,57,58,42].
In general, in order to assess if an operational optimisation problem can be solved through DRL, the former needs to be mapped into a Markov Decision Process (MDP). As shown in [59], the mathematical formalisation of an MDP is composed of (i) an agent capable of actions ( ) and (ii) a reactive environment, that outputs new environmental states ( +1 ) and rewards ( +1 ) based on the received action from the agent. The defined quantities , and are random variables, with well-defined probability distributions. The interaction between agent and environment can be visualised in a general agent-environment MDP framework in Fig. 1.
As is the case with this work, the simulation of the environment (for acquiring +1 and +1 ) can be performed with numerical models [46,47,48]. The employment of numerical models allow for utilising model-free DRL methods, where the MDP's agent learns by experience -sampling states and rewards from the environment for every action . In non-deterministic scenarios, the selection of possible actions by the agent (as a function of input states) is a Figure 1: General agent-environment interaction in an MDP, showcasing the resulting state-action, next reward triples sequence. Edited from [59].
stochastic process known as policy, mathematically described as a conditional probability distribution of the form: With a defined (i) policy (for choosing the agent's actions) and (ii) a numerical model of the environment (modelfree DRL approach), simulation sequences of the MDP can be run ( Fig. 1). At each cycle of the MDP, multiple time-steps = 0, 1, 2, 3... observations ( ) of the agent-environment interaction can be sampled. These are organised as a sequence of state-action, next reward triples: where, , , +1 are instances of the random variables ( , and +1 ). While the sequence of state-action pairs defines a trajectory : the summation of the observed sequence of rewards yield the total return at time-step : where is a discount factor between 0 and 1. The objective of reinforcement learning problems is to find an optimal policy * that maximises the expected return of rewards [ ] conditioned on any initial state, i.e. * = argmax [ | = ], ∀ .
A policy * for the agent can be obtained (i) indirectly, through state-value or action-value functions (e.g. Deep Q-Network) [46] or (ii) directly, with policy based methods (e.g Policy Gradient) [60]. In this work, Proximal Policy Optimisation (PPO) [61] (a Policy Gradient algorithm) is utilised for training the DRL agent. In PPO, a neural network representation (actor neural network) is utilised for parametrising the policy, enabling the sampling actions of as a function of environmental inputs . For the case of discrete actions (as is the case in this work), the neural network outputs the probabilities of each possible action in that state using a softmax layer. A full mathematical description of the PPO algorithm, in the context of TRS application and for both training and test stages, can be found in [42]. The agent-environment MDP modelling and training stages for the La Rance TRS are described in Section 6.2.2.

State of Art TRS Simulation and Operation.
In this section, state of art methods for simulating, operating and parametrising the hydraulic structures that compose a TRS are detailed and discussed. While some of the methods are adopted in this work (e.g. 0D TRS model), others are contrasted against experimental measurements from the La Rance tidal barrage, indicating the need of developing novel approaches that are capable of yielding more realistic results.

0D Simulation
Derived from mass conservation, 0D models correspond to the simplest numerical model representation for TRS, capable of estimating lagoon water motion as a function of total flow rate from turbines and sluices. Since turbine models predict flow-rate and power as a function of water head between ocean and lagoon, 0D models also allow for estimating power production of TRS subjected to water head variations. When the goal is the optimisation of TRS operation for maximising power generation or revenue, 0D models are usually chosen, given their computational efficiency. Furthermore, 0D models have presented good agreement of results with more complex finite-element 2D models when considering "small-scale" projects (e.g. Swansea Bay Tidal Lagoon) [39,34,36,6,62]. Formally, 0D models can be written as: where is the lagoon water level (in meters), is the total directional water flow rate ( 3 ∕ ), predicted for both turbines and sluices using function approximations, and ( ) is the variable lagoon area ( 2 ). From Eq. (6), the lagoon water level at the next time-step ( +1 ) can be numerically calculated by a backward finite difference method: where is the water level at time-step and Δ the discretized time. In this work, Eq. (7) represents the simulation environment for our MDP.

Two-Way with Pumping Operation Scheme
Given a 0D TRS model, state-of-art optimisation routines require a fixed TRS scheme of operation as a basis for optimising the operational time sequence of turbines and sluices. From the literature [34,36,35], two-way with pumping schemes ( . . ) have showcased the best results for maximising either power or revenue generation. . . schemes are characterised by the ability of the TRS to generate power either during the flood or ebb tides, with the possibility of utilising turbines as pumps to increase the water head difference between ocean and impounded lagoon. As discussed in the seminal work of Gibrat from 1955 [63] (prime chief investigator of La Rance Tidal Barrage project), the augmentation of TRS operation with pumping capabilities can increase energy extraction of these systems significantly. Indeed, state-of-art research [34,36] have supported this claim, simulating that a 20 − 40% increase of energy output is expected when implementing pumping in classic two-way ( . ) scheme approaches. However, the literature interpretation of . . schemes is still not aligned with observed measurements of the only case study in the world: La Rance. For instance, in Figs. 2a and 2b, predicted lagoon water level variations for the state-of-art literature interpretation of . . schemes are coloured according to the operational mode chosen for turbines and sluices, respectively, with the sequence of operations for the hydraulic structures being dictated by literature constraints [34,36]. While the logic of operation for turbines and sluices is mostly identical to the observed in La Rance (green, orange and black regimes in Figs. 2a and 2b), literature constraints limit turbine pumping stage to occur only when negative head differences (against gravity) are observed (red regimes in Fig. 2a). In contrast, measurements from the La Rance TRS [1,44], showcase pump operation in both positive and negative water head scenarios, with pump shutoff negative heads (null pump flow rate) up to 6 . Furthermore, state-of-art optimisation routines need to choose to fix either power input or pump flow-rate to reduce computational costs [39] -differently from what is observed in La Rance [1], where fine tuned power input is provided for turbines in pump mode. In Section 6, we show that the operational strategy devised by our trained DRL agent is capable of filling this gap in the literature, operating the 0D La Rance model in a more realistic fashion.

Hydraulic Structures Parametrisation
With a given TRS model and operational scheme, function approximations for simulating turbines and sluices are required. Regarding TRS turbines in pump mode, literature research has been limited. In fact, recent studies have simulated pump operation either by fixing the efficiency (for any head difference) to 70% − 40% [39,64], or by directly following experimental pump efficiency curves from [64]. For negative head differences, the literature has also adopted an idealised pump efficiency interpretation [34] (disregarding efficiency variations with pump rotation), cast as:  where is the electrical power available to the pump and is the rate of work exerted from the pump to the fluid: where is the seawater density (1024 ∕ 3 ), the gravity acceleration (9.81 ∕ 2 ), the pump flow rate and ℎ the negative head surpassed during pumping. From Equations 8 and 9 we obtain: Although Eq. 10 can help estimate pump flow rate, given a negative head and input power, it cannot explain maximum pump shutoff heads (where = 0), or maximum pump flow rates that occur when ℎ = 0. These issues, combined with the fact that turbine specifications (e.g. diameter, capacity, applied power input) were omitted in the [64] study, and that pump efficiencies are only available for negative heads in the small range [≈ .3 , ≈ 1.9 ], indicate the urgent need of more accurate and generalisable TRS pump models. In order to fill this gap, a novel and generalisable TRS pump model is developed in Section 4.5 and applied to our parametrised 0D La Rance model.
Besides "pumping mode", turbine modelling also require functions for simulating "power generation" and "idling" modes of operation. In the literature, turbine hill charts (e.g Sulzer Escher Wyss of Zurich, Andritz [2,15]) have been used for estimating power output of bulb turbines of various diameters. In Section 4.2, parametrisation techniques based on experimental measurements from La Rance are utilised for deriving hill charts for the La Rance tidal barrage. When operating in "idling" mode, turbines act as sluices, aiding the sluicing stage of operation. For estimating flow rates in this mode, the orifice equation has commonly been employed: where flow rate is a function of ℎ (the head difference between ocean and lagoon), is the dimensionless discharge coefficient (greatly dependent on sluice gate design [2]), and the sluice/orifice area. The choice of best values for turbines and sluices for the La Rance case study are presented in Section 4.4. Finally, in order to more accurately simulate the starting and end of operation of the TRS hydraulic structures, a novel momentum ramp function is developed and presented in Section 4.1.

La Rance Tidal Barrage Components Parametrisation
In this section, we propose techniques to reverse engineer the hydraulic structures that compose a real TRS into parametric functions. The available data utilised in this section is mostly obtained from a 1975 study [1], executed in collaboration with Electricité de France (EDF), the company responsible for La Rance operation still to this day. Data from this study are utilised in order to create parametric models for turbines in ebb and flood operation (in both power generation and pump modes), sluices and equivalent impounded wetted area. Furthermore, a novel ramp function, named momentum ramp function, is developed in order to help estimate flow rate and power variations when opening/closing TRS's hydraulic structures.
From [1], the observed variations of lagoon water level and power output/input for two days of observations are digitised, yielding Figs. 3a and 3b. In Fig. 3a, the observed operational modes correspond to a conventional ebb-only generation ( ) scheme, while Fig. 3b showcase a . . scheme. The resolution of the digitised data is represented by a "×" label. The initials for the operational modes that were set for each scheme of operation are also presented at the top of each image. The detailings of each labelled initial, numbered in order of occurrence in Figs. 3a and 3b, are shown in Table 1.  It is worth emphasising that during . at the scheme, turbines are expected to operate alone. This contrasts with the . . scheme, where at the end of . and . , turbines are expected to operate together with sluices (i.e., the variant operation of TRS described in [42]) [2,1].
Design specifications for La Rance, taken from [1], are shown in Table 2. Also, [1,43] present data showcasing turbine efficiency as a function of water head ℎ (for . and . modes of operation). From these data, 2 order approximations are derived and shown in Figs. 4a and 4b. Furthermore, data presenting the expected pump flow rates as a function of ℎ for La Rance's bulb turbine in . and . operational modes, with fixed (maximum) power input of = 6 , are also available in Table 3. From these data, 2 order approximations are derived and shown in Fig. 5. Following Table 3 and Fig. 4, we note that pump shutoff head is ℎ = −6 for both . and . modes, and that maximum flow rates, when ℎ = 0 , measure = 252.2 3 ∕ and = 215.8 3 ∕ ,   ( 2 ) 900 (a) . : respectively.
[1] also provides various estimates for the volume ( ) of water stored in the La Rance estuary as ocean tide fluctuates to a height , defined from the lowest tidal level in the equinoctial low tide ( = 0), shown in Table 4.
Finally, an upper bound for the maximum turbine flow rate (for any mode of operation) is set to 280 3 ∕ , utilising site measurements from La Rance as reference [45]. In the following sections, the data presented will be utilised to parametrise all elements that compose the 0D La Rance model. As a first step, since sluices and turbines (in both power generation and pump modes) require a ramp function to be simulated, a novel momentum ramp function for the hydraulic structures is presented in the next section. Table 3 Measured . and . flow rates at La Rance, for a fixed power input = 6 (from [1,44]).

Momentum Ramp Function
In order to model flow rate variations from opening/closing the hydraulic structures that compose a TRS, a ramp function is required. Given that solutions from the literature try to solve this problem with a heuristic approach [65,34,36], we propose a new derivation for a ramp function more grounded on physical principles. In order to do so, we utilise an electro-hydraulic analogy of a direct current circuit, where the circuit's inductance ( ) and resistance ( ) are analogous to the hydraulic inertance and resistance, respectively. These types of analogies have been extensively used in the literature, with applications for both pipe fluid flow and open channel flow [66,67]. The derived ramp function is an expansion of the heuristic momentum ramp function presented in [42]. A complete derivation of the proposed ramp function is presented in the Supplementary Material.
The obtained numerical form of the momentum ramp function is presented as: where ( + 1) represents the flow rate at the next time-step, ( ) the flow rate at the present time and controls the intensity of flow-rate updates. The term represents the steady-state flow rate estimate for the hydraulic structure as a function of head difference. As an example: for sluices, can be obtained from the orifice equation (Eq. 11), while for turbines in power generation mode, can be obtained from parametrised turbine hill charts. Furthermore, since in steady-state regime power output relates linearly to turbine flow rate, the numerical form of the momentum ramp function can also be applied to predict the power output evolution: where ( + 1) is the power output at the next time-step and ( ) the power output at the present time. The term represents the steady-state power output estimate for the turbine as a function of head, i.e. a turbine hill chart estimate. The numerical form of the momentum ramp function in Eq. (12) simplifies the very complex phenomena of opening/closing the turbines and sluices that compose a TRS. Indeed, beyond the hydraulic resistance and inertance of these systems, the opening/closing of turbines and sluices also involve adjusting the pitching of runner blades/guide vanes (for turbines) and aperture of gates (for sluices). Nevertheless, we show in following sections that the developed ramp function can accurately help estimate power output and flow rate evolution for both starting/closing stages of turbines and sluices, given appropriate values. As a lower bound for estimates (for both sluices and turbines), we utilise the value = 1.091 , which guarantees a precision of 10 6 for the complete opening/closing of hydraulic structures in a 15 time interval (analogous to the tuned ramp function in [42]).

Turbines -Power Generation Mode
Given power production and water level variations for the ocean and impounded lagoon (for the numbered . and . turbine modes of operation in Fig 3), we can draw interpolated (parametric) curves for the turbine power output as a function of head, with time as hidden parameter: where " " corresponds to the th occurrence of . and . turbine modes. The interpolated ( ) and ( ) curves, for every " " are shown as solid curves in Figs. 6a, 6b, 6c, 6d, 6e and Figs. 7a, 7b, respectively. These figures showcase power production as a function of head difference, where time evolution is presented by labels over the interpolation. Considering scenarios where the starting phase for the turbines is available ( and ), the interpolated results show a two-step process where initially (i) power outputs rapidly increases from a starting non-zero head difference until reaching a plateau (maximum head difference and power output). (ii) Then, from this plateau, water head variations, therefore power generation, slowly decrease with time, until power production is ceased. We interpret the first process as non-steady acceleration stage where the turbine, initially at rest, is submitted to a starting operational water head , accelerating until maximum power generation is achieved. Conversely, the second process is interpreted as a quasi steady-state deceleration stage, where the turbine is submitted to a slowly decreasing water head, taking ≳ 3 hours to reach . With this interpretation, we take the average of all available quasi steady-state phases for . and . modes, considering power output to be a pure function of head difference "ℎ", thus obtaining hill charts ( (ℎ), (ℎ)) for both modes of operation. A comparison of our parametrised hill charts against interpolated hill charts from [1] is shown in Figs. 6f and 7c, for . and . modes of operation, respectively. The differences observed between these hill charts could be due to a series of factors not available to us, such as the pitching of runner blades/guide vanes. Nevertheless, proceeding with our parametrised hill charts enabled a satisfactory 0D simulation of turbines, as will be shown in the following sections.
In order to simulate the smooth power output evolution for both accelerating and decelerating stages of turbine operation, the momentum ramp function, presented in Section 4.1, is utilised. A sketch showcasing the result of augmenting the parametrised hill chart with the momentum ramp function can be seen in Fig. 8.
For best adjusting the predicted power output (from Eq. (13)) as a function of water head, best values for accelerating and decelerating stages of . and . modes of operation ( , and , respectively) are found by minimising the Sum of Squares Residuals (Eq. 17) between each interpolated hill chart ( ( ), ( )) and the parametrised hill chart ( (ℎ), (ℎ)) augmented with the momentum ramp function (Eq. 15,16). It is worth noting that, when minimising the residuals, the water head input for both charts is also a function of time.

Equivalent Lagoon Wetted Area
So far in the literature, and to the best of our knowledge, 0D numerical simulations of TRS have utilised bathymetric data for estimating a "flat" lagoon wetted area for TRS. Although this technique can enable accurate simulation of TRS for "small-scale" systems, such as the SBL, as the impounded area and length increases (like in Severn barrage), this technique starts presenting significant deviations when compared to more realistic 2D models [39].
In this section, we showcase a simple methodology that can be applied to any coastal region to extract an equivalent lagoon wetted area representation. This methodology was applied in our La Rance case study, since bathymetric data for the barrage was not available. In fact, knowing the real impounded area of La Rance is even more complicated, because, like any estuary, La Rance presents a severe accumulation of silt and sand due to siltation (a natural phenomenon due to sedimentation), amounting to approximately 50, 000 3 of sediment deposited each year in the estuary [68,69] -altering La Rance´s bathymetry. In Section 5 we show that the method proposed here enabled an accurate prediction of lagoon water level evolution when operating turbines (in power generation and pump modes) and sluices, even though the geographical maximum lagoon wetted area spans 22 2 (twice that of SBL) and, given its narrow 720 barrage, has an approximate length > 30 [5]. As shown in [27], analytical estimates for the available energy in a TRS can utilise the impounded volume of water instead of bathymetric representations. In a similar fashion, we utilise the stored water volume data shown in Table 4, to derive an equivalent lagoon wetted area for the La Rance 0D model. In the literature, this equivalent area is attained from the inter-tidal prism volume estimate, which have been used to estimate the impounded water volume of rivers that end in tidal regions [70]. In its simplest form, a constant wetted area is used, so the stored volume of water Δ is proportional to variations in tidal level (tidal range) [70]. We note that this simplified model yielded fairly good results when applied to our La Rance 0D model. However, considering that there are several model approximations that can be used to estimate as a function of tidal level variations [71], we proceed our study with a linear approximation for (which gave improved results). For the sake of simplicity, we use the tidal datum (height from the lowest equinoctial low tide, defined as = 0), instead of the tidal prism. So that, where 0 is the equivalent wetted area at = 0 and 2 the slope. From this approximation, the stored volume (Δ = − 0 ) at a tidal datum , obtained by integrating Eq. 18 from 0 to , becomes a second order polynomial: Now the coefficients and 0 can be estimated with a least square method, utilising the point measurements ( , Δ ) from Table 4, as a target. The obtained quadratic fit for Δ is shown in Fig. 9, together with found values for and 0 . With these parameters, the wetted area given by Eq. 18 is also plotted and shown as an insert in Fig. 9.

Sluicing
For parametrising the sluicing stages shown in Fig. 3a and 3b, we need to define appropriate (i) values for our momentum ramp function (Eq. 12) and (ii) discharge coefficient ( ) for the orifice equation (Eq. 11), responsible for estimating flow rates from the hydraulic structures. In contrast with turbines (in power generation mode) that have accelerating and decelerating stages (Section 4.2), sluices allow for the free passage of water with minimum resistance, independent if head differences are increasing or decreasing. Due to this symmetric behaviour, we assume a constant value for our momentum ramp function estimate. Since we do not have measured data to fit a best value for sluices, we adopt the lower bound = 1.091 . Beyond sluice gates, the sluicing stages shown in Fig. 3a and 3b, need also to account for turbine flow rates operating in "idling" mode. For simplification purposes, we assume that turbine operation in idling mode have a similar behaviour to sluices, since there is no electrical energy conversion. Therefore, the same = 1.091 is used. This similar behaviour is also supported by the literature, since flow rate estimates for sluices and idling turbines utilise the same "orifice" equation [28,17,30].
For best fitting Eq. 11 to La Rance, appropriate and , for sluices and idling turbines, respectively, need to be found. For sluice gates, we bound the estimate between two experimental values from the literature. As a lower bound, we have = 1, from experimental results of a large sluice gate experiment by [72]. As an upper bound, we utilise measurements from [1], which indicate that, at its maximum, sluice gates from La Rance provide a flow rate of 9600 3 ∕ . Since the total sluice gate area for La Rance is 900 2 , we attain an upper bound of = 1.077. Similarly, idling turbines flow rate is bound by experimental measurements, at La Rance, from [45]. In this work, flow rate measurements from a turbine unit under a fixed 4 head was recorded in the range 182.2 3 ∕ to 280 3 ∕ . From this data, upper and lower bound discharge coefficients for turbines and sluices are defined as: For choosing best and values within bounds (Eq. 20), we utilise the 0D model for La Rance (Eq. 7), with the equivalent lagoon wetted area derived in the previous section, to predict lagoon water level variations during each sluicing stage " ". The predicted levelŝ are compared with measured data and a time-normalised sum of squared residuals (NSSD) is calculated: The best and that minimise for all sluicing stages are = 1.017 and = .967. However, since adopting = = 1 does not change results significantly, for the sake of simplification, we resume our work with a discharge coefficient of "1", for both sluices and idling turbines. Comparison between̂ and results, with = = 1, are shown in Fig. 10.

Turbines -Pumping Mode
As shown in Section 3, current idealised pump models for TRS (Eq. 10) fail to explain expected behaviours such as maximum pump shutoff head ℎ (when = 0), maximum pump flow rates (for ℎ = 0) and pump operation under positive head scenarios (aided by gravity) -all observed in La Rance measurements [1]. For contemplating these behaviours, we utilise pump affinity laws in order to derive a general formulation for the pump flow rate as a function of head difference and applied power input: = (ℎ , ). The affinity laws express the mathematical relationships between several variables involved in the performance of kinetic pumps (centrifugal or axial). These laws show that, under dynamically similar conditions, dimensionless parameters remain constant. They are useful for predicting pump performance changes when varying either (i) pump operational speed or (ii) pump impeller diameter [73]. Since bulb turbines operating in reverse have a similar behaviour to axial-flow pumps [74], we continue this section with the assumption that affinity laws can be applied to bulb turbines in pump mode.
Each affinity law postulate can be expressed as a set of three equations, where pump flow rate , negative head ℎ and input power are estimated for a point " 2" using a known point " 1" as reference. Then, following the affinity laws, values for " 2" are obtained as a function of either pump rotation ( ) or pump impeller diameter ( ), following the first or second postulate from the affinity laws, respectively. From the first postulate [73]: For the relationship shown in Eq. 22 to be true, pump efficiency must remain relatively constant as we move from " 1" to " 2", for a fixed diameter pump [73]. Utilising the first postulate of pump affinity laws (and assuming a near constant efficiency for any change in rotational speed) is sufficient for estimating a general pump equation for La Rance. The first postulate can also be manipulated, so that and ℎ are obtained as a function of power input variation: For both . and . operational modes, the second order approximation for flow rate (ℎ , ) as a function of head for a fixed (maximum) power input of = 6 , shown in Fig. 5, is utilised. For simplification purposes, { = 6 } = 6 . Also, since pump operational head ℎ depends on power input (Eq. 24), we have for the quadratic approximation (ℎ ( 6 ), = 6 ) = (ℎ ( 6 )), so that: Eq. 25 expresses a characteristic pump curve, where ( 6 ) is the maximum pump flow rate expected, for ℎ = 0 and = 6 . Knowing that is zero at the shutoff head (ℎ ), the quadratic approximation can be expressed in terms of its roots, yielding a convenient form: For deriving a characteristic pump curve for any power input ( ) and pump head (ℎ ( )), we expand all terms in Eq. 26, by utilising Eq. 24 and Eq. 23, thus obtaining Eq. 27. For convenience, (6 ∕ ) = .
(ℎ ( 6 )) = (ℎ ( )) 1∕3 , Performing the substitution of terms from Eq. 27 into Eq. 26, returns Eq. 28 allows for estimating pump flow rates for any given ℎ and , including regions of pump shutoff head ℎ and maximum pump flow rate . Finally, in order to estimate pump flow rates for positive head scenarios, we assume that the maximum attained for ℎ = 0 is summed with the gravitational flow rate estimate (orifice equation for turbines) shown in Section 4.4. The maximum flow rate allowed during pumping (upper bound " ") is = 280 3 ∕ , as defined in Section 4. Examples of predicted pump flow rates, for . and . modes of operation are shown in Fig.  11. For verifying the quality of our pump (ℎ , ) estimate (Eq. 28), we utilise the 0D model for La Rance (Eq. 6), with the equivalent lagoon wetted area derived in Section 4.3, to predict lagoon water level variationŝ during each . and . pumping stage. The predicted̂ values, for a given (measured) power input and varying ocean levels, are compared with measured lagoon water levels and shown in Fig. 12. As shown in Fig. 12, the derived pump model for La Rance presented good agreement of results when predicting lagoon water level variations against measured data. These results were obtained considering a constant (lower bound) value for the momentum ramp function = 1.091 . The required steps to generalise the developed pump equation for bulb turbines of various diameters and power capacity are presented in the Supplementary Material.

Validation of the Parametrised 0D La Rance Model
With the developed parametric models for turbines (in power generation and pump modes), sluices, equivalent lagoon area and momentum ramp function, we can now verify the accuracy of our 0D La Rance model in predicting lagoon water level variations and power output.
Utilising Fig. 3 as reference, we set initial lagoon water levelŝ and turbine output power̂ (ℎ, , ) to the same initial measured values ( and , respectively) at La Rance. Henceforth, we vary the operational mode for the hydraulic structures following the same timing as in Figs. 3a and 3b. When operating turbines in pump mode, the measured is applied to our pump model, so that pump flow rates can be predicted. A comparison of the 0D La Rance model predictions with measured data is shown in Fig. 13.
As noted in Section 4, sluice operation is expected at the end of power generation stages for the . . scheme (Figs. 13b, 13d and 13f). Since the timing for starting sluice operation was not provided by [1], the showcased results for the . . scheme simulation assume a best fit between predicted and measured lagoon water levels. Nevertheless, the agreement between predicted and measured power outputs for both and . . schemes is shown to be satisfactory. Indeed, by integrating predicted and measured power, energy deviation is only 4.7% and 2.1%, for and . . schemes comparisons, respectively.
While results in Fig. 13 aim to validate the parametrisation techniques applied in reverse engineering La Rance into a 0D model, an optimal and comparable strategy for the control sequence of turbines and sluices is still required. In Section 6, as a second validation step, we show that our trained DRL-Agent is capable of operating the 0D La Rance model with such strategy.

Operational Optimisation of the AI-Driven La Rance Model
In this section a DRL agent (modelled with Unity ML-Agents) is trained to operate the parametrised 0D model representation of the La Rance tidal barrage. We show that the obtained AI-Driven La Rance model is able to achieve an operational strategy that is comparable in (i) energy extraction capabilities and (ii) scheme of operation to the actual strategy utilised in La Rance. For our comparison analysis, we utilise measurements of the yearly net energy measured in [75] and the observed sequence of operation of hydraulic structures from [1]. It is important to highlight that the actual operation in La Rance, in contrast with the AI-Driven strategy, has the objective of maximising revenue instead of energy [76]. However, as noted in [35], deviations in energy extraction when comparing revenue and energy based TRS optimisation are expected to be around 4−5% only. Therefore, the comparisons we are showcasing are technically sound. In order to train and test our DRL agent, representative tidal data at the location of La Rance, are required. In the next section, we present a free software utilised for tidal prediction that can provide such data.  : Comparison between predicted̂ (yellow solid curves) and measured (×) lagoon water level variations during pumping stages ( , , ), along with measured ocean water levels (blue solid curves) and power input (red solid curves and black crosses in , and ), used as inputs for the 0D La Rance model.

JTides Training Data Validation
JTides is a free, worldwide, tidal and current prediction software that utilises harmonic decomposition techniques for predicting ocean tides in several locations of the planet [77]. Although JTides has been utilised for research [78,79], its application in the field of tidal power has not yet been explored.    from La Rance and, as the latter, experiences one of the highest tidal ranges on the planet (up to 10 [80]). In order to assess if tidal predictions from St. Helier are appropriate for La Rance, tidal predictions provided by EDF, the company responsible for operating La Rance, are utilised as reference. The tidal predictions from EDF were available through the web-page (https://www.edf.fr/usinemaremotrice-rance/marees-en-rance), being updated every week, providing 3 weeks of forecast. A whole year of tidal predictions were collected manually, for comparison with JTides predictions. By comparing JTides and EDF predictions, we note that results have the same pattern, although with a small deviation at tidal range's extremes, with tidal predictions by EDF consistently predicting higher tidal amplitudes. By assuming that EDF and JTide's waves are similar, a deviation coefficient between tidal predictions can be obtained through the simple method of root mean square differences (RMS) [81]. With this assumption, we can estimate a correction factor to be applied to JTides's prediction ( × ), in order to reduce the deviation between JTides and EDF estimates: where and are the ocean predictions for JTides and EDF (oscillating around mean water level), respectively, and is the number of data points for a whole year. With this method, a ≈ 1.10 was obtained. A comparison of tidal prediction elevations from EDF and × is showcased in Fig. 14, for one year round. From the observed residuals, we see that the agreement between × and EDF is consistent throughout the year, apart from very few isolated spikes. Therefore, we resume our work with the assumption that × tidal predictions are representative of expected ocean water levels at La Rance. With this assumption, we utilise the × software capabilities to generate training data (from 2013 until 2038) for our DRL agent.

Unity 3D & Unity ML-Agents
The Unity3D graphics engine is a popular game developing environment that has been used to create games and simulations in 2D and 3D since its debut in 2005. It has received widespread adoption in other areas as well, such as architecture, engineering and construction [82].
Unity ML-Agents is an open-source project that allows for designing environments where a smart agent can learn through interactions [82]. It has been chosen in this project due to ease of implementation, built-in PPO algorithm and visual framework for visualising real-time control of TRS.

Agent-Environment MDP Modelling and Training
To formalise La Rance operation as a DRL problem (and solve the DRL problem through PPO) we need to design an MDP in Unity ML-Agents with environment, agent, actions, states and reward components (Section 2).
For the environment component (as in [42]), simple representative 3D models for turbines, sluices, ocean and lagoon are created in Unity3D and then imported to a Unity ML-Agents project. Together, these 3D elements visually compose the training environment for our MDP. In this environment, the equations for simulating turbines (in power generation, pumping and idling modes), sluices and equivalent lagoon wetted area are extracted from Section 4. For both training and test stages × outputs are used for our ocean component, while lagoon water level motion is dictated by Eq. 7. The developed 3D representations of turbines and sluices change colours according to the operational mode chosen by the agent. For the turbine, green represents power generation mode, orange -idling mode, blackoffline mode (zero flow rate) and red -pumping mode. Similarly, sluices change colour between orange and black for sluicing and offline modes, respectively. Fig. 15 shows a capture of the Unity ML-Agents MDP environment representation of the parametrised 0D model of the La Rance tidal barrage during . , with the representative models for sluice and turbines in idling and pump generation modes, respectively. Ocean and Lagoon surface level motion are also represented.
As in [42] the DRL agent (actor neural network) is responsible for controlling turbine and sluice operational modes through node outputs , according to a vector of input states . In this case study however, a discrete control solution was adopted when utilising the PPO algorithm, which allowed for the agent to learn a policy that uses turbines in pump mode -in contrast with continuous control solution that ignored turbine pumping capabilities completely. The control solution outputs 3 branches in the last layer of the policy network. The first branch is responsible for controlling sluices, while the second branch is responsible for setting the turbine operational mode. Finally, the third branch controls the input power available to turbines in pump mode (Fig. 16). Each branch ramifies into possible discrete actions that can be taken (i.e. a probability mass function), allowing for the full range of combinations between turbines and sluices to be explored during training. The possible actions for each branch are showcased in Table 5, while the required input states for the policy neural network are presented in Table 6.
As shown in Table 5, values have been discretised, with an upper bound of 4 , for each bulb unit. This upper bound has been selected, given that initial training sessions for La Rance never surpassed 4 power input,   for each of the 24 bulb units available (Table 2). Furthermore, as discussed in [1], even though each unit can receive 6 of power input, optimal results were obtained with smaller values, with pumping sometimes being restricted to 50 (for all 24 units), depending on tidal range [1]. Parallel training is performed with 64 copies of the environment in Unity and observing the average monthly energy reward obtained. By starting with the initial hyperparameter settings from the optimised model in [42] as reference, we note a significant improvement of results when increasing the number of units and hidden layers of the policy neural network. After fixing the complexity of the neural network, converging to a stable plateau was possible by tuning hyperparameters and for multiple runs (Fig. 17), with being the learning rate and a linear decaying hyperparameter responsible for controlling how much the agent explores the environment during training (i.e., increasing leads to more random actions at the beginning of training) [42]. From initial = 0.038 and = 0.0001,

Agent Performance Evaluation
With our trained DRL agent, we proceed to compare the yearly energy extraction capabilities of our real-time AI-Driven model against measured data by EDF [75]. In order to do so, tidal predictions for the reference years in [75] are produced with × and used as inputs for the AI-Driven La Rance model. A comparison between the predicted and measured yearly energy generated by La Rance is shown in Fig. 18, where we can see a satisfactory agreement of results, with La Rance's AI-Driven predictions and EDF measurements averaging 521. 5 and 507.4 , respectively. The 2.6% average gain from the DRL operation is expected, given that energy oriented optimisation schemes have been shown to attain up to 5% more energy than revenue oriented optimisation strategies [35].
From the collected data provided by the AI-Driven model, we observe that the resulting optimal policy chooses to operate La Rance with a . . scheme strategy, independently of tidal range (Fig. 19). The obtained optimal policy consistently starts turbine pump mode when a positive pump head is still available (Figs. 19a and 19b), which is (i) not possible for state-of-art methods and (ii) the same strategy adopted in the actual operation of La Rance (Fig. 3b). Also exclusive to the AI-Driven model is its capability of fine-tuning power input for turbines in pump mode (Figs. 19c and 19d), in a similar manner to what is observed in La Rance (Fig. 3b). Furthermore, we note that for tidal ranges above ≈ 3 , turbine idling mode is not utilised by the agent (i.e. turbine in power generation mode directly switch to pumping mode), indicating the DRL agent capability of adjusting its strategy according to the observed tidal range.
Finally, by observing sluice operation in Fig. 20, we note that sluices are operated independently from turbines, characterising the variant operation of TRS (shown to be superior to the classical operation of TRS when the goal is maximising power generation [42]).
A comparison of the predicted and measured operation of La Rance, for the same measured tide and starting lagoon water levels from [1], is showcased in Fig. 21. From the results, we see how the strategy and predicted lagoon water

Conclusion
In this work, an artificial intelligence (AI) driven representation of a constructed TRS was developed using a 0D TRS parametrised model operated through DRL techniques. Both the 0D TRS model and the AI-Driven representation were then validated, with model predictions showcasing good agreement of results with measured data. For our case study, we utilised the La Rance tidal barrage -the oldest and most successful TRS ever constructed.
The developed methodologies for reverse engineering the hydraulic structures of La Rance (assembled into the 0D TRS model) are generalisable and can be applied to other constructed TRS. Furthermore, novel representations for lagoon wetted area, turbines in pump mode and momentum ramp functions can be applied to future TRS projects (e.g. Swansea Bay Tidal lagoon).
Once trained with the goal of maximising the net energy output, the DRL agent extracted as much energy from the 0D TRS model as reported by La Rance's measurements for a series of analysed years (with an average gain of 2.6%). For acquiring such results, the optimal control strategy devised by the trained DRL agent used (i) real-time control of hydraulic structures (ii) fine tuned power input for turbines in pump mode, (iii) pumping with positive head differences (aided by gravity) and (iv) independent operation of sluices. Apart from the latter, these abilities are unique to the developed method (in contrast with constrained state-of-art operation optimisation methods), allowing the AI-Driven representation of La Rance to showcase results which are more realistic than TRS model simulations presented in the current literature.
The broad applicability of the proposed method shall help to develop new projects with more realistic optimised operation of TRS at a moment where renewable and clean sources, such as tidal power, become more attractive to mitigate climate change, while maintaining the possibilities of economic growth at low environmental impact.   : Lagoon water level variations and predicted flow rate coloured according to sluice operation from the AI-Driven La Rance model. In ( ), ocean is coloured in blue. In both images, background is coloured following turbine operation, with green and red representing power generation and pumping modes, respectively.
(a) Measured and predicted lagoon water levels for the same real ocean signal.
(b) Measured and predicted power output/input for the same real ocean signal. Figure 21: Comparisons between measured lagoon water level variations and power output/input from La Rance's . . scheme operation against learned operational strategy by the AI-Driven La Rance model.