{'fraction_non_alphanumeric': 0.04094184193396332, 'fraction_numerical': 0.021032075598751993, 'mean_word_length': 4.562741915345237, 'pattern_counts': {'":': 1, '<': 5, '<?xml version=': 0, '>': 4, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Deep Learning Algorithms for Hedging with Frictions

This work studies the deep learning-based numerical algorithms for optimal hedging problems in markets with general convex transaction costs on the trading rates, focusing on their scalability of trading time horizon. Based on the comparison results of the FBSDE solver by Han, Jentzen, and E (2018) and the Deep Hedging algorithm by Buehler, Gonon, Teichmann, and Wood (2019), we propose a Stable Transfer Hedging (ST-Hedging) algorithm, to aggregate the convenience of the leading-order approximation formulas and the accuracy of the deep learning-based algorithms. Our ST-Hedging algorithm achieves the same state-of-the-art performance in short and moderately long time horizon as FBSDE solver and Deep Hedging, and generalize well to long time horizon when previous algorithms become suboptimal. With the transfer learning technique, ST-Hedging drastically reduce the training time, and shows great scalability to high-dimensional settings. This opens up new possibilities in model-based deep learning algorithms in economics, finance, and operational research, which takes advantages of the domain expert knowledge and the accuracy of the learning-based methods.


Introduction
As observed in many empirical papers, markets are imperfect, meaning that arbitrary quantities cannot be traded immediately at the quoted market price because of taxes, regulations, and the limited liquidity of the assets. Typical examples include linear transaction taxes as well as fixed transaction costs. As reported and studied in [3,4,6,42], empirical estimates of actual transaction costs typically correspond to a 3/2-th power of the order flow. Accordingly, the large trading volume quickly impacts the market liquidity, which, in turn, drastically changes the agents' behaviors. Hence, optimally scheduling the order flow in anticipation of market liquidity shortage is crucial.
There is a large body of literature on optimal execution strategies as well as dynamic portfolio optimization models with market illiquidity and price impacts, see, e.g. [7,20,22,38,41,43,54], and with recent results in [9,24,31,55]. By assuming the transaction costs as a quadratic function of the size of the order flow, the optimal trading policy can be given in a closed-form, as shown in [24,41]. However, no closed-form solution is available under general nonlinear transaction costs with general market dynamics. To obtain tractable results, researchers then focus on the small costs limit, as in [5,10,31,40,46,54,55]. These elegant asymptotic formulas were proved rigorously by [2,35,39,54]. Two important follow-up questions therefore arise. The first one is about the quality of the leading-order approximation, which needs to be well-studied and quantified, especially in empirical examples. The second is about the smallness assumption, whether that is an absolute or a relative quantity.
From the numerical perspectives, recent advances in highly accurate machine learning models have introduced powerful new tools for studying high-dimensional optimization problems, such as hedging with frictions. The FBSDE solver, developed by Han, Jentzen, and E in [32], can solve a dynamic portfolio optimization problem by finding the solution to a BSDE system. For the first time, this algorithm overcame the curse of dimensionality in numerical solutions to highdimensional SDE and associated PDE, as pointed out by [11,28]. The convergence analysis is established by [33] under the same short-term existence assumptions in [21]. At a higher level, the FBSDE solvers find the solution of the FBSDEs through a supervised learning framework, since the accuracy of the terminal value of the backward components is served as the goal functional of the algorithm. However, this algorithm does not scale well with the trading time horizon and the time discretization. For calibrated trading parameters as in [26], the time horizon for the algorithm to work is required to be unreasonably small. In the meantime, with the development of modern modelfree techniques, reinforcement learning algorithms are also widely used in single-agent optimization problems. Indeed, as shown in the groundbreaking Deep Hedging algorithm [14,15], and similar works in the same flavor [12,16,34,36,37,44,45,48,50,51], we treat the utility functions as targets and directly parametrize and learn the optimal trading policy. Moreover, reinforcement learning frameworks are introduced and analyzed rigorously in [57,58]. Therefore, just like in [45], a natural question is how these methods compare in practice, and our goal is to understand and compare the two types of methods, especially the advantages and drawbacks of supervised learning and reinforcement learning algorithms.
This paper studies the optimal hedging problem in frictional markets in a general setting and examine the machine learning-based numerical algorithms. First, we explore popular machine learning architectures, including the FBSDE solver and the Deep Hedging algorithms. We implement both methods, document the tuning procedures, and discuss their advantages and disadvantages. Then we compare our numerical results with the leading-order approximation formulas. These comparisons help justify the accuracy of the approximation trading formulas and provide us with ideas on the usage of each method in practice. In summary, the FBSDE solver only performs well under short trading horizons, while it fails to work in intermediate trading horizons. The Deep Hedging algorithm has stable and reliable performance for short and moderately long trading horizons, but it does not scale well when the trading horizon becomes long. In contrast, the leading-order approximations work well under long trading horizons, while it deviates from the ground truth by a significant amount shortly before maturity. To fully utilize the convenience of the leading-order formulas and the accuracy of the learning-based algorithms, we employ the transfer learning ideas and propose a new algorithm which we referred to as Stable Transfer Hedging (ST-Hedging) algorithm. ST-Hedging algorithm achieves the same state-of-the-art performance in short time horizons by the FBSDE solver and in intermediate time horizons by the Deep Hedging algorithms, and it allows us to directly work with long trading horizons. ST-Hedging also shows great scalability for multiple stocks with correlations. In addition, the convenience of implementation and convergence speed of ST-Hedging algorithm outperforms the FBSDE solver and the Deep Hedging algorithms, which shows the great potential of model-based learning algorithms.
The rest of the paper is organized as follows. First, in Section 2, we introduce the market model with frictions, the preference for individual agents, and the admissible strategies. Then, the FBSDE solver, Deep Hedging and ST-Hedging algorithms are introduced in Section 3, with details on the implementations and comparisons. Finally, we compare the performance of different learning-based algorithms and the leading-order approximation formula in Section 4. For better readability, the derivation of leading-order approximations and all proofs are collected in Appendix A.
Notation. We fix a filtered probability space (Ω, F , (F t ) t∈[0,T ] , P) with finite time horizon T > 0, where the filtration is generated by a 1-dimensional standard Brownian motion W = (W t ) t∈[0,T ] .

Model Setup
Here, we assume the randomness in the market is generated by the 1-dimensional Brownian motion (W t ) t∈[0,T ] . Suppose we receive a cumulative random endowment as dζ t = ξ t dW t , for a general process ξ ∈ H 2 .
There are two assets in the market for us to hedge the risk, the first one is safe, with price normalized to one, and the second is risky, with general dynamic where the expected return process satisfies the no-arbitrage condition µ = σκ for a market price of risk κ ∈ H 2 . One readily verifies that the Bachelier models and Geometric Brownian motions satisfy this requirement.
Transaction costs. A popular class of models originating from the optimal execution literature focuses on absolutely continuous trading strategies, cf. [3,4], and penalizes the trading rateφ t = dϕ t /dt with an instantaneous transaction cost λ t G(φ t ). With proper definition of matrix-valued operation, this formulation of transaction costs can be generalized to multiple risky assets with cross-asset costs.
G is used to model the "shape" of transaction costs. Portfolio choice problems for the most tractable quadratic specification G(x) = x 2 /2 are analyzed in models by [5,13,25,30,46,52]. In the small transaction costs limit as in [10,18,31], single-agent models are solved explicitly for the more general power costs G(x) = |x| q /q, q ∈ (1, 2] proposed by [3]. Below, we introduce the general smooth convex cost functions G as studied in [26,29], which includes the special examples mentioned above: Assumption 2.1. (i) The transaction cost G : R → R + is convex, symmetric, and strictly increasing on [0, ∞), differentiable on [0, ∞), and satisfies G(0) = 0; (ii) The derivative G is also strictly increasing and differentiable on (0, ∞) with G (0) = 0; (iii) There exist constants C > 0, K ≥ 2 and x 0 > 0 such that The power costs and their linear combinations are included, and the proportional costs can also be studied as the singular limit of power costs with the power approaching 1. The process λ models the level of the transaction costs parameter. Like in the partial-equilibrium model of [46], we allow the transaction cost to fluctuate randomly over time: Here, the constant λ > 0 modulates the magnitude of the cost (this scaling parameter will be sent to zero in the small-cost asymptotics, where explicit formula is available as in [31,17]. We restate this results in Appendix A.1). The strictly positive stochastic processes (Λ t ) t∈[0,T ] describes the fluctuations of liquidity over time, and allows to model "liqudity risk" as in [1], for example.

Preferences and Admissible Strategies.
In order to capture the risk-aversion, we consider the linear-quadratic model, i.e., we maximize our expected returns while penalizing for the corresponding quadratic variations: As studied in [19,24,52], we trade off expected returns against the tracking error relative to the exogenous target position −ξ/σ. We can also consider more sophisticated preferences such as exponential or power utility. Notice that the deep reinforcement learning algorithms such as Deep Hedging, can be easily generalized to sophisticated preferences or utility functions, but the usage of FBSDE solvers is limited since there might not exist a FBSDE system to describe the optimal solution of the hedging problem. See Section 4.4 for details.
To make sure all terms are well defined, we focus on admissible strategies that satisfy the following integrability conditions: Moreover, we impose the transversality condition to exclude Ponzi scheme, by ruling out arbitrarily large risky positions: By strict convexity of the goal functional (2.3), optimality of a trading rateφ is equivalent to the first-order condition that the Gateaux derivative lim ρ→0 (J T (φ + ρη) − J T (φ))/ρ vanishes for any admissible perturbation η, cf. [23]: As in [9], this can be rewritten using Fubini's theorem as Since this has to hold for any perturbationη t , the tower property of conditional expectation yields for a martingale dM t = Z t dW t that needs to be determined as part of the solution. Solving for the dynamics of the agents' optimal trading rates would introduce the dynamics of the transaction costs. Accordingly, it is preferable to instead work with the marginal transaction costs transaction costs as the backward process that describes the optimal controls: and from (2.6) we can infer that Y T = 0. With this notation, the corresponding trading rates arė Thus, the optimal position ϕ and the corresponding marginal transaction costs Y in turn solve the nonlinear FBSDE For constant quadratic costs λx 2 /2 and constant volatility σ, this FBSDE becomes linear and can in turn be solved by reducing it to a standard Riccati equation system [9,13,47]. For volatilities and quadratic costs that fluctuate randomly, these ODEs are replaced by a backward stochastic Riccati equation, compare [8,41]. With nonlinear costs, no such simplifications are possible. In fact, the wellposedness of the system is generally unclear even for short time horizons since no Lipschitz condition for (G ) −1 is satisfied.

Reparametrization.
In the frictionless analogue of (2.3), pointwise maximization yields the agent's individually optimal strategies, i.e., (2.10) In particular, we can write the dynamic of the frictionless strategy as In real-world applications, people are more interested in the changes induced by transaction costs relative to the frictionless version of the model. To facilitate both the numerical and the analytical analysis, we define ∆ϕ t := ϕ t −φ t (2.12) for the difference between the frictional and frictionless positions, which we expect to vanish as the transaction costs tend to zero. In view of the dynamic (2.11) strategyφ in the frictionless market and the forward equation (2.8), this process has dynamics Accordingly, plugging (2.10) into (2.9), the backward process Y therefore becomes (2.14) Remark 2.2. The analysis is focused on the optimal marginal transaction costs Y instead of the optimal trading ratesφ, to avoid the potential extra requirement on the differentiability of (G ) −1 . Moreover, this choice of backward component has linear dynamics, and absorb all the nonlinearity into the dynamic of the forward component, which makes our asymptotic analysis much easier and the numerical solution much more stable. We choose the deviation of the frictional position with respect to its frictionless counter party as the forward variable, since it exactly is the "fast" variable in the theoretical study in the single agent problem as in [9,46,55]. Empirically this choice performs the best in FBSDE solver. Notice that the driftb for the frictionless strategy is usually neglected in the small-costs regime, and the frictionless volatilityā makes the FBSDE system non-deterministic.

Deep Learning-based Numerical Algorithms
For practitioners, it is crucial to get the optimal trading strategy numerically, especially in a timeefficient manner. Therefore, efficient numerical methods to solve the optimization problem (2.3) are in great need. With the development of GPUs and highly accurate machine learning models, the optimal hedging problems can be solved numerically by using learning-based algorithms. There are two numerical approaches to do so: numerically solving the FBSDE system (2.13) -(2.14) or directly targeting the goal functional (2.3). In this section, we first present two popular deep learning-based numerical algorithms guided by these ideas: the FBSDE solver, and the Deep Hedging algorithm. Then, we highlight the advantages and disadvantages of these two methods in practice, especially when both of them fail to work. Finally, we introduce a transfer learning-based method, the ST-Hedging algorithm, to efficiently solve the optimal hedging problems when others fail to do so.
A natural idea to start with is to solve the associated FBSDE system (2.13) -(2.14) numerically. Since the dimension of the FBSDEs grows quickly with the number of assets and agents, and the boundary conditions are unclear, classical numerical methods such as finite differences fail to work. However, the learning-based algorithms can bypass the need to identify the correct boundary conditions and overcome the curse of dimensionality. The first learning-based introduced in Section 3.1 is the FBSDE solver, proposed in the spirit of the BSDE approach by Han, Jentzen, and E in [32]. The algorithm approximates the dependence of the backward components on the forward components by a deep neural network, and it solves the FBSDE through simulation and stochastic gradient descent methods. It can also handle higher-dimensional settings, e.g., random and time-varying transaction costs, returns, and volatility processes. In addition, the FBSDE solver can even solve equilibrium models as is used in [26]. FBSDE solver works very well when the time horizon is not too long, and the convergence of this method with respect to a special class of FBSDE systems is established in small time durations in [33]. However, the FBSDE solver does not scale well when the trading horizon or time discretization is large. In our example, with calibrated market parameters from [26], the FBSDE solver fails to converge if we consider a more than one trading year trading horizon (approximately 252 trading days). Furthermore, in some complicated cases, we cannot even characterize the optimal trading problem with a system of FBSDEs.
In order to overcome the difficulties mentioned above, we consider deep reinforcement learning framework that is known as Deep Hedging in Section 3.2. The name for this type of algorithms comes from the groundbreaking Deep Hedging work of Buehler et.al. [14,15]. Pioneered by [12,34,36,37,44,45,48,49,50,51], various reinforcement learning algorithms are implemented and perform extreme success in portfolio optimization problems with transaction costs. Reinforcement learning can even solve the Nash equilibria, see [16] for details. The key idea is to directly parametrize the optimal trading rate and optimize the discretized version of preference (2.3). Then, through stochastic gradient descent based algorithms and back-propagation, the reinforcement algorithm updates the parameters of the networks until a (local) optimizer is found (see [56] for a detailed introduction). However, Deep Hedging algorithms require a huge number of simulated sample paths and finer discretization of the trading time horizon, which makes the training time significantly longer than FBSDE solver.
One major drawback for both algorithms is that their performances become suboptimal when the trading time horizon is long. Just as in the experiments illustrated in Section 4.1 and Section 4.2, when the investment horizon is more than one trading year, FBSDE solver fails to converge. Although Deep Hedging still works, it suffers from overfitting and becomes difficult to tune. Moreover, when there are more than one stocks in the market with cross-sectional effects, the FBSDE solver and Deep Hedging both fail to converge even with intermediate trading horizons. In other words, these methods do not generalize well with the increase of the dimensions due to coupling.
In parallel, as in [5,10,31,40,46,54,55], researchers have focused on the asymptotic optimal trading strategy in the small transaction costs limit. Especially, there are explicit asymptotic formulas available for the optimal trading strategies. A nature following-up question is if we can take advantage of both the advantages of the deep learning algorithms and the asymptotic approximations. With this idea, we propose our ST-Hedging algorithm that allow us to have the convenience of the explicit asymptotic approximation and the accuracy of the deep learning algorithm. ST-Hedging reaches state-of-the-art performance comparing to Deep Hedging for long investment horizons as in Section 4.1 and Section 4.2. In addition, it is scalable with respect to dimensions as in Section 4.3.
The organization of the rest of the section is as follows. First, we introduce the FBSDE solver in Section 3.1 and the Deep Hedging algorithm in Section 3.2. In Section 3.3, we introduce the leading-order approximation formula and propose our ST-Hedging algorithm.

FBSDE Solver
Let us describe the algorithm in more detail. The key observation is that the dynamic of the system (2.13) -(2.14) is pinned down if the volatility Z of the backward component Y is specified. Indeed, let us fix a time partition 0 = t 0 < . . . < t N = T , where t m = mT /N and ∆t = T /N . Let (∆W m ) N m=0 be iid normally distributed random variables with mean zero and variance ∆t. At time t m , we have µ tm , σ tm and λ tm from the market and the endowment volatility ξ tm , the driftb tm and volatilityā tm for the frictionless strategy of the agent. With (∆W m , ∆ϕ tm ) as input for each time step, the discrete-time analogue of the forward update rule for the FBSDE system (2.13) -(2.14) becomes Together with a guess for the initial values of the backward components, we can then simulate the system with a standard forward scheme and check whether it satisfies the terminal condition as Y T = 0. Searching for the correct initial values is relatively straightforward. The more difficult challenge is parametrizing the "controls" Z -which are functions of time and the forward processes -and updating the corresponding initial guesses until the terminal condition is matched sufficiently well. In [32], Han, Jentzen, and E introduced the innovative way to parametrize Z. At each time point t m , they parametrized Z tm with a (shallow) network structure F θm , with time t m value of the forward components as the inputs. Together with the guess of the initial value of the backward component, which we denote as Y θ 0 , the system can then be simulated forward in time. In this way, the shallow network structures are concatenated over time and become a deep neural network architecture, since the number of layers of the architecture and the number of parameters in the networks grows linearly in the number of discretization steps N . In other words, we input simulated Brownian paths into the deep network structure, and it outputs the terminal values of the backward components.
Our task is to update the parameters {Y θ 0 , θ m , m = 0, . . . , N } until the terminal conditions Y T = 0 are matched sufficiently well. This iterative update of the network parameters until Y T = 0 is the essence of supervised machine learning tasks. State-of-the-art performance is achieved through back-propagation and stochastic gradient descent-type algorithms, see [27] for more details.
Remark 3.1. Recall that there is no assumption on the dynamic of the market. Here, however, we need to mildly assume that we can simulate the return µ and volatilities σ, the driftb and volatilitiesā for the frictionless positionφ from (2.11), as well as the endowment volatilities ξ. Now, we focus on the parametrization of {Z tm } N m=0 within a function class {F θ : θ ∈ Θ}. A very popular class of functions in the machine learning community is the "singular activation function Rectified Linear Unit" (ReLU ): ReLu(x) = max{x, 0}. A popular class of approximation function F is the convolution of linear functions with ReLu activations. For the numerical experiments in Section 4, at each time t m , the F θm we use is a neural network with one hidden layer of 15 hidden units with batch normalization (BN), and that is Recall that with the Brownian motion W as a 1-dim process, and the forward component ϕ as a 1-dim vector, Z is also a 1-dim vector. w 1 θm ∈ R 15×2 and b 1 θm ∈ R 15 are called the input weights and biases of the network, whereas w 2 θm ∈ R 1×15 and b 2 θm ∈ R 1 are called the output weights and biases of the network. We summarize the structure of FBSDE solver in Algorithm 1: Calculate the gradient of Loss with respect to θ; 16 end 17 return: (local) optimizer θ * = {Y θ * 0 , θ * m , m = 0, . . . , N }. As studied by [33], when the short time existence of the FBSDE system is available, the convergence rate of the FBSDE solver is guaranteed. In reality, FBSDE system with short time existence does not always generalize to global existence. That is to say, if we only extend the time horizon with all the parameter remain the same, the FBSDE solver might not converge anymore. Especially, since only the terminal value of the backward components are compared, the FBSDE solver fails to work efficiently with even intermediate trading horizons.

Deep Hedging Networks
Let us fix a time partition 0 = t 0 < . . . < t N = T , where t m = mT /N and ∆t = T /N . Under this partition, for a single simulation, the discretized version of the goal functional (2.3), becomes where ϕ follows the discretized version of update process At each time point t m , we directly parametrize the trading strategyφ tm using a (comparatively shallow) network F θm . For the numerical experiments in Section 4, the neural network we use has three hidden layers with 10, 15 and 10 hidden units, respectively. We also implement the batch normalization (BN) at each hidden layer and that is, for l = 1, 2, 3: Recall that with time t as a real number, the Brownian motion W as a 1-dim process, and the forward component ϕ as a 1-dim vector,φ is also a 1-dim vector. w 1 θm ∈ R 10×3 and b 1 θm ∈ R 10 are called the input weights and biases of the network, and w 2 θm ∈ R 15×10 , b 2 θm ∈ R 15 , w 3 θm ∈ R 10×15 and b 3 θm ∈ R 10 are called the hidden weights and biases, whereas w 4 θm ∈ R 1×10 and b 4 θm ∈ R 1 are called the output weights and biases of the network. We summarize the Deep Hedging algorithm in Algorithm 2 as follows:

Stable-Transfer Learning Algorithm: ST-Hedging
The optimal hedging problem can be treated in two folds. On the numerical aspect, as the development of deep learning-based algorithm, we can also solve the system through various different scheme, with is discussed in details in Section 3. On the theoretical aspect, one can study the asymptotic limiting behavior as the trading horizon T goes to ∞. Researchers always focus on small transaction costs for tractability as in [5,10,31,40,46,54,55]. A natural question is, when the trading horizon T is identified as long and whether it is related to the smallness assumption of the transaction costs level λ. It turns out, the "magic" quantity is √ λ/T ; namely, the smallness assumption on λ should not only be an absolute quantity, but also a relative quantity, depending on whether √ λ/T is of higher order of 1. Even more surprisingly, O( √ λ/T ) is the approximation accuracy for the leading-order approximation (3.4) under rather general condition (see the discussion in Appendix A), that is independent of the choice G of the transaction costs.
More specifically, the leading-order asymptotic optimal strategyφ = (φ t ) t∈[0,T ] for the frictional mean-variance preference (2.3) is (with a little abuse of notation) essentially given by: where g is the unique solution to the following ODE with proper "mean-reverting" requirement: This leading-order formula (3.4) and the associated ODE (3.5) are proposed and studied formally in [10,18,31,40,55], and we refer the readers to Appendix A for details of the derivation and rigorous proof in a special case. Notice that once we obtain the numerical solution of (3.5), the leading-order formula can be applied immediately.
A nature following-up question is if we can take advantage of both the advantages of the deep learning algorithms and the leading-order approximations. More precisely, can we have the convenience of the leading-order approximation formula and the accuracy of the deep learning algorithm near the very end of the terminal date. Based on the leading-order approximation accuracy O( √ λ/T ), we answer the above question in affirmation. To fully utilize the advantages, we need to adjust the deep learning-based algorithm. While we can obtain good performance under the current setup for the Deep Hedging algorithm, the hedging policy learned by the deep neural network becomes more and more unstable for longer time horizons. This is because the variance induced by the Brownian motion W t is proportional to the time t, hence the hedging positions ϕ t is going to blow up as t increases. To point out, with more than one stocks, the coupling effects will further contribute to the instability of ϕ t . Instead, inspired by the variable choice of the FBSDE solver, we use the "fast variable": ∆ϕ t = ϕ t −φ t . As the apriori study in [5,10,31,40,46,54,55], the "fast variable" ∆ϕ t −φ t is a mean-reverting process that quickly oscillates around zero. In particular, the variance of ∆ϕ t is upper bounded hence stabilized as time increases. As long as the deep learning algorithm on a short trading horizon is well-trained, then it can be easily adapted to arbitrarily long trading horizons without scaling up the errors. Numerical experiments have demonstrated the success of our variance reduction technique as in Section 4. This choice is reflected in the "stable" part of the ST-Hedging.
For the "transfer" part of ST-Hedging, we leverage transfer learning technique to automatically pick the optimal switching time from the approximation formulas to accurately learning of the optimal strategy via deep learning-based algorithm. By an initialized κ, we initialize our starting location t M near the end of the trading horizon by the suggestion from the approximation accuracy T − t M = O( √ λ). Then, with semi-well trained network, we simulate the rest of hedging positions through the stable hedging strategy, and compare it with the asymptotic hedging strategy of the accumulated gain. Then, we select the optimal M * such that the stable hedging loss is less than the asymptotic optimal trading loss by the largest amount and convert it back to κ = (T − t M * )/ √ λ. We repeat the process iteratively until κ converges, which equivalently means the optimal switching time t M * is obtained. Notice that we alternatively learn the optimal trading strategy and the optimal switching time, and the output of ST-Hedging is the time t M * to switch from the asymptotic strategy and the network parameters for the learnt trading strategy afterwards.

Experiments and Comparison Results
In this section, the FBSDE solver in 3.1, the Deep Hedging algorithm in 3.2 as well as the proposed ST-Hedging algorithm in 3.3 are implemented. We test them on various models which even includes more than one stocks and/or with stochastic liquidity risk, and we discuss and document their advantages and disadvantages in these empirical implementations in Section 4.4.
To illustrate the performance of the FBSDE solver, the Deep Hedging and the ST-Hedging algorithm, we consider two calibrated Bachelier models without liquidity risk: a quadratic transaction costs model, and a general power costs model with elastic parameter q = 3/2. In all experiments, the parameters are from the calibration of real-world time series data in [26]. The agent's risk aversion is set to be γ = 1.66 × 10 −13 , the total shares outstanding in the market is s = 2.46 × 10 11 , and the stock return is µ = 0.072 with the stock volatility being σ = 1.88 1 . All models were trained using the free GPU service on Google Colab. The implementation details and codes can be found here: https://github.com/xf-shi/ML-for-Transaction-Costs/blob/main/README.md.

Quadratic Transaction Costs
With G(x) = x 2 /2, on top of the leading-order approximation, the FBSDE system (2.13) -(2.14) becomes linear and the optimal trading strategy is given explicitly by 2 : The remaining parameters are taken from the calibrations from Section 5 in [26], where under quadratic costs the endowment volatility parameter ξ = 2.19×10 10 and the liquidity level parameter is λ = 1.08×10 −10 . With the trading horizon T = 10, 21, 42, 252, 2520 trading days, the performance of both learning methods and the leading-order approximation are summarized in Table 1 -5 and illustrated in Figure 1 -5. To account for the large Monte Carlo error from the long trading horizons such as T = 2520 trading days (which is approximately 10 trading years), we evaluated the performance of our models using 100 million sample paths.
When the trading horizon is relatively short (T ≤ 42 trading days), the performance of the FBSDE solver is the closest to the ground truth path-wisely, see Figure 1 -3. It means that when the FBSDE solver converges, it can learn the optimal trading strategy perfectly. It takes approximately 1-2 hours to complete the training with fine tuning. However, as is already experienced in [26], when the trading horizon is longer than one trading year, the FBSDE solver fails to converge.
On the other hand, the Deep Hedging algorithm can still work well once the hyperparameters are appropriately tuned, as illustrated in Figure 4 -5 and Table 4 -5. However, we can see that underfitting phenomena exist, and it becomes more severe as the trading horizon increases, meaning that the hyperparamters become harder to tune. The training time of Deep Hedging varies with the length of the trading horizon. For short horizons, the training time is approximately 3-4 hours with fine tuning, which is longer than FBSDE solver. For longer trading horizons, the training and fine tuning time required can be as long as weeks, or even months, which is too long for practical use.
In addition, the performance of the leading-order asymptotic approximation (3.4) becomes better and better as the time horizon becomes longer. From the mean squared error ofφ T /s in the last column of Table 1 -5 we can see that shortly around the terminal time, the leading-order approximation diverges substantially from 0. This observation is consistent with the figures ofφ and ϕ as in Figure 1 -5. In other words, we are giving up the performance at the very end of the trading horizon to get an overall near-optimal performance without spending hours or days on tuning the hyperparameters in the machine learning algorithms. In fact, when the trading horizon is as large as T = 2520 trading days, leading-order approximation achieves the closest expected utility to the ground truth, while Deep Hedging is hard to train as the variance of the parametrized position ϕ t becomes very large with long trading horizon.
The analogous experiments for power costs with q = 3/2 are also done with calibrated parameters from Section 5 in [26], where the endowment volatility is ξ 1.5 = 2.33 × 10 10 and liquidity level λ 1.5 = 5.22 × 10 −6 . In the general superlinear power transaction costs case, i.e. G(x) = |x| q /q with q ∈ (1, 2), the ground truth is no longer available in closed form. Nevertheless, we can still compare the numerical results from the machine learning algorithms and our leading-order asymptotic results. The models are evaluated using 100 million sample paths in order to account for the large Monte Carlo error from long trading horizons.
With the dynamics of the system being nonlinear, the FBSDE solver converges on even shorter trading horizons, i.e., T < 21 trading days, shown in Figure 6 -7 and Table 6 Table 8 -10. However, when the trading horizon is as long as 10 trading years, Deep Hedging becomes suboptimal because of the large variance of the position ϕ t . Similar as in the quadratic costs case, with the trading horizon increasing, the performance of the leading-order approximation is getting closer to the optimal results learnt by the machine learning algorithm, which justifies the approximation result of the leading-order approximation (3.4) empirically. ST-Hedging consistently achieves the best performance utility across all trading horizons (Table 6 -10). In short trading horizons, it achieves similar results as deep hedging (Figure 6 -8). As the trading horizon gets larger, the noise of Deep Hedging becomes more and more severe, while ST-Hedging is still stable (Figure 9 -10). The training and fine tuning time for these algorithms are similar to the quadratic costs case, with ST-Hedging requires significantly less training time and effort for tuning than the other methods.      3.89 × 10 9 ± 3.39 × 10 9 1.53 × 10 −8 FBSDE Solver NaN NaN Leading-order Approximation 3.84 × 10 9 ± 3.39 × 10 9 1.11 × 10 −4 Table 8: Expectation and standard deviation of preference J T , and mean squared error ofφ T /s for trading horizon T = 42 days with calibrated parameters under q = 3/2 costs. 3.84 × 10 9 ± 9.18 × 10 9 9.27 × 10 −9 FBSDE Solver NaN NaN Leading-order Approximation 3.84 × 10 9 ± 9.18 × 10 9 7.93 × 10 −5 Table 9: Expectation and standard deviation of preference J T , and mean squared error ofφ T /s for trading horizon T = 252 days with calibrated parameters under q = 3/2 costs. NaN NaN Leading-order Approximation 3.78 × 10 9 ± 2.59 × 10 10 8.24 × 10 −5 Table 10: Expectation and standard deviation of preference J T , and mean squared error ofφ T /s for trading horizon T = 2520 days with calibrated parameters under q = 3/2 costs.

Multi-asset Example via ST-Hedging
To illustrate the scalability of our ST-Hedging algorithm, we consider a model with three risky assets in the market with cross sectional effect. In this experiment, the risky assets are driven by a 3-dim Brownian motion, and their corresponding annualized arithmetic variance is Σ =   72.00 71. 49  The three stocks have outstanding shares are 1.15 × 10 10 , 3.2 × 10 9 and 2.3 × 10 9 , respectively. The annualized arithmetic returns for the three stocks are set to be 2.99, 3.71, and 3.55. Further, the risk aversion is set to be γ = 7.424 × 10 −13 , and the endowment volatility matrix is set to be The transaction costs parameters are set as 1.269 × 10 −9 , 1.354 × 10 −9 , and 1.595 × 10 −9 . Finally, the trading time horizon is set to be T = 2520 trading days, which is 10 trading years, and the switching threshold we choose is 100 days before maturity.
The performance of the ST-Hedging algorithm and the comparisons to the leading-order formula and the ground truth are shown in Figure 11 and Table 11. We can easily see that the overall performance of ST-Hedging is comparable with the leading-order approximation, and can also accurately learn the trading strategy when it is close to maturity. Moreover, the training time for the ST-Hedging algorithm is significantly smaller than the Deep Hedging algorithm, with better scalability of the number of stocks in the market. The training and fine tuning time for multiple assets is still 1-2 hours, which shows the scalability of ST-Hedging algorithm with dimensions. Method J T (φ) ± std E[|φ T | 2 /s 2 ] ST-Hedging 1.60 × 10 11 ± 3.58 × 10 6 (2.4, 4.7, 3.6) × 10 −9 Leading-order Approximation 1.60 × 10 11 ± 3.58 × 10 6 (4.5, 3.5, 1.6) × 10 −7 Ground Truth 1.60 × 10 11 ± 3.58 × 10 6 (0.0, 0.0, 0.0) Table 11: Expectation and standard deviation of preference J T , and mean squared error ofφ T /s for trading horizon T = 2520 days with calibrated parameters under quadratic costs.

Comparison of Algorithms
Beyond the experiments in Section 4.1 -4.3, we implement the FBSDE solver, the Deep Hedging algorithm and the ST-Hedging algorithm and test them in various market settings. The details and codes are available here: https://github.com/xf-shi/ML-for-Transaction-Costs. Empirically, the appearance of the extreme value in the dynamic of the FBSDE solver may affect the calculation precision and thus jeopardize the performance. In particular, the performance and the harness of tuning of the FBSDE solvers becomes significantly worse if we increase the number of forward and backward variables with respect to multiple stocks settings, because of the coupling in the FBSDE system. In comparison, the Deep Hedging still works well, which shows its good scalability with respect to the increase of dimensions. A major drawback of both algorithm is they do not generalize well to long trading horizons. The FBSDE solver fails to converge even in intermediate trading time horizon, whereas the Deep Hedging algorithm still works but requires significant training time. To overcome the above mentioned difficulties in scalabilities, our proposed ST-Hedging algorithm fully utilizes the convenience of the leading-order formula and the accuracy of the learning-based algorithms. In our comparison experiments, ST-Hedging beats the performance of the leading-order formula and shows great scalability with respect to the increase of the stocks and the increase of trading time horizon, with less training time required. In fact, ST-Hedging can be generalized as long as the leading-order approximation is available. These comparison results help explain the leading-order asymptotic formula and provide us with ideas on the usage of each method in practice. Moreover, it opens door to the development of model-based algorithms. We summarize the observations (High/Medium/Low) in

Conclusion and Looking-forward
This paper studies the optimal hedging problems in frictional markets with general convex transaction costs on the trading rates, and propose the ST-Hedging algorithm to fully utilize the leading-order asymptotic formulas and the deep learning-based algorithms. Under the smallness assumption on the magnitude of the transaction costs, the leading-order approximation of the optimal trading speed can be identified through the solution to a nonlinear SDE. Models with arbitrary state dynamics generally lead to a nonlinear FBSDE system, but we can still solve the optimization numerically through learning-based algorithms. We implement the FBSDE solver, the Deep Hedging algorithms and the ST-Hedging algorithms and compare their performances with the leading-order approximation. The leading-order approximation works well under long trading horizons, while it deviates from the ground truth by a significant amount only shortly before maturity. The FBSDE solver only performs well under short trading horizons, while it fails to work in long trading horizons. In contrast, the Deep Hedging algorithm has stable and reliable performance for short and intermediate trading horizons, while it starts to get unstable when the trading horizon is getting large. In addition, the hyperparameter tuning of the Deep Hedging algorithm becomes inefficient when the model has more sophisticated dynamics. With the development of transfer learning concept and the initial choice given by O( √ λ/T ), the proposed ST-Hedging algorithm fully utilizes the leading-order asymptotic formula and learning-based algorithms. Indeed, ST-Hedging shows the best performance in all our experiments, with a drastically reduced training time. Moreover, the success of ST-Hedging algorithm shows great potential in model-based algorithms, to leverage the knowledge from the domain experts and the accuracy of learning-based algorithms. This work is also a preliminary documents for learning-based numerical solution to frictional equilibrium models.
where G * is the Legendre transform of G.
Remark A.3. If the time horizon T is large, then the solution to the optimal trading strategy should become stationary. Such a stationary solution should in turn solve the essential nonlinear ODE (3.5). Far from the terminal time T , it is natural to expect that the correct solution is still identified by the same growth condition in the space variable as (A.1).
For power functions G(x) = |x| q /q, q ∈ (1, 2], the Legendre transform is where p = q/(q − 1) is the conjugate of q. In this case, with proper inner and outer rescaling coefficients, (3.5) is exactly the same ODE which plays an important role in Lemma 19 and Lemma 21 in [31] and Lemma 3.1 in [18].
A fast mean-reverting SDE The second ingredient is the existence and uniqueness of a strong solution to a fast mean-reverting SDE.
Lemma A.4. Let g be the solution to the ODE (3.5) from Lemma A.2. There exists a unique strong solution of the SDE Moreover, this process is a recurrent diffusion.
With these two ingredients on hand, we now present our first results in the following theorem: Theorem A.6. Let g be the solution to (3.5) and (∆ t ) t≥0 the solution to (A.2). Then under Assumption 2.1 and Assumption A.1, for all competing admissible strategiesψ, we have J T (ψ) ≤ J T (G ) −1 g(∆; γ, σ,ā, λΛ) λΛ Theorem A.6 shows that, under Assumption A.1 the smallness is not only an absolute quantity on √ λ, but also on the relative quantity √ λ/T , i.e. the smallness of λ should also be relative quantity with respect to the trading time horizon. Notice that the order of the smallness is derived as a coarse upper bound for every transaction costs function G that satisfies Assumption 2.1. In fact, with the specific form of the transaction costs G, we can have finer estimations, as in the example of quadratic costs shown in Corollary B.2. For the general power costs case and proportional costs case, we refer the reader to the discussion of Theorem 3.3 in [18] and Theorem 4.2 in [26]. hence θ t = t 0ψ u − G −1 g(∆ u ; γ, σ u ,ā u , λΛ u ) λΛ u du = ψ t −φ t .