{'fraction_non_alphanumeric': 0.06409911123081066, 'fraction_numerical': 0.015935003142113296, 'mean_word_length': 3.730148619957537, 'pattern_counts': {'":': 0, '<': 2, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
A semi-Markovian approach to model the tick-by-tick dynamics of stock price

We model the stock price dynamics through a semi-Markov process obtained using a Poisson random measure. We establish the existence and uniqueness of the classical solution of a non-homogeneous terminal value problem and we show that the expected value of stock price at horizon can be obtained as a classical solution of a linear partial differential equation that is a special case of the terminal value problem studied in this paper. We further analyze the market making problem using the point of view of an agent who posts the limit orders at the best price available. We use the dynamic programming principle to obtain a HJB equation. In no-risk aversion case, we obtain the value function as a classical solution of a linear pde and derive the expressions for optimal controls by solving the HJB equation.


Introduction
Market microstructure is a branch of finance that deals with the details of how the trading occurs in the financial market. According to the National Bureau of Economic Research(NBER), the study of market microstructure is concerned with the theoretical, empirical, and experimental research on the economics of security markets, including the role of information in the price discovery process, the definition, measurement, control, and determinants of liquidity and transactions costs, and their implications for the efficiency, welfare, and regulation of alternative trading mechanisms and market structures [1].
In this work, we aim to study the market microstructure by modelling the tick by tick dynamics of stock price via a semi-Markov process where the duration between two ticks may not be exponential [5].

Price Dynamics
2.1. Overview of the Model. The ask or bid prices are quotations by agents who want to sell or buy certain units of a financial asset respectively. The lowest ask price and the highest bid price are called the 'best ask' and 'best bid' prices respectively. At a given time the bid-ask spread corresponds to an interval, whose upper limit is the best ask price and the lower limit is the best bid price. There are two types of orders prominent in the financial market which we consider in our model known as the limit order and the market order. The limit order is posted by a trader who wishes to buy or sell at a price different than the current best price and it is executed only when a potential sell or buy order arrives matching On the contrary, the market order is placed by a potential buyer or seller who wishes to buy or sell at the current best ask or best bid price and the order is exercised immediately. On the basis of 'size', the market orders can be termed small or big. The small orders do not cause any change in the price of the asset. On the other hand, the large market orders may exhaust all the liquidity available in the market. At these occasions the stock price moves down or up if the market order is of sell or buy type respectively. For the purpose of modelling, from now onward 'price' refers to the mid-price of the stock which is the arithmetic mean of the best bid and the best ask price.
We model the return dynamics of price through a semi-Markov process which changes its state whenever the stock price goes up or down. In the most natural way, the state space can be thought of having just two elements each one representing positive and negative price jump direction. Since the direction of the price jump can be same in two consecutive jumps, we consider an auxiliary state process where the state space is not binary rather it has four states but a given state communicates to only two other states that are specified beforehand for that state. This helps us to obtain the state process as a semi-Markov process with an embedded Markov chain that covers all the possibilities of price transition exactly once.
The price process is a pure jump process and we assume that the absolute value of simple rate of return always remains constant denoted by δ. We also assume that the bid-ask spread is equal to 2pδ where p is the current mid-price. The price process together with state process and its associated age process form an augmented Markov process obtained as a solution of stochastic integral equations involving integration with respect to a Poisson random measure. Next we formulate the required mathematical set up and express the stock price dynamics formally in mathematical terms.

Construction of underlying process.
Assume that X is a Markov chain on the binary set {o, e} with transition matrix P and the auxiliary chain X is on {1, 2, 3, 4}, with . If the projection map ϕ : {1, 2, 3, 4} → {o, e} is such that ϕ(1) = ϕ(3) = o, and ϕ(2) = ϕ(4) = e, then X and ϕ(X ), the projection of auxiliary chain, have identical law. Here, X does not transit to the present state even if X does. This is further clarified by the diagram in Figure 1. We recall that transition to the same state by a continuous-time process on a discrete state-space is imperceptible. However, in the above manner, such a process can be formulated as the projection of its auxiliary process. We make use of the auxiliary chain for modelling the price direction process.
Let (Ω, F , {F t } t≥0 , P) be a complete filtered probability space satisfying the usual hypothesis. The state space {1, 2, 3, 4} is denoted by X . We define an equivalence relation '≡' on X such that for i, j ∈ X , we say i ≡ j if and only if both belong to either {1, 2} or {3, 4} and we say i ≡ j otherwise. We denote the set {(i, j) ∈ X 2 | i ≡ j} by X . For each ν ∈ {+, −}, let h ν be a non-negative measurable function defined on [0, ∞) and for each s ∈ [0, ∞), let H + (s) and H − (s) be two right open, left closed non-overlapping intervals with length equal to h + (s) and h − (s) respectively. We denote H + (s) ∪ H − (s) and h + (s) + h − (s) Figure 1. The diagrammatic representation of transition probability matrices of a binary Markov chain and its auxiliary process by H(s) and h(s) respectively for each s ≥ 0.
For the brevity of notation, for each s ≥ 0, (i, j) ∈ X we denote Hᾱ (i+j) (s) and hᾱ (i+j) (s) by H ij (s) and h ij (s) respectively. Now we define the real valued measurable maps g 1 , g 2 and g 3 on the set X × [0, ∞) × R as follows: where 1 A denotes the indicator function of set A and δ > 0.
Let S 0 , I 0 and P 0 be F 0 -measurable independent random variables. We make the following assumptions which are applicable throughout this paper.
h ν (y) < ∞ and we denote it by c 1 .
The probability distributions of the random variables S 0 , I 0 and P 0 have full support.
Let ℘(ds, dz) be a Poisson random measure defined on [0, ∞) × R with intensity measure equal to the product Lebesgue measure on [0, ∞) × R and it is adapted to the filtration {F t }. Let us consider the following processes: It is known that under the assumption (A2), there exists a pathwise unique strong solution (P, I, S) := {(P t , I t , S t )} t≥0 to the above system of equations which is right continuous having left limits, locally bounded and strongly Markov (see Theorem 3.4 (p-474) of [2]). It is evident from the above equations that P , I are piecewise constant and S is a piecewise linear process and these processes have jump discontinuities at the same time.
2.3. Law of the underlying process. It can be seen that under the assumptions (A2) and (A3), I is a semi-Markov process with S being the 'age' process [7]. We now define few parameters associated with the process (I, S). Further for each (i, j) ∈ X , we define a map p ij on the set [0, ∞) such that for y ≥ 0, Due to assumption (A4), the map p ij is well-defined. Now let F and f be two maps defined on [0, ∞) such that for y ≥ 0, The assumption (A1) affirms that F , its derivative on the set (0, ∞) given by f , and p ij for all (i, j) ∈ X are continuously differentiable functions. Owing to the assumptions (A3), (A4) and the expression of F , it follows that F is a strictly increasing function, it never attains the value 1 for any y ∈ [0, ∞) and lim y→∞ F (y) = 1.
It is known that p ij (y) is the conditional probability that the process transits next to the state 'j' given that the process is currently in state 'i' and the next transition happens after 'y' unit of time has been elapsed since the previous transition. Further, F denotes the conditional cumulative distribution function of the holding time associated with the process I given the current state [7]. It turns out that F is independent of the state in this scenario. We also see that for y ≥ 0, Now let ψ be a real-valued continuous map defined on the set [0, ∞) × X × [0, ∞) which is compactly supported as well as continuously differentiable function in the third variable. Let G ψ be real valued map on (0, T ] × R such that for (v, z) ∈ (0, T ] × R, We now calculate the infinitesimal generator of the process (P, I, S). Using Itô's formula, is finite using the boundedness of ψ and assumption (A2). Hence we can rewrite the last term of (2.12) as where℘ is the compensated Poisson random measure. Using (2.12) -(2.14), we infer that the infinitesimal generator A of the process (P, I, S) is given by 3. An applicatory non-homogeneous terminal value problem for each φ ∈ V , where w ∈ V and g is a continuous function on [0, ∞) with at most linear growth. Then the following hold.
(1) T maps V to V .

3) The following equation has a unique solution in
where

Continuity of T φ
We note that since F and g are continuous functions, T 1 is continuous. Next we prove T j 2 is continuous. First we check the continuity of T j 2 with respect to t-variable. Now for (t, p, i, s) ∈ D and ε ∈ (−t, T − t), ε = 0, consider Using the fact that f, p ij and φ(·, p(1 + δα(j)), j, 0) are continuous and the domain of integration is contained in a compact set, therefore using bounded convergence theorem we get Hence the continuity of T j 2 with respect to t-variable has been proved.
Next we prove the continuity of T j 2 with respect to s-variable. For (t, p, i, s) ∈ D and ε ∈ (−s, s), ε = 0, consider Since f , p ij , F and φ(·, p(1 + δα(j)), j, 0) are continuous, proceeding similarly as above, we see that We have proved that T j 2 is continuous with respect to s-variable. We now prove the continuity of T j 2 with respect to p-variable. Again for (t, p, i, s) ∈ D and ε ∈ (−p, p), ε = 0, we have . Again as f and p ij are also continuous and the integration is over a compact set, using bounded convergence theorem we have Thus, we have proved the continuity of T j 2 with respect to p-variable.
Summarising all the three cases we conclude T j 2 ∈ C(D).
Since F and f are continuous maps, continuity of T 3 can be proved in the similar way as was the case in proving continuity of T j 2 . Since the continuity of T 1 , T j 2 and T 3 has been established, therefore using (3.3) we conclude that T φ is continuous. Now, in order to prove the assertion (1), We note that for the case φ = 0, for all (t, Since g has at most linear growth, we have sup For each φ ∈ V , the above inequality and continuity of Then the equation (3.9) implies that T is a contraction. Hence the assertions (1) are (2) are proved. As V is a Banach space and a solution of (3.2) is a fixed point of the contraction map T on V , using Banach Fixed Point theorem we conclude that the equation (3.2) has a unique solution in V . This completes the proof of assertion (3).
T φ belongs to cl(D t,s ).
We first prove T 1 belongs to C 1,1 (D) and hence also is in cl(D t,s ). Now, since F is con- The continuous differentiability of F and the continuity of g again imply that ∂T 1 ∂t and ∂T 1 ∂s are continuous, hence T 1 belongs to C 1,1 (D) and hence belongs to cl(D t,s ) as well Moreover, D t,s T 1 is given by Next we prove T j 2 belongs to C 1,1 (D) and thus belongs to class cl(D t,s ).
, ε = 0, and then using (3.7) we have Proceeding similarly as in the case of proving continuity with respect to t along with the application of Fundamental theorem of calculus for second term of the above equation, we get Therefore, partial derivative of T j 2 with respect to t-variable exists on D o and it is given by (3.13).
Proceeding similar to the case of proving continuity for T j 2 with respect to s, we infer (3.14) Therefore, partial derivative of T j 2 with respect to s exists on D o and it is given by (3.14).
Further since F, p ij , f are continuously differentiable functions and φ is also continuous, proceeding as in the case of proving continuity for T j 2 , we conclude that . Hence it has been proved T j 2 ∈ C 1,1 (D). Thus it also lies in class cl(D t,s ) and using (3.13) and (3.14), we see that for (t, p, i, s) ∈ D o , We further prove T 3 belongs to class cl(D t,s ). For this consider (t, p, i, s) ∈ D o , ε ∈ (−s, s) ∩ (−t, T − t), ε = 0, Since F is differentiable, w(·, p, i, ·) is continuous and the integration is over a compact set, using bounded convergence theorem and Fundamental theorem of calculus, we have Therefore, D t,s T 3 exists and it is given by (3.16). We also note that the continuity of f, F and w implies that D t,s T 3 is continuous and thus, T 3 ∈ cl(D t,s ).
Since we have proved T 3 , T j 2 and T 3 belong to cl(D t,s ), therefore using (3.3) we conclude T φ ∈ cl(D t,s ), thus proving assertion (4). Now for proving assertion (5), we assume w s := ∂w ∂s belongs to C(D o ). Let (t, p, i, s) ∈ D o , ε ∈ (−t, T − t), ε = 0, then Since F is differentiable and we have assumed the existence of ∂w ∂s in this case, then using the similar arguments as in the case of proving the existence of (3.17) Therefore ∂T 3 ∂t exists on D o if w s exists on D o and it is given by (3.17). Now again for (t, p, i, s) ∈ D o , ε ∈ (−s, s), ε = 0, consider Again since F is differentiable and w s exists, we have Hence ∂T 3 ∂s exists on D o and it is given by (3.18). Moreover, the continuity of f, w and w s ensures that ∂T 3 ∂t and ∂T 3 ∂t are continuous, using the same arguments as we had in the case of proving continuity of T j 2 . Thus, T 3 ∈ C 1,1 (D). As we have proved T 1 , T j 2 and T 3 (if we assume w s exists and is continuous), therefore T φ ∈ C 1,1 (D) in this case. This completes the proof of assertion (5).
Now using (2.10), (3.2) and the fact that j ≡i p ij (s) = 1, we have Also by plugging t = T in (3.1), we get φ(T, p, i, s) = g(p). Since φ belongs to V ∩ cl(D t,s ), we conclude the partial differential equation Similar to (2.11), we define Then proceeding as in (2.12)-(2.13), we get (3.21) Since φ and w are continuous and φ belongs to D t,s , using Itô's formula, we obtain where M := {M t } t≥0 is given by the stochastic integral M t : It follows from the above assertion, the equation (3.21) and the assumption (A2) that Let T n denote the time of n th jump of the process (P, I, S). Let for t > 0, N (t) denotes the number of jumps of the process (P, I, S) up to time 't'. Again we observe that the process P can be written as (1 + δα(I T k )) a.s.. (1 + δ) n P(N ℘ (T ) = n) = e 2c 1 T δ < ∞. |G φ (v, z)|dz dv ≤ t(4c 1 ||φ|| V + P 0 (1 + δ) N℘(T ) ) a.s.. Therefore, using (3.24) and the assumption (A5) we have Since G φ is a predictable process as it is a left continuous process, hence max{G φ , 0} and max{−G φ , 0} are positive predictable processes. Therefore, 0}℘(dz, dv) are martingales with respect to the filtration {F t } (see Remark 6.4c (p-300) of [2]). Hence M is also a martingale with respect to {F t }.
Moreover, since the process N φ = M + φ(0, P 0 , I 0 , S 0 ) is also a martingale using the above assertion, we have i.e., φ(t, P t , Using Markovity of the process (P, I, S) we see that the above equation can be written as almost surely for every t ∈ [0, T ]. Furthermore, the assumption (A6) ascertains that the distribution of (P t , I t , S t ) has full support for every t ∈ [0, T ] (see lemma A.1) then evaluating (3.26) for w ∈ {P t = p, We clearly see from (3.27) that every solution has the above stochastic representation therefore we conclude that the solution φ is unique. Hence the equation ( In [4], it has been shown that π is the unique viscosity solution to a linear partial differential equation with a terminal condition. We prove below that π belongs to C 1,1 (D) and it uniquely satisfies a similar terminal-value problem in the classical sense.
Proof. We consider the case when g is the identity map on [0, ∞) and w = 0 in the equations (3.19) and (3.20). Then using (3.27) we see that the unique solution of (3.19)-(3.20) in the class V ∩ cl(D t,s ) is nothing but π. Moreover since w = 0, theorem 3.2 asserts that the solution of (3.19)-(3.20) belongs to V ∩ C 1,1 (D). Hence π belongs to V ∩ C 1,1 (D).
4. The Market-making strategy and the portfolio of the agent 4.1. Formulation of the model. The incoming market orders can be categorised into two kinds, small market orders and big market orders depending on the size of the order. The 'small' market orders do not cause any change in the best bid/ask price whereas the big market orders lead to a jump in the best bid/ask price that can be upwards or downwards depending on the situation. The outstanding limit orders are documented in a Limit Order Book (LOB) which is maintained by the specialist at the exchange. We are analysing the market-making problem from the point of view of an agent whose strategy is to continuously place limit orders of constant size 'L' on both side of the LOB at the best price. We assume that the market order dynamics is not affected by the presence of the agent and the incoming orders get executed by the agent only on one side of the LOB at a time. (A8) λ + and λ − are continuous functions.

Financial interpretation.
Here ν represents the two sides of the LOB where by convention, we assume that ν = − represents the bid side and ν = + represents the ask side of the LOB. Also, ϑ ν gives the probability distribution of the random number of units of the stock getting traded on the side ν of the LOB, and k ∈ K denotes the number of quantities of the stock that get executed by the agent. The arrival of and execution of a small market order on side ν with 'k' units being traded by the agent is captured by the interval Λ νk (s) where 's' denotes the age of the process. The intensity of incoming small market orders is given by λ ν (s).
On the other hand, in the case of big market orders, the kind of order that must have arrived or the side that gets executed is determined by the current and the last price jump direction.The table below demonstrates this in different scenarios. Moreover, the interval H ij (s) corresponds to the case when a big market order arrives causing a jump in the stock price with the state process changing its state to 'j' from 'i and 's' being the time elapsed since last jump, with the intensity of the arrival of big market orders equal to h ij (s). The disjointness of these intervals ensures that only one kind of order is being executed on one side of the LOB at a time.  The wealth and inventory processes. The strategy adopted by the agent is described using a stochastic processl = (l + ,l − ) wherel + andl − are predictable processes taking values in the control space {0, 1} × {0, 1} denoted by L. We also recall that the bid-ask spread was assumed to be equal to 2pδ where p is the mid-price. Now for l = (l + , l − ) ∈ L, let g 4 and g 5 be two real valued maps defined on the sets X × [0, ∞) × [0, ∞) × R and X × [0, ∞) × R respectively as follows: The table below describes the change in portfolio of the agent when an incoming market order arrives, subject to the control l, thus explaining the rationale behind the definition of functions g 4 and g 5 .
We now describe the portfolio of the agent associated with the control processl using the processes X and Y which denote the wealth and the inventory of the agent respectively and are given by where X 0 and Y 0 are F 0 -valued random variables valued in R and Z respectively. We now find the infinitesimal generator of the process (P, I, S, X, Y ) in the above mentioned case of a constant control taking value l (= (l + , l − )) ∈ L. To this end, consider ψ be a real-valued continuous map defined on the set [0, ∞) × X × [0, ∞) × R × Z which is compactly supported and continuously differentiable function in the third variable. LetG ψ be real valued map on (0, T ] × R such that for (v, z) ∈ (0, T ] × R, (4.5) Using Itô's formula we have, Hence using (4.7) we see that t 0 RG ψ (v, z)dz dv is finite due to the assumptions (A2), (A7) and the fact that ψ is compactly supported. So we can write (4.8) Therefore we conclude using (4.6)-(4.8) that the infinitesimal generatorÃ l of the process (P, I, S, X, Y ) associated with the constant control taking value l is given bỹ Since for every i ∈ X , the set {ᾱ(j) | j ≡ i} = {+, −} therefore we can replace ν byᾱ(j) in the above equation and we obtaiñ A l ψ(p, i, s, x, y) = ∂ψ ∂s (p, i, s, x, y) − ψ(p, i, s, x, y)) + h ij (s)(ψ(p(1 + δα(j)), j, 0, x + α(j)lᾱ (j) K(p(1 + δα(j)) − α(j) ), i, s, x, y)) . After the formulation of market-making problem, the next goal is to maximize the risk-sensitive terminal expected portfolio value of the agent with a constant risk-aversion parameter η > 0 taking into account all possible strategies. We aim to maximize the following function Jl associated with the control processl where The market-making optimization problem is to maximize Jl over the class L of all possible strategies and the corresponding utility function is given by where c = 2(K + 1) max {c 1 , c 2 }.
Proof. Referring to Table 2, we see that the change in portfolio value (when a limit order gets fulfilled) corresponding to every control processl is always less than K(pδ + ) where p denotes the mid-price just before the execution of the limit order. In another words for t ∈ [0, T ] and for everyl ∈ L, this means Then using the dynamics of process P and the above inequality we obtain N (t)) . Proceeding similarly as in (3.24) and using the fact that intensity measure of ℘ is the product Lebesgue measure, we obtain and as (5.5) is true for every control processl ∈ L, we have shown that (5.3) holds for every (t, p, i, s, x, y) ∈ [0, T ] × [0, ∞) × X × [0, ∞) × R × Z.
The above lemma proves that the value function J is well defined and finite.