{'fraction_non_alphanumeric': 0.044688104671519395, 'fraction_numerical': 0.012776805184434411, 'mean_word_length': 4.322870688245872, 'pattern_counts': {'":': 0, '<': 15, '<?xml version=': 0, '>': 10, 'https://': 0, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Statistical Properties of the Keyboard Design with Extension to Drug-Combination Trials

The keyboard design is a novel phase I dose-finding method that is simple and has good operating characteristics. This paper studies theoretical properties of the keyboard design, including the optimality of its decision rules, coherence in dose transition, and convergence to the target dose. Establishing these theoretical properties explains the mechanism of the design and provides assurance to practitioners regarding the behavior of the keyboard design. We further extend the keyboard design to dual-agent dose-finding trials, which inherit the same statistical properties and simplicity as the single-agent keyboard design. Extensive simulations are conducted to evaluate the performance of the proposed keyboard drug-combination design using a novel, random two-dimensional dose--toxicity scenario generating algorithm. The simulation results confirm the desirable and competitive operating characteristics of the keyboard design as established by the theoretical study. An R Shiny application is developed to facilitate implementing the keyboard combination design in practice.


Introduction
The objective of phase I clinical trials is to identify the maximum tolerated dose (MTD) of a new drug, which is defined as the dose with the dose-limiting toxicity (DLT) probability closest to the target toxicity rate. Conventionally, phase I dose-finding designs are generally classified as algorithm-based and model-based [1,2]. Algorithm-based designs implement simple, prespecified rules to guide dose escalation and de-escalation. Examples include the 3+3 design, accelerated titration design [3], and the biased-coin design [4] and its variations [5,6]. The algorithm-based designs are easy to implement, but usually have unsatisfactory operating characteristics and lack theoretical support [1,2]. Model-based designs that utilize some parametric dose-toxicity models to make dose escalation/de-escalation decisions have been proposed to improve upon the performance of algorithm-based designs. The most well-known example is the continual reassessment method (CRM) [7], which begins with a prior guess of the dose-toxicity curve and then continuously updates the estimate of the curve based on the accruing toxicity outcomes from patients in the trial to guide the dose assignment. Various extensions of the CRM have been proposed, including dose escalation with overdose control [8], time-to-event CRM [9], Bayesian model averaging CRM [10], partial order CRM [11], and bivariate CRM [12]. For a comprehensive review of the CRM and related methods, see the book by Cheung [13]. Compared to algorithm-based designs, modelbased designs typically have superior operating characteristics. However, because modelbased designs require repeated model fitting and estimation, they are more complicated to implement and many practitioners view the decisions of the model-based designs as coming from a "black box".
A class of new designs, known as "model-assisted" designs, has drawn substantial attention from practitioners because of its simplicity and desirable performance. Similar to model-based designs, these designs utilize probability models for efficient decision making, but their dose assignment rules can be pre-tabulated before the onset of the trial in a fashion similar to the assignment rules of the algorithm-based designs. Examples of model-assisted designs include the modified toxicity probability interval (mTPI) design [14], Bayesian optimal interval (BOIN) design [15] and keyboard design [16]. In particular, the keyboard design is a seamless improvement of the mTPI design that addresses the latter's overdosing issue.
The keyboard design keeps the simplicity of the mTPI design, but has higher accuracy to identify the MTD and substantially better overdose control [16].
The objective of this paper is to investigate the statistical properties of the keyboard design, including optimality, coherence and consistency, which are not studied in the introductory paper [16]. Establishing these statistical properties is not only of theoretical interest, but also of practical importance because it assures practitioners that the design has certain guaranteed desirable behaviors. Specifically, we show that the decisions of the keyboard design are optimal under the "0-1" loss function; the design is also long-memory coherent in the sense that it never escalates (or de-escalates) the dose if the observed toxicity rate at the current dose is higher (or lower) than the target toxicity rate; last, the keyboard design is consistent in that its dose assignment converges to the target dose under large samples.
Armed with sound statistical properties and competitive operating characteristics, we further extend the keyboard design to handle drug-combination trials. Numerous designs have been proposed for finding the MTD for drug-combination trials: a design based on the order of the restricted inference [17], a six-parameter model with continuous doses [18], a copula-type regression model [19], latent contingency tables [20], a Bayesian hierarchical model [21], sequential dose-finding strategy [ proposed keyboard combination design enjoys the same statistical properties, while retaining the same transparency and ease of implementation as the single-agent keyboard design. A simulation study based on random and objective scenarios shows that the keyboard combination designs have very competitive performance compared to that of some existing, more complicated designs. To facilitate the use of the designs, we also develop an R Shiny application for implementing the keyboard combination designs that will be freely available at http://www.trialdesign.org.
The rest of this paper is organized as follows. In Section 2, we review the keyboard design and study its statistical properties. In Section 3, we extend the keyboard design to two-agent combination trials. In Section 4, we propose a novel algorithm to generate random two-dimensional dose-toxicity scenarios, and conduct extensive numerical studies to investigate the operating characteristics of the keyboard combination design based on the randomly generated scenarios. In Section 5, we illustrate how to use the R Shiny application to design a drug combination trial. We provide some conclusions in Section 6.
2 The keyboard design and its statistical properties

The keyboard design
The keyboard design is a Bayesian dose-finding design originally proposed for phase I singleagent trials. Let φ denote the target toxicity rate specified by an investigator, and p d ∈ (0, 1) be the toxicity probability of the dose level d. The keyboard design starts by specifying a target toxicity interval I target = (φ− 1 , φ+ 2 ) (referred to as the target key), where 1 and 2 denote tolerable deviations from φ such that any dose with a toxicity probability within that interval can be practically viewed as the MTD. Then, the keyboard design populates this interval toward both sides of the target key, forming a series of equally-wide keys that span the range of 0 to 1 (see Figure 1). For example, given φ = 0.2, the target key may be defined as (0.15, 0.25) with 1 = 2 = 0.05. Then, one key of width 0.1 is formed on the left side of the target key, i.e., (0.05, 0.15), and seven keys of width 0.1, i.e., (0.25, 0.35), · · · , (0.85, 0.95), are formed on the right side of the target key. We denote the resulting intervals/keys as I 1 , · · · , I K . In some cases, the DLT probability values at the two ends (e.g., < 0.05 or > 0.95 in the example) may not be covered by the keys because they are not wide enough to form a key of width 0.1. As explained in Yan et al. [16], ignoring these "residual" DLT probabilities at the two ends does not pose any issue for decision making.
Assume that at a decision time, n d patients have been treated at the current dose d, and y d of them have experienced DLT. A standard beta-binomial model is posited, where Unif(0, 1) serves as a non-informative prior distribution for p d . Given the observed data D d = (n d , y d ), the posterior distribution of p d follows a beta distribution, p d |D d ∼ Beta(y d + 1, n d − y d + 1). To make the dose assignment, the keyboard design identifies the strongest key I max that has the largest posterior probability, i.e., In other words, the strongest key I max represents the interval where the true value of p d is most likely located. Graphically, the strongest key is the one with the largest area under the posterior distribution curve of p d (see Figure 1). By continuously identifying the strongest key I max after each cohort, the dose-assignment rule for the next new cohort is described as follows: • If the strongest key is on the left side of the target key (denoted by I max ≺ I target ), escalate the dose to the next higher level; • If the strongest key is the target key (denoted by I max ≡ I target ), retain the current dose level and stay at that level; • If the strongest key is on the right side of the target key (denoted by I max I target ), de-escalate the dose to the next lower level.
Because the decision of dose escalation and de-escalation only depends on the local data at the current dose, the decision rule of the keyboard design can be pre-tabulated, making the design transparent and easy to implement. Table 1 shows the dose escalation and deescalation rule for targets φ = 0.2 and 0.3.

Statistical properties
In this section, we study the statistical properties of the keyboard design. Let a ∈ {E, R, D} denote the three possible decisions, where E, R, and D denote escalating, retaining and deescalating the dose, respectively, and let k * be the index of the key to which p d belongs. As the keyboard design is consisted of two sequential actions (i.e., identify the strongest key first and then make dose-assignment decisions according to the strongest key), under the Bayesian decision theoretic framework, we define a "0-1" loss function as follows, L(a,k | I k * ) = 1 − 1 (k | I k * ) 2 (a |k, I k * ), where 1 (k | I k * ) is the loss function corresponding to the action of identifying the strongest key, given by with 1{·} being the indicator function, and 2 (a |k, I k * ) is the loss function corresponding to the action of making dose-assignment decisions according to the strongest key, given by 1, a = E and Ik ≺ I target 1, a = R and Ik ≡ I target 1, a = D and Ik I target 0, otherwise .
Since both 1 (k | I k * ) and 2 (a |k, I k * ) are "0-1" functions, L(a,k | I k * ) is also a "0-1" loss function. Given the prior p d ∼ Unif(0, 1), it is easy to obtain that the prior model probabilities of I k are equal and that under each I k , p d follows a uniform prior distribution with the support I k . The expected loss function is then and a is taken as the correct action with respect to the location of I max . Therefore, we conclude that the decision rules of the keyboard designs correspond to an optimal rule. Theorem 1 (Optimality). The dose escalation and de-escalation rule in the keyboard designs is optimal in the sense that it minimizes the posterior expected "0-1" loss function L(a,k | D d ).
Cheung [13] introduced the concept of coherence and defined it as a design property by which dose escalation (or de-escalation) is prohibited when the observed toxicity rate in the most recently treated cohort is larger (or smaller) than the target toxicity rate. Liu and Yuan [15] extended that concept and defined two different types of coherence: short-memory coherence and long-memory coherence. They referred to the coherence proposed by Cheung as short-memory coherence because it concerns the observations from only the most recently treated cohort, ignoring the observations from the cohorts that were previously treated.
Long-memory coherence is defined as a design property by which dose escalation (or deescalation) is prohibited when the observed toxicity rate in the accumulative cohorts at the current dose is larger (or smaller) than the target toxicity rate. From a practical viewpoint, long-memory coherence is more relevant because when clinicians determine whether a dose assignment is practically plausible, they almost always base their decision on the toxicity data that have accumulated from all patients, rather than only the most recent cohort (which rarely includes more than three patients) treated at that dose. In addition, phase I trial patients are very heterogenous and the observations from a cohort (≤ 3 patients) are highly variable. Thus, it can be undesirable to evaluate whether dose escalation or de-escalation is appropriate based on only a single cohort. For example, suppose the target DLT rate φ = 0.3 and at the current dose, one of the latest enrolled 3 patients experienced DLT but none of 6 patients previously treated at the same dose had DLT. As the overall observed DLT rate at the current dose is 1/9, escalating the dose should not be regarded as an inappropriate action, although it violates the short-memory coherence. It can be shown that the keyboard design is long-memory coherent.
Theorem 2 (Coherence). The keyboard design is long-memory coherent in the sense that the probability of escalating (or de-escalating) the dose if the observed toxicity rate in the accumulative cohorts at the current dose is greater (or less) than the target toxicity rate, The proof of Theorem 2 is provided in Appendix B, which relies on an equivalent form of the keyboard designs (as shown in Lemma A). However, since the keyboard designs integrate all the accumulative data at the current dose when making decisions and does not allow dose skipping, it is not necessarily short-memory coherent.
Coherence is a finite-sample property. Theorem 3 shows that the keyboard design has a desirable large-sample convergence property.
Theorem 3 (Convergence). As the number of treated patients goes to infinity, the dose assignment of the keyboard design converges almost surely to the dose level d with The proof of Theorem 3 is given in Appendix C. Although a good large-sample property does not guarantee that a design will perform well in the small sample sizes of phase I clinical trials, it provides a basic requirement for the design to perform well. If a design does not converge to the target dose under large samples, it most likely will not perform well when finding the target dose in small sample sizes.

Drug-combination keyboard designs
We now extend the keyboard design from single-agent trials to combination trials with J levels of drug A and K levels of drug B. Let p jk denote the toxicity probability of the combination of dose level j of drug A and dose level k of drug B, denoted as (j, k), 1 ≤ j ≤ J, 1 ≤ k ≤ K. Let n jk and y jk respectively denote the number of patients treated and the number of patients who have experienced toxicity at d = (j, k). As the decision of dose escalation/de-escalation of the single-agent keyboard design is solely based on the local data at the current dose, its dose transition rule (as shown in Table 1) can be directly applied to drug-combination trials. A single-agent trial using the keyboard design can be summarized in the following three steps.
Step 1. The trial starts by treating the first cohort of patients at the lowest dose combination (1, 1).
Step 2. At the current dose combination (j, k), given the observed data D jk = (n jk , y jk ), we identify the strongest key I max based on the posterior distribution of p j,k : If I max ≺ I target , we escalate the dose; if I max I target , we de-escalate the dose; otherwise, if I max ≡ I target , we stay at the current dose.
Step 3. The process continues until the prespecified maximum sample size is achieved.
For combination trials, the difficulty is that when we decide to escalate (de-escalate) the dose, there is more than one option: we can escalate (de-escalate) either the dose level of drug A, the dose level of drug B, or both simultaneously. To address this issue, we define admissible dose escalation and de-escalation sets A E 1 = {(j + 1, k), (j, k + 1)} and In addition, we define admissible dose escalation and de-escalation sets that include diagonal dose movements, Five dose assignment algorithms for drug-combination trials have been considered.
Algorithm 1: Fixed non-diagonal escalation/de-escalation (key1) • Escalation: escalate to the dose combination that belongs to A E 1 and has the highest value of Pr(p jk ∈ I target |D jk ).
• De-escalation: de-escalate to the dose combination that belongs to A D 1 and has the highest value of Pr(p jk ∈ I target |D jk ).
Algorithm 2: Fixed non-diagonal escalation and diagonal de-escalation (key2) • Escalation: escalate to the dose combination that belongs to A E 1 and has the highest value of Pr(p jk ∈ I target |D jk ).
• De-escalation: de-escalate to the dose combination that belongs to A D 2 and has the highest value of Pr(p jk ∈ I target |D jk ).
Algorithm 3: Fixed diagonal escalation/de-escalation (key3) • Escalation: escalate to the dose combination that belongs to A E 2 and has the highest value of Pr(p jk ∈ I target |D jk ).
• De-escalation: de-escalate to the dose combination that belongs to A D 2 and has the highest value of Pr(p jk ∈ I target |D jk ).
Algorithm 4: Randomized non-diagonal escalation/de-escalation (key4) • Escalation: randomly escalate to the dose combination that belongs to any one of two combination components in A E 1 with randomization probabilities proportional to their respective posterior probabilities Pr(p jk ∈ I target |D jk ).
• De-escalation: de-escalate to the dose combination that randomly belongs to any one of two combination components in A D 1 with probabilities proportional to their respective posterior probabilities Pr(p jk ∈ I target |D jk ).
Algorithm 5: Randomized diagonal escalation/de-escalation (key5) • Escalation: randomly escalate to the dose combination that belongs to any one of two combination components in A E 2 with randomization probabilities proportional to their respective posterior probabilities Pr(p jk ∈ I target |D jk ).
• De-escalation: randomly de-escalate to the dose combination that belongs to any one of two combination components in A D 2 with the randomization probabilities proportional to their respective posterior probabilities Pr(p jk ∈ I target |D jk ).
For algorithms 1-3 with fixed allocation rules, if there are multiple optimal dose combinations with the same value of Pr(p jk ∈ I target |D jk ), we randomly choose one with equal probability.
The trial is completed when the maximum sample size is reached. Given all observed data, we use matrix isotonic regression [30] to obtain the estimate of p jk and select the MTD as the combination with a toxicity estimate that is closest to the target. When there are ties, we randomly choose one as the MTD. For patient safety, during trial conduct, we apply the following overdose control rule: whenever a dose satisfies Pr(p jk > φ|D jk ) ≥ c (i.e., the dose is overly toxic), where c is a prespecified probability cutoff, we eliminate that dose from the trial. In addition, we impose an early stopping rule: if the lowest dose combination (1, 1) is overly toxic, i.e., Pr(p 11 > φ|D 11 ) ≥ c, we terminate the trial early. Throughout the paper, we set c = 0.95.

Numerical study 4.1 A random matrix scenario generator
The evaluation and comparison of phase I designs are often based on a limited number of prespecified dose-toxicity scenarios, which is prone to favor a specific design intentionally or unintentionally. To avoid that issue, Clertant and O'Quigley [31] proposed a pseudo-uniform algorithm to generate random toxicity scenarios for more objective and unbiased comparisons for single-agent phase I designs. To the best of our knowledge, there is no accessible algorithm for generating random scenarios for a two-dimensional dose matrix, where the partial order constraint for a two-dimensional toxicity space poses major difficulty. To fill this gap, we have developed a fast and efficient algorithm to generate random combination scenarios as described below.
Algorithm: a random matrix scenario generator Step 1: Uniformly choose a dose level (j, k), 1 ≤ j ≤ J, 1 ≤ k ≤ K from the J × K drugcombination space, and set p jk = φ.
Step 2: Given the dose combination (j, k), specify a "pivotal" path from p 11 to p 1K : The doses in the pivotal path are monotonically ordered in toxicity and partition the J ×K dose matrix into upper block (UB above the path) and lower block (LB under the path) matrices.
Step 3: Generate the toxicity probabilities for the doses on the pivotal path as follows: generate an ordered uniformly-distributed random sample of toxicity probabilities with length j + k − 2 from Unif(0, φ) and assign them to (p 11 , · · · , p j1 , · · · , p jk−1 ), and generate an ordered uniformly-distributed random sample with length J + K − j − k from Unif(φ, p max ) and assign them to (p jk+1 , · · · , p jK , · · · , p JK ), where p max is the upper bound for the maximum toxicity rate and is generated from a Beta distribution with mean µ and variance µ(1 − µ). In this paper, we take µ = 1 − exp{−(J × K)/8}, which can satisfactorily provide the upper bound for p JK .
The resulting toxicity probability matrix meets the requirement of a partial order for drug combinations. For illustration of the proposed random matrix scenario generator, a concrete example of generating a 3 × 3 toxicity probability matrix is given in Appendix D. Under the above algorithm, one dose has the toxicity probability equal to the target φ (i.e., step 1); however, as the toxicity probabilities of other doses are randomly generated, some may be fairly close (although not exactly equal) to φ. As a dose with the toxicity probability within [φ − 1 , φ + 2 ] is deemed acceptable (as the MTD), and the proposed algorithm is able to produce multiple MTDs in the randomly generated toxicity probability matrix.

Simulation configuration
We conduct simulation studies to investigate the performance of the drug-combination keyboard designs based on the random matrix scenarios proposed in Section 4.1. We consider  Table 1. The maximum sample size is set at 48 for scenarios with 2 × 4 dimension and at 60 for scenarios with 3 × 5 and 4 × 4 dimensions, where the cohort size in all cases is fixed at one for both designs. For fair comparisons, we apply the early stopping rule as discussed in Section 3 to both designs.

Simulation results
Tables 2 -4 present simulation results for the 2 × 4, 3 × 5 and 4 × 4 drug combinations, respectively. For a comprehensive comparison, we calculate the following performance metrics across each 1000 random scenarios for POCRM and the keyboard design.  (Table 3), the PCS of POCRM is 25.1%, while key1 to key5 have 8.0%, 7.8%, 7.4%, 7.8% and 7.0% more chances to identify the correct MTDs, respectively. The PCS difference among five versions of the keyboard combination design is not significant. Overdose and underdose control : The percentage of overdose assignment is defined as the percentage of patients who are assigned to a dose combination with a true toxicity probability that is larger than φ + 2 . The percentage of underdose assignment is defined as the percentage of patients who are assigned to a dose combination with a true toxicity probability that is less than φ − 1 . Such two metrics respectively quantify the safety and conservativeness of a design. For overdosing, in general, the POCRM allocates fewer patients to doses above the MTD than the keyboard designs. However, for underdosing, the POCRM tends to treat more patients at doses below the MTD.
Incoherent index : Long-memory incoherence in the escalation percentage is defined as the percentage of dose escalation withp jk > φ. Such a metric can partly quantify the aggressiveness of a design. According to Table 5, we can conclude that the POCRM is not long-memory coherent. A specific example showing the incoherence (including both short and long memory) of the POCRM can be found in Appendix E. For example, after patient 7 was treated at dose (1, 4), 1/1 patient had DLT at that dose, and the POCRM escalated the dose to (3, 4). When patient 9 was treated at dose (1, 4), 1/2 patients had DLT, and the POCRM again escalated the dose to (3, 3).
In summary, the simulation results show that the keyboard combination designs generally outperform the POCRM in terms of MTD identification accuracy and treatment efficiency.
Among the five versions of keyboard designs, the designs (key1-key3) based on fixed allocation rules usually lead to more patients being treated at the MTDs than those (key4-key5) that use randomization. We recommend key1 and key2 for general drug-combination trials.
While key2 has larger admissible dose escalation and de-escalation sets, key1 tends to be more efficient yet slightly more aggressive than key2.

Web application
To facilitate the use of the keyboard combination design, we develop an easy-to-implement web application using R Shiny. Figure 4 shows the graphical user interface of the application.
After users input their design parameters (e.g., the maximum sample size, cohort size, and the dose combination scenarios), the web application generates the operating characteristics of the keyboard combination design that can be included in the trial protocol. When conducting the trial, users can simply choose the "Next Dose" button, and the R Shiny application will recommend the dose combination to be assigned to the next cohort of patients based on the updated accumulated information. Our web application will be freely available at http://www.trialdesign.org. A reference manual is also provided at that website for illustration and implementation.

Conclusion
The statistical properties of the keyboard design have been comprehensively investigated in this paper. In particular, we have shown that the keyboard design possesses optimal allocation rules such that the posterior expected "0-1" loss function can be minimized. The dose transition is long-memory coherent in the sense that when the observed toxicity rate at the current dose is larger (smaller) than the target toxicity rate, the keyboard design prevents the next patients from receiving overly toxic (subtherapeutic) doses. In addition, the keyboard design exhibits a convergence property that is similar to that of the standard interval designs, and its asymptotic patient allocation converges to the target dose level.
We have further proposed a class of two-dimensional keyboard designs for dose finding in   Strongest key Target key Figure 1: An example based on the observed data (y d , n d ) = (2, 5) for the target toxicity rate of 0.2 to demonstrate the target key I max and the strongest keys I target . In this example, the strongest key (i.e., (0.35, 0.45)) lies on the right side of the target key (i.e., (0.15, 0.25)), so dose escalation is warranted.

A Lemma A: Simplified rule for the keyboard designs
The keyboard designs introduced in Section 2 adopt K keys I 1 , . . . , I K for decision making.
In this Lemma, we show that the decision-making procedure of the keyboard designs can be further simplified to designs using only three keys.
Three cases are considered.
Then, according to the unimodality of the binomial likelihood, it is easy to obtain that g(ξ ) < g(ξ 1 ) < g(ξ 0 ) and thus Pr( Similarly, given a key k with I k I k † +1 (or K 2 ), we can also show that Pr( Therefore, the keyboard designs based on three keys imply those based on K keys in terms of dose retainment.
if I max ≡ I target , then we immediately obtain that 0 = argmax i∈{0,1,2} Therefore, the keyboard designs based on K keys also imply those based on three keys in terms of dose retainment.
• For dose escalation, {Pr(K i | D d )}, then g(ξ 1 ) > g(ξ 0 ) > g(ξ 2 ) which is due to the unimodality of the binomial likelihood, leading to I max ≺ I target . Therefore, the keyboard designs based on three keys imply those based on K keys in terms of dose escalation.
if I max ≺ I target , then using the unimodality of the binomial likelihood again, we obtain that g(ξ 2 ) < g(ξ 0 ) < g(ξ 1 ) < g(ξ ), where Pr(I max | D d ) ∝ g(ξ ), leading to Therefore, the keyboard designs based on K keys also imply those based on three keys in terms of dose escalation.
• For dose de-escalation, it can be shown similarly that the keyboard designs based on three keys are equivalent to those based on K keys in terms of dose de-escalation.

B Proof of Theorem 2 -Coherence
Proof. According to Lemma A, the decision of the keyboard designs is solely based on the maximum value among {g(ξ 1 ), g(ξ 0 ), g(ξ 2 )}, or equivalently, among {Pr( we have the following possible scenarios. • Ifp d ≤ φ − 1 , according to the unimodality of the binomial likelihood, we always have g(ξ 2 ) < g(ξ 0 ), which indicates that Pr(K 2 | D d ) < Pr(K 0 | D d ), then the chance for dose de-escalation is zero.
• Ifp d ≥ φ + 2 , according to the unimodality of the binomial likelihood, we always have g(ξ 1 ) < g(ξ 0 ), which indicates that Pr(K 1 | D d ) < Pr(K 0 | D d ), then the chance for dose escalation is zero.
• If φ ≤p d < φ + 2 , we define the following difference function, Differentiating d( 1 , 2 ) with respect to 1 , we have is an increasing function of 1 , and we only need to consider the case when 1 = 0. That is, if we can show that d(0, 2 ) > 0 then based on the monotonicity of d( 1 , 2 ), we have d( 1 , 2 ) > d(0, 2 ) > 0.
When 1 = 0, the difference function reduces to Again, differentiating d(0, 2 ) with respect to 2 , we have Denote the log-likelihood ratio of φ + 2 versus φ − 2 as which is an increasing function with respect to y d . Sincep d ∈ (φ, φ + 2 ) and y d is a integer value, it suffices to show that d(0, 2 ) > 0 with D d = (y d + 1, n d ),p d = (y d + 1)/n d , and φ = y d /n d .
• If φ− 1 <p d < φ, based on a mimicking approach of the case in which φ ≤p d < φ+ 2 , it can be shown that Pr(K 2 | D d ) < Pr(K 0 | D d ) and the chance for dose de-escalation is zero.
Based on the above argument, the keyboard design is long-memory coherent.

C Proof of Theorem 3 -Convergence
Proof. In what follows, we provide a heuristic proof of the convergence of the keyboard designs. As the sample size increases, it is well known that the posterior distribution of p d under the beta-binomial model asymptotically converges to a normal distribution: From the above, we see that the mode of the asymptotic distribution is a consistent estimate of p d . That is, under large samples, the strongest key I max converges to the key I k with p d ∈ I k . Therefore, based on the keyboard design rules, • if I max ≺ I target , we escalate the dose, and I max , the strongest key of the next dose level, is on the right side of I max ; • if I max I target , we de-escalate the dose, and I max , the strongest key of the next dose level, is on the left side of I max .
After a long run, as long as the next dose level hits the dose with p d ∈ I target , according to the asymptotic posterior distribution of p d , we have Pr(I max = I target | D d ) = 1 as n d → ∞.
As a result, the asymptotic dose movement of the keyboard designs can only settle on a dose with the toxicity probability p d within the target key I target .
D An example of generating a random 3 × 3 toxicity probability matrix.
For illustration of the proposed random scenario generator for drug-combination trials, we consider a 3 × 3 toxicity probability matrix with the target toxicity rate φ = 0.2,   p 11 p 12 p 13 p 21 p 22 p 23 p 31 p 32 p 33   .
According to the proposed algorithm as described in Section 4.1, Step 1: Randomly choose an element from the 3 × 3 matrix with an equal probability (of 1/9) and set it as φ. For instance, dose combination (2, 2) is selected, and now, the probability matrix is The maximum sample size is 40, and we utilize "pocrm.sim" in the R package "pocrm" to simulate one trial based on the random seed of 2. The other implementation details of the POCRM are exactly the same as those used in the reference manual of "pocrm".
We provide step-by-step movements of POCRM in Table A1. From the table, we can see that for this simulated trial, there are 4 incoherent dose movements.  The first six patients were treated in the start-up phase. Actions E, R, D denote the decisions of dose escalation, retainment, and de-escalation, respectively. The incoherence indicator is calculated based on the estimated toxicity order.