{'fraction_non_alphanumeric': 0.02694578636607622, 'fraction_numerical': 0.007192699946323135, 'mean_word_length': 4.730161984826737, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 6, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Bayesian Sequential Design Based on Dual Objectives for Accelerated Life Tests

Traditional accelerated life test plans are typically based on optimizing the C-optimality for minimizing the variance of an interested quantile of the lifetime distribution. The traditional methods rely on some specified planning values for the model parameters, which are usually unknown prior to the actual tests. The ambiguity of the specified parameters can lead to suboptimal designs for optimizing the intended reliability performance. In this paper, we propose a sequential design strategy for life test plans based on considering dual objectives. In the early stage of the sequential experiment, we suggest to allocate more design locations based on optimizing the D-optimality to quickly gain precision in the estimated model parameters. In the later stage of the experiment, we can allocate more samples based on optimizing the C-optimality to maximize the precision of the estimated quantile of the lifetime distribution. We compare the proposed sequential design strategy with existing test plans considering only a single criterion and illustrate the new method with an example on fatigue testing of polymer composites.


Background
For many long life products or systems, accelerated lifetime tests (ALTs) [1,2] are broadly used to accelerate the failure process by exposing the units under harsher conditions than usual and to collect timely information for effectively

Related Literature
Traditional non-Bayesian methods for designing an ALT are based on properties of maximum likelihood (ML) estimators. Meeker and Escobar [1] provided a general guideline for planning life tests to obtain the precise prediction at the normal use condition. Several work such as Chernoff [3] and Meeker and Hahn [4,5] studied optimum designs and compromise plans and outlined practical guidelines for planning an efficient ALT. Some recent developments on ALT include Ye et al. [6], Pan and Yang [7], Tsai et al. [8], and Ng et al. [9]. Escobar and Meeker [2] provided a review on the accelerated test models. Limon et al. [10] provided a review on planning and analysis of the accelerated tests for assessing reliability.
Bayesian techniques have been broadly used in design of experiments. Bayesian methods assume prior distributions for the unknown parameters and the inference of unknown quantities are based on the posterior distribution of the parameters based on the observed data. Chaloner and Verdinelli [11] provided a comprehensive review of Bayesian experimental design techniques for linear and non-linear models, among which Bayesian D-optimal designs and Bayesian C-optimal designs are popular choices that have been used broadly in reliability test plans. Zhang and Meeker [12] developed a Bayesian test plan based on minimizing the pre-posterior expectation of the posterior variance over the marginal distribution of all possible test data. Hong et al. [13] proposed new Bayesian criteria based on the large-sample approximations, which offer simplified solutions to the Bayesian test plans. Sequential test plans have been popular for reliability tests that are either very expensive or very time-consuming.
Instead of determining the design locations (i.e., the stress levels) for all test units prior to the experiment, the test in a sequential design is determined and implemented one at a time given the current information gained from the observed data. At each step, the next optimal design point is determined by optimizing a design criterion summarized over the posterior distribution of the parameters given all the data observed so far. It was first introduced in the non-Bayesian framework using the ML estimator with the D-or C-optimality criterion for designing nonlinear experiments [14,15].
In the Bayesian framework, sequential designs based on considering the D-optimality criterion were used often. For example, Dror and Steinberg [16] developed the Bayesian sequential D-optimal designs for generalized linear models.
Roy et al. [17] and Zhu et al. [18] proved convergence properties of the Bayesian sequential D-optimal designs for different forms of models.
In planning for the fatigue tests for polymer composites, King et al. [19] proposed an optimum test plan for the constant amplitude cyclic fatigue testing of polymer composites material. Lee et al. [20] proposed a sequential Bayesian C-optimal test plan for the polymer composites fatigue testing, which selected the sequential design points based on optimizing the posterior asymptotic variance of an interested quantile across a range of normal use conditions.

Overview
The remaining of the paper is organized as follows. Section 2 provides the basic background on the ALT plans. Section 3 discusses the Bayesian sequential design which helps improve the efficiency of the fatigue testing plan when imprecise prior knowledge on the planning parameter values is available. Section 4 proposes a new method based on considering dual objectives in Bayesian sequential design, where D-optimality is employed initially to quickly improve the precision of the estimated model parameters in the early runs and followed with C-optimal sequential runs to further improve the precision of the estimated reliability quantile at the specified normal use conditions. The new method will be illustrated in Section 5 using the polymer composites fatigue testing example, and compared with the Bayesian sequential designs that consider only a single criterion. In the end, Section 6 offers additional discussion and conclusions.

Accelerated Life Test Plans
In this section, we give an introduction to the general ALT plans. We focus on the ALT plan with a single accelerating factor such as in the fatigue testing example for polymer composites. The ALTs involve testing units at different stress levels of the accelerating factor for quickly obtaining failure information for the product. Hence, the ALT plans require the determination of the levels of the accelerating factor to be implemented in the tests (i.e., the stress levels) and the sample size at each level (how many units to be tested at each stress level). Similar to the regular design of experiments, where optimal designs (e.g., D-, A-, G-, or I-optimality [21]) are often chosen for achieving the best precision of some interested quantities (e.g., model parameters or predictions throughout the design space) under an assumed response model, the ALT plans are often based on optimizing the precision of some interested reliability metric, such as a quantile of the lifetime, given an assumed ALT model. Because the data is collected under the elevated stress levels to make inference on reliability at the normal use conditions, extrapolation on the accelerating factors are naturally involved in ALT plans and hence requires strong assumptions on the specified ALT models.
For modeling, the log-location-scale family of distributions (e.g., the Weibull and Lognormal) are often used to model the accelerated lifetime. The models often assume a common scale parameter but allowing the location parameter to change at the different stress levels through a parametric model. An optimal ALT plan is often determined based on minimizing the asymptotic variance of a reliability metric under the assumed model, which is dependent on the model parameters (e.g., the scale and location parameters) that are unknown prior to the data collection. Hence, the efficiency of the ALT plan is largely dependent on the choice of the parameter values at the planning stage.
For the polymer composites fatigue test example, let T denote the cycles to failure which is assumed to follow a log-location-scale family of distribution with the cumulative distribution function (cdf) and the probability density function (pdf) given in the forms below In the equations above, µ and ν are the location and scale parameters, respectively. The Φ(·) and φ (·) are the cdf and pdf of the standard normal distribution. Let θ = (µ, ν) denote the vector of the unknown parameters included in the model. The scale parameter ν is often assumed to be constant over the different stress levels, while the location parameter µ = µ β (x) is assumed to be dependent on the stress level x through model parameters β . A physically motivated nonlinear model [22] is used to model the relationship between the cycles-to-failure and the stress level as given in the form In Eqn. In reliability analysis, the quantile of the lifetime in the lower tail of the distribution (p ≤ 0.5) is often of interest for capturing important reliability characteristics. Let ζ p,x denote the pth quantile at the stress level x, which is related to the ALT model parameters through the following form where z p is the pth quantile of the standard log-location-scale distribution, and x is a specified stress level under the normal use condition. The pth quantile of the lifetime can be estimated by substituting the model parameters in Eqn.
(2) by its estimators,β andν. The ALT plan can be chosen based on minimizing the asymptotic variance of the estimated pth quantile life, denoted by AVar log ζ p , x . Considering in many real applications the stress level can vary across a range of use conditions, a use stress profile shown in Figure 1 is considered for the polymer composites fatigue test plan. The use condition stress levels range between all the use stress levels with x i = q i σ ult for q i ∈ [0.05, 0.25] and {w 1 , · · · , w k } denote their relative frequencies in the use profile with ∑ K k=1 w k = 1. The total weighted asymptotic variance at all use stress levels is defined as for capturing the overall asymptotic variance throughout all possible use conditions. The ML estimation is often used to calculate the weighted total asymptotic variance given in Eqn. (3). The ML approach estimates the parameters based on maximizing the likelihood function of the parameters given the observed data. For the polymer composites fatigue test, the failure time data were collected within a predetermined test duration, and some test units had not failed by the end of the test period, which were censored observations. Let (x i ,t i , δ i ) represent the observed data for the ith test unit, where δ i is the censoring indicator and δ i = 1 if the ith unit is censored and δ i = 0 otherwise, and t i is the failure time when δ i = 0 and the censoring time when δ i = 1. Given the observed test data (x n , t n , δ n ), where x n = (x 1 , · · · , x n ) , t n = (t 1 , · · · ,t n ) , and δ n = (δ 1 , · · · , δ n ) , the likelihood function is given by The ML estimate of θ is the solution to the equation ∂ l (θ ) /∂ θ = 0 and the asymptotic variance of the ML estimator, θ is given by Then the total weighted asymptotic variance in Eqn. (3) can be calculated as where Note that using the ML estimation when there is a small number of observed failures with a high censoring rate can result in unstable estimates. If the censoring time is random, then the expectation-maximization approach can be used to obtain more stable estimates of the model parameters (e.g., Park and Lee [23]).
The C-optimal design [11], which minimizes the asymptotic variance of the estimated quantile of the lifetime distribution at a range of normal use conditions measured by Eqn. (6)  [20] based on using a Bayesian sequential design, which selects the next single design point based on optimizing Eqn.
(6) over the posterior distribution of the parameter θ given the currently observed data.
The sequential design framework used in this paper is similar to Lee at al. [20]. Here we provide a brief description of the general framework within the context of the fatigue test plan for the polymer composites. Given a use stress profile, the (n + 1)th design point was selected based on minimizing the average posterior asymptotic variance over the posterior distribution of the model parameters given the first n observations, which is calculated by: where (7), the posterior distribution updated by the first n observations, π (θ |x n , t n , δ n ), can be considered a prior distribution of parameters prior to selecting the (n + 1)th design point. A Markov chain Monte Carlo (MCMC) method was developed in Lee et al. [20] to approximate the posterior distribution π (θ |x n , t n , δ n ), where θ = (A, B, ν), for the selection of sequential optimal design points. Uniform distributions were assumed for both the environmental and physical parameters A and B, as in , and a 1 , a 2 , b 1 and b 2 are constants specified based on the anticipated ranges of the parameters A and B. An inverse gamma distribution with the shape parameter κ and the scale parameter γ was used for ν 2 as it is the conjugate prior for the lognormal distribution, which is one of the commonly used log-location-scale distributions in reliability analysis.
In Lee et al. [20], two algorithms were provided to evaluate the anticipated asymptotic variance in Eqn. (7) and to select the optimal next design location in the sequential Bayesian design. At each iteration of the sequential Bayesian design, the two algorithms are used (i) to draw samples from the posterior distribution π (θ |x n , t n , δ n ) and use the Monte Carlo integration for approximating Eqn. (7), and (ii) to determine the optimal design location that minimizes Eqn. (7) over all possible design points.
To encourage broad applications, an R package named "SeqBayesDesign" [24] was developed to implement the sequential Bayesian design developed in Lee et al. [20]. The package can be applied to determine the sequential Bayesian design for traditional ALTs and the constant amplitude fatigue test based on either the log-normal distribution or the Weibull distribution. Users can easily implement the method by providing the inputs on the historical test data, anticipated use condition levels, the prior information on the model parameters which can be specified in the form of the uniform or normal distribution, and the candidate design points. After specifying the input settings, the optimal design point for the next step will be calculated by the two algorithms.

Sequential Design Under Dual Objectives
In this section, we describe a new sequential design strategy based on dual objectives. We first describe the motivation for considering dual objectives. Typically, the optimal design based on a single criterion can sacrifice its performance on aspects that are not measured by the chosen criterion. For example, the D-optimal designs maximize the informa-tion gain measured by the determinant of the information matrix, |I n (θ , x n ) |. These designs allocate the design points to obtain the most precise estimates of the model parameters based on the data collected at the accelerated stress levels, and may not offer sufficient efficiency when predicting reliability at the normal use conditions which requires extrapolation beyond the range of the observed data. On the other hand, the C-optimal design, which focuses on obtaining the most precise prediction at the specified normal use condition(s), may be suboptimal for precisely estimating the model parameters. In recent decades, it has become more desirable to consider multiple aspects for design selection.
The desirability function approach by Derringer and Suich [25] has been broadly used to choose an optimal design based on combining multiple criteria through a user-defined desirability function that heavily relies on subjective user choices on weighting, scaling and metric form prior to understanding the design performance and potential trade-offs.
Lu et al. [26] introduced the Pareto front approach to design selection and optimization by intentionally separating the objective and subjective selection stages and allows to make informed decisions based on quantitatively understanding the trade-offs and robustness to different user priorities.
We consider dual objectives in the sequential Bayesian design context. Given the sequential designs are often used More specifically, suppose we want to design an N-run sequential Bayesian test plan and we want to choose the first N 1 runs based on the Bayesian D-optimality and the next N − N 1 runs based on the Bayesian C-optimality. Then, among the first N 1 runs, the (n + 1)th run at x n+1 is selected based on maximizing the expected D-optimality over the posterior distribution of the model parameters based on the first n observed test units as given by where |I n+1 (θ , x n+1 ) | is the determinant of the information matrix based on the first n + 1 runs which can be updated from the information matrix for the first n runs, I n (θ , x n ), by the information gained from the additional observation at x n+1 as in Then the optimal design point for the (n + 1)th run is selected by Then from the N 1 + 1 run on, the remaining design locations will be selected based on minimizing the total weighted asymptotic variance of the interested quantile of the lifetime distribution given in Eqn. (6) by seeking the optimal design location at (n + 1)th observation as until all N runs are selected and executed.
The implementation of the dual objective Bayesian sequential design using the R package "SeqBayesDesign" [24] is convenient. This package allows the users to generate the Bayesian sequential optimal design based on a user choice on the optimization criterion between the D-optimality and C-optimality. By changing the optimization criterion among the sequential runs, we can easily implement the dual objective Bayesian sequential design for any desired fractions of the D-optimal and C-optimal runs. In the next section, we will illustrate the proposed method using the polymer composites fatigue testing example and compare a few Bayesian sequential test plans with different fractions of Doptimal runs to the Bayesian sequential designs considering only a single criterion. The R code for implementing the proposed method for the polymer composites fatigue test example is available from the authors upon request.

Application in Polymer Composites Fatigue Testing
In this section, we illustrate the sequential Bayesian design considering dual objectives using the polymer composites fatigue testing example. The original data consisting of 14 observations from a fatigue testing experiment for glass fibers were summarized in Lee et al. [20].  Figure 2 shows the stress-life relationship for all the 14 observations and the subset of three that are considered as limited existing data in our example. We adapted the Bayesian sequential algorithms de-veloped in Lee et al. [20] for considering the dual objectives and implemented using the R package "SeqBayesDesign" [24].    Table 1. Due to the sampling variation, we simulated 100 test plans based on each strategy and summarize the average performance across the 100 simulations to compare the design performance. Figure 3 shows the AVar from Eqn. (6) of the 12 sequential runs for the five design strategies. We can see for the first sequential run, all five designs substantially reduce the AVar by almost the same amount. Starting from the 2nd run, the sequential Coptimal design (shown in the red line) achieves the minimum AVar consistently for all remaining runs. The sequential D-optimal design (shown in the blue line) increases the AVar at the 2nd run and then starts to reduce the AVar steadily afterwards. However, it has consistently higher AVar than the sequential C-optimal design since it focuses primarily on obtaining the most precise estimates of the model parameters instead of making the most precise prediction of reliability at the normal use conditions. However, the difference between the two designs becomes noticeably smaller as more sequential runs are observed. The other three designs (c-e) from Table 1 achieve AVar values between those offered by  Table 1. the D-optimal and C-optimal designs. Generally, the more D-optimal runs generated at the early stage of the sequential designs, the more closely the design performs compared to the 12 run sequential D-optimal design for those early runs that are generated based on D-optimal criterion. But for the later runs generated based on the C-optimal criterion, the design usually offers slightly better AVar than the D-optimal design. On the other hand, the fewer D-optimal runs the design has at the early runs, the faster the AVar value is improved and approaching the best AVar offered by the C-optimal design for all the sequential runs.
In addition, we compare the overall precision of the estimated model parameters among the five design strategies.
Particularly, we adopt the M measure for quantifying the relative error of the estimated parameters from Lee et al. [20]. Particularly, define where θ j represents the jth parameter in the assumed ALT model, andθ j,k represents the estimated value of θ j from the kth simulation trial for k = 1, · · · , K = 100. Hence, m(θ j ) measures the relative mean squared error of the estimate of the jth parameter. Then the M measure, defined as M = ∑ 3 j=1 m (θ j ), measures the total relative mean squared error of all the parameters, which quantifies the overall precision of the estimated model parameters from the ALT model. The smaller M value indicates more precision of the estimated parameters. Figure 4 shows the M measure for five designs shown in Table 1. The sequential C-optimal design has the largest M and hence the least precision of all the estimated parameters for all the 12 sequential runs. This can be expected as the C-optimal design focuses more on obtaining the most precise estimation of the interested quantile at the normal use conditions and less on obtaining the most precise  Table   1. estimation of model parameters. It is interesting that the D-optimal design does not offer the most precise estimation of parameters at the early runs but does catch up later as more runs are implemented. Also, some of the sequential dual objective designs (e.g., design c with 6 D-opt runs followed by 6 C-opt runs) slightly outperform the D-optimal design at early runs (between the 2nd and the 7th sequential runs). This could be resulted from sampling variation and the fact of D-optimality being a better measure of the precision of estimated model parameters for the larger sample cases. As more sequential runs are obtained (8 or more runs), the D-optimal design offers the best precision of estimated model parameters. For our case study, when less than 8 sequential runs are allowed, the dual objective sequential designs offer slightly better precision for both the estimation of model parameters and the prediction at the normal use conditions. Figure 5 shows the fraction of sample allocated to different stress levels for the five designs averaged over the 100 simulation trials. A few patterns can be observed prominently. First of all, all the test plans allocate majority of the runs to the extreme stress levels within the range of consideration for q ∈ [0.35, 0.75]. The sequential C-optimal design places about 2/3 of the total sequential runs to the lowest stress level at q = 0.35. In contrast, the sequential D-optimal design places about 60% of the total runs to the highest stress level at q = 0.75. This is intuitive as the C-optimal aims for improving the estimation at the normal use conditions at much lower stress levels than the design region, and hence tends to allocation more runs closer to the region of prediction. On the other hand, the D-optimal design aims to improve the estimated model parameters, which are estimated more precisely when more failures are observed at the higher stress levels assuming the failure mechanism does not change under more stressed conditions. Other sequential plans based on considering dual objectives show slightly more balanced performance between the two optimal designs considering only a single criterion. But they are consistently much closer to the C-optimal design by allocating more  Table 1. runs closer to the normal use considerations. For all the test plans, there are only a small number of runs (≤ 1%) located at the second lowest stress level at q = 0.4 among the 100 simulation trials. Figure 6 displays the faction of sample allocation at each of the sequential runs across the 100 simulation trials for the five designs. The dark-to-light gray shades indicate large to small frequency of sample allocation at the different stress levels. We can see there is a large middle region in each panel (for each design) shown in the lightest gray indicating a rare sample allocation to the middle stress levels between q = 0.4 to q = 0.7. By contrasting the Coptimal and D-optimal designs, we can see while the earlier runs are more evenly split between the high and low stress levels, the D-optimal design seems to allocate more runs at the higher stress levels for the later runs, while the C-optimal design tends to have more runs at the lower stress levels at the later stage of the sequential experiment.
And this pattern matches our understanding that more tests at the higher stress levels can improve the estimation more efficiently while tests at the lower levels are more helpful for improving the prediction at the normal use conditions.
We also did the comparison between the several strategies based on other data sets, including the Data Set 2 in Lee et al. [20], and observed similar patterns. Hence, the results are not shown here.

Discussion and Conclusions
In this paper, we extend the Bayesian sequential test plans by considering dual objectives when selecting the sequential design points. Bayesian sequential tests are particularly useful when there is little prior information on the lifetime distribution and also tight constraints on the total number of test units or the number of units that can be tested simultaneously. In this case, it is helpful to timely update our understanding as more valuable information is collected during the sequential experiment. We choose to use D-optimality to guide the early design selection for quickly improving the precision of estimated ALT models to help the planning strategy focus in the right region and then switch to C-optimality to seek precise prediction when extrapolating into the normal use conditions.
We apply the dual objective Bayesian sequential design to the polymer composites fatigue test problem, in which case there is limited prior knowledge on the new composites materials and the testing requires using an expensive equipment with limited availability and hence there is little chance of testing multiple units at the same time. Our comparison between the dual objective and single objective Bayesian sequential designs showed more robust and balanced performance of the dual objective design. Especially when very limited test units are possibly to be considered, the dual objective designs have shown to offer noticeably more precise estimation of model parameters than the C-optimal design and better prediction across a range of possible normal use conditions than the D-optimal design. As more test units are affordable, the dual objective designs have shown to catch up with the single criterion optimal design more quickly than the optimal design considering only the other criterion and eventually offer near-optimal performance on both the estimation and prediction.
The implementation of the method is convenient and straightforward with the "SeqBayesDesign" R package [24] for the polymer composites fatigue test example and other applications using similar ALT models. However, the proposed method is very general to be applied to broad applications with different lifetime distributions, ALT models, or even different sequential testing strategies involving blocking or other constraints. The method should also adapt for higher dimensional problems with more than one accelerating factors.