{'fraction_non_alphanumeric': 0.029709035222052066, 'fraction_numerical': 0.004747320061255743, 'mean_word_length': 5.4375, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 1, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Bayesian Lifetime Regression with Multi-type Group-shared Latent Heterogeneity

Products manufactured from the same batch or utilized in the same region often exhibit correlated lifetime observations due to the latent heterogeneity caused by the influence of shared but unobserved covariates. The unavailable group-shared covariates involve multiple different types (e.g., discrete, continuous, or mixed-type) and induce different structures of indispensable group-shared latent heterogeneity. Without carefully capturing such latent heterogeneity, the lifetime modeling accuracy will be significantly undermined. In this work, we propose a generic Bayesian lifetime modeling approach by comprehensively investigating the structures of group-shared latent heterogeneity caused by different types of group-shared unobserved covariates. The proposed approach is flexible to characterize multi-type group-shared latent heterogeneity in lifetime data. Besides, it can handle the case of lack of group membership information and address the issue of limited sample size. Bayesian sampling algorithm with data augmentation technique is further developed to jointly quantify the influence of observed covariates and group-shared latent heterogeneity. Further, we conduct comprehensive numerical study to demonstrate the improved performance of proposed modeling approach via comparison with alternative models. We also present empirical study results to investigate the impacts of group number and sample size per group on estimating the group-shared latent heterogeneity and to demonstrate model identifiability of proposed approach for different structures of unobserved group-shared covariates. We also present a real case study to illustrate the effectiveness of proposed approach.


Introduction
Product reliability analysis is helpful to facilitate life cycle assessment of engineering components and systems. Typically, the lifetime data (e.g., product failure time) in field operation stage provides useful information about product reliability. An accurate predictive model of the product lifetime is of great importance for achieving proactive and effective maintenance policy [1]. The conventional approaches often assume that the lifetime observations are independent and identically distributed. However, this assumption is no longer valid in the application areas of modern industry [2][3][4]. For example, the failure time observations of products manufactured in same batch are correlated and dependent due to similar manufacturing process conditions (e.g., temperature, humidity, pressure), while they can be dramatically different from the other batches [3]. Such violation against the assumption of independent and uncorrelated samples refers to the intra-group dependencies. The intra-group dependencies quantified by group-specific heterogeneity can be partially explained by observed covariates. Nevertheless, the remaining unexplained part is rather substantial and can only be described by group-shared (e.g., batch-specific, region-specific) unobserved covariates [3]. In this work, we use the term group-shared latent heterogeneity to represent the influence of group-shared unobserved covariates on lifetime data. The group-shared unobserved covariates carry important information about product reliability in design and manufacturing stage as well as in field operation stage. Ignoring such information will induce modeling inaccuracy and lead to wrong conclusions about physical nature of products. There is a need to develop a modeling approach with group-shared latent heterogeneity and estimation techniques to account for the influence of group-shared unobserved covariates. A comprehensive understanding of group-shared latent heterogeneity is mandatory for cost effective decisions in both design and production stage as well as in field operation stage. In design and manufacturing stage, successful modeling of group-shared latent heterogeneity can provide quality metrics to product suppliers and deliver useful information to manufacturer about effective design settings and manufacturing variables. This further can help engineers to produce high quality and reliability products. In field operation stage, adequate quantification of group-shared latent heterogeneity can improve the accuracy of product reliability evaluation in field operating environment and further facilitate conditionbased maintenance decisions.
Better modeling of group-shared latent heterogeneity requires a basic understanding of the source of the influencing group-shared unobserved covariates. The unobserved covariates shared within group mainly originate from the following sources in real world. Some of the influencing factors of product reliability are known and explanatory (e.g., material quality indicators, design settings, manufacturing process variables), and they are shared in same group (e.g., same batch in production stage, same region in operation stage). However, they are not available due to the privacy and ownership issue of company warranty data [4]. In addition, the limited sensing techniques and measurement capability as well as high costs make the covariates unavailable, such as operating environment conditions in field operation stage [5]. Moreover, the unobserved covariates can also be unknown/unexplained factors due to the limited knowledge available for new material products [6]. In this work, the data quality issue attributed to the unavailability of covariates information is the main focus and will be addressed via comprehensive investigation.
In addition to the source of group-shared unobserved covariates, it is essential to comprehensively explore the possible structures of such covariates information and further develop an effective modeling approach for characterizing the group-shared latent heterogeneity. There are three main types of covariates information, including qualitative type, quantitative type, and mixed-type [7,8]. The qualitative covariates can be nominal, such as different types of material descriptors and different design configurations [4,7], or ordinal, such as different levels of material quality and different use conditions [8,9]. Such qualitative type of covariates information can be represented by discrete random variables. Moreover, the covariates information can be quantitative and represented by continuous random variables. The quantitative covariates can be the continuous variables for process conditions (e.g., pressure, ambient temperature) in manufacturing stage [7,10,11], or the continuous variables for environmental operating conditions (e.g., loadings, temperature, humidity) in field operation stage [8,12]. Further, the covariates information can also become mixed-type (e.g., both qualitative and quantitative) and represented by a combination of continuous and discrete random variables [13,14]. When such complex and multi-type covariates are shared within same group (e.g., same batch during manufacturing, same region in field operation) and unavailable due to data quality issue, these covariates can be specified as group-shared unobserved covariates and their structures involve multiple different types. The complex structures of group-shared unobserved covariates raise the challenges to the quantification of multi-type group-shared latent heterogeneity in lifetime data analysis.
In the existing literature of product lifetime modeling, most of them only considered the observed heterogeneity [15,16]. Many conventional approaches were developed under the assumption that the lifetime observations within same group are independent. They often used distribution-based methods to characterize the heterogeneity of product lifetime as a whole, or utilized regression-based models to quantify the influence of observed covariates alone. Although the influence of unobserved covariates was considered in some recent studies, these existing methods mainly focused on capturing the intra-individual dependency of product lifetime in a repairable system. A continuous random variable was often introduced to explain part of individual heterogeneity and to represent unit-to-unit variation. These lifetime models with continuous latent variable include frailty model [17] and its multivariate variation [18] in survival analysis, and models of failure time data which consider the influence of unknown use conditions in field operation stage [19]. Among the existing approaches based on frailty model, different types of continuous distribution can be specified for the frailty term, such as Gamma distribution [20][21][22], and generalized inverse Gaussian distribution [23]. Other analytical models are also developed to investigate the continuous type of individual heterogeneity of product lifetime, such as continuous stochastic process with random coefficients [24]. Nevertheless, few studies in product reliability analysis are found to address the intragroup dependency and to capture the influence of group-shared unobserved covariates. Further, limited studies account for the discrete type and mixed-type of latent heterogeneity, especially at group level. There is a gap between the existing approaches and the goal of improved accuracy of lifetime modeling in presence of different structures of group-shared unobserved covariates. It becomes mandatory to resolve the within-group dependency of lifetime data [25] due to the unavailability of group-shared covariates and to address the data quality issues via statistical modeling approach. Moreover, the conventional estimation methods, such as maximum likelihood estimation [26], cannot be applied to estimate the group-shared latent heterogeneity because the latent variables will be integrated out via the marginal approach. Besides, the lifetime observations could become independent given that they are conditioned on the group-shared latent heterogeneity, which also increases the difficulty to capture the intra-group dependency of lifetime data.
To fill the above research gaps, we propose a generic modeling approach to characterize the lifetime data (e.g., product failure time) with both observed heterogeneity and multi-type group-shared latent heterogeneity. Specifically, we comprehensively investigate different structures of group-shared unobserved covariates and different types of their influences, namely, discrete group-shared latent heterogeneity (GSLH-D), continuous group-shared latent heterogeneity (GSLH-C) and mixed-type (e.g., both continuous and discrete types) group-shared latent heterogeneity (GSLH-M). According to the principle of missing information [27], a flexible dependence structure is introduced to capture the latent heterogeneity shared within group. Besides, the proposed model is flexible to analyze failure time data with different specifications of accelerated failure time (AFT) model (e.g., Log-normal, Weibull). Further, estimation algorithm under Bayesian framework is developed to jointly quantify both the observed heterogeneity and multi-type group-shared latent heterogeneity. The developed estimation procedure is able to address the intra-group dependency while membership information and considerable sample size in each group are not required. Moreover, we conduct comprehensive comparison study to demonstrate the effectiveness and model identifiability of the proposed approach.
The rest of this paper is organized as follows. In Section. 2, we present the model details of the proposed approach with multi-type group-shared latent heterogeneity. In Section. 3, model estimation details are elaborated to account for different structures of group-shared unobserved covariates. In Section. 4, we first conduct a numerical study to compare the prediction performance of proposed approach with other alternative approaches. We also investigate the quantification of different types of group-shared latent heterogeneity via empirical analysis. Then, we present a real study to demonstrate the effectiveness of proposed approach and illustrate its ability of identifying the structure of group-shared unobserved covariates. In Section. 5, we draw the conclusive remarks of this work.

Model Formulation
Consider a heterogeneous population of non-repairable units in n groups (e.g., a population of product items manufactured in n batches), there are lifetime observations (e.g. product failure time) within each group , ∀ = 1 … . We can employ AFT model to analyze the lifetime data due to its modeling flexibility and sound model interpretability. The general form of AFT model can be expressed as where is a vector of covariates of j th unit in i th group and is a vector of corresponding coefficients at logarithmic scale. 0 is the average lifetime in absence of the covariates at logarithmic scale. represents the measurement error of j th unit in i th group, which is assumed to be independent with zero mean and bounded variance. With different assumptions on , the AFT model can be specified as different regression models, such as Log-normal regression model and Weibull regression model. These two specifications of AFT model are considered in this work. Further, we manifest the components of the covariates to characterize the intra-group dependency.
The covariates embrace the observed covariates ̃, such as stress factor and operating temperature, and the unobserved covariates, such as manufacturing variables which are not available from warranty data due to data privacy issue. The unobserved covariates contribute to the latent heterogeneity of product lifetime. In this work, we focus on the lifetime modeling of non-repairable products and thus the individual latent heterogeneity due to correlated and repeated measurements becomes negligible. Thus, the latent heterogeneity only consists of the group-shared latent heterogeneity induced by the unobserved covariates shared within each group i (e.g., unavailable material quality variables of suppliers in same region, unavailable manufacturing process variables in same batch). The proposed model can then be expressed as where represents group-shared unobserved covariates for group i and is a vector of corresponding coefficients at logarithmic scale. T̃ captures the influence of observed covariates on the lifetime of unit j in group i at logarithmic scale. Further, we denote = T to quantify the group-shared latent heterogeneity. Different structures can be specified for and consequently becomes multi-type random quantity.
(i) All 's are qualitative random factors with K discrete values, such as the unavailable material quality descriptors with K-levels from different suppliers in same region. = T can then be specified as categorical random variable, i.e., ~ ( , ) where ( , ) refers to categorical distribution with K discrete values, i.e., , ∀ = 1 … , and a vector of . Such discrete type of group-shared latent heterogeneity is annotated as Type I (GSLH-D). (ii) All 's are quantitative random factors, such as the unavailable ambient temperature of product items in same batch during manufacturing process. = T then becomes continuous random variable, i.e., ~ (•) where (•) represents continuous density. Such continuous type of group-shared latent heterogeneity is annotated as Type II (GSLH-C).
is a combination of both qualitative and quantitative factors, such as the unavailable regionshared categorical hotness degree and other continuous environmental conditions in field operation.
= T becomes mixed-type (both continuous and discrete) random variable, i.e., ~ (•) with probability , ∀ = 1 … such that ∑ = 1 =1 . (•)′ represent K different continuous densities. Such mixed-type group-shared latent heterogeneity is annotated as Type III (GSLH-M). Attributed to the multi-type group-shared latent heterogeneity , the proposed modeling approach is generic and flexible to capture the influence of group-shared unobserved covariates with different structures. In addition, the proposed lifetime model is free of any assumptions on the concrete form of failure hazard. The conventional frailty model (with continuous latent variable) [17,18] and mixture model (with discrete latent variable) [28,29] can be perceived as special cases of proposed model. We will elaborate the details of estimation procedure as follows. In this study, we focus on characterizing the intra-group dependency and thus we assume the groupshared latent heterogeneity is not correlated among different groups. In the following subsections, we will elaborate the details of obtaining exact inferences of both unknown parameters and 's as well as hyperparameters for each type of group-shared latent heterogeneity.

Type I Group-shared Latent Heterogeneity: GSLH-D
We first investigate the discrete structure of group-shared unobserved covariates. The group-shared latent heterogeneity can be represented by discrete random quantity. Specifically Since > 0, gamma prior can be assigned. As no conjugate prior is readily available, we can adopt Metropolis-Hasting algorithm [30] to generate the posterior samples of .
Moreover, with reparameterization technique, the proposed model with Type I group-shared latent heterogeneity can be reduced to the mixture regression model [28,29]. The following proposition clarifies the relationship between the mixture regression model and the proposed model. The proposition implies that the baseline hazard becomes subpopulation-specific instead of being shared among different subpopulations. The Type I group-shared latent heterogeneity can be quantified by the subpopulation-specific parameters . The proof details of the proposition can be found in Appendix A.

Type II Group-shared Latent Heterogeneity: GSLH-C
Further, we investigate the continuous structure of group-shared unobserved covariates. The groupshared latent heterogeneity can be captured by continuous random quantity, i.e., ~(•) where (•) represents arbitrary continuous density function with hyper-parameters . When the group-shared latent heterogeneity is Type II, the proposed model in Equation.2 is reduced to the frailty model [17] and the frailty term can be quantified by exp( ). The joint posterior density then becomes The estimates of unknown parameters and 's as well as hyper-parameters can be derived directly. The full conditional posteriors of and can be expressed as

Type III Group-shared Latent Heterogeneity: GSLH-M
The above GSLH-D model and GSLH-C model can be perceived as special cases of the GSLH-M model, with only discrete component and only continuous component respectively. In the GSLH-M model, the group-shared unobserved covariates are combination of both discrete and continuous quantities. The group-shared latent heterogeneity can be represented by mixed-type random variable, i.e., The conditional posteriors for and can then be derived as Based on above derivations, we develop a generalized estimation algorithm for the proposed approach, as shown in Algorithm 1. max is the maximum number of iterations. Without any available prior knowledge of (or ), the prior can be elicited from non-informative Dirichlet conjugate prior, e.g., Jefferys prior with = 0.5 (or = 0.5) [28], which indicates the structure proportions of subpopulations. For the sampling of unknown parameters and random quantities 's, the posterior samples can be drawn conveniently if conjugate prior is available. Otherwise, Metropolis-Hasting algorithm [30] can be applied to generate the posterior samples.

4.1.Numerical Case Study
To illustrate the effectiveness of proposed modeling approach, we first provide simulation study with ground-truth settings to comprehensively investigate different types of group-shared latent heterogeneity and to compare the performance of proposed approach with alternative approaches. In this work, we consider two subpopulations (K=2) for simplicity in the case of mixed-type group shared latent heterogeneity. It can be easily extended to multiple subpopulations via using Dirichlet distribution instead of Beta distribution. In each type of group-shared latent heterogeneity, the continuous component is generated randomly based on normal distribution while the discrete component is generated randomly with a total proportion of 0.35 for subpopulation 1. We consider two different model specifications without losing generality, including Log-normal and Weibull models. Figure.1 shows different structures of groupshared unobserved covariates in Log-normal specification. S1 represents the scenario of Type I groupshared latent heterogeneity due to the influence of discrete group-shared unobserved covariates. S2 refers to the scenario of Type III group-shared latent heterogeneity due to the influence of mixed-type (e.g., both discrete and continuous) group-shared unobserved covariates. S3 represents the scenario of Type II group- Figure 1: Different types of group-shared unobserved covariates, (a) S1, (b) S2, (c) S3 shared latent heterogeneity due to the influence of continuous group-shared unobserved covariates. As shown in Figure. 1, the identifiability of subpopulations is gradually reduced from S1 to S3 and the discrete component finally diminishes in S3. The distribution specifications of different structures of group-shared unobserved covariates in Weibull model are similar. To explore various types of group-shared latent heterogeneity, we specify the values of unknown model parameters and use the ground-truth as evaluation benchmarks. Further, we create 12 simulation scenarios in total with different settings of sample size per group M and number of groups n. We then conduct empirical analysis to investigate all three different structures of group-shared unobserved covariates. We assign less informative priors to all unknown parameters and implement the proposed Bayesian estimation algorithm to estimate model parameters.
Based on the estimated models, we use Kaplan-Meier (K-M) survival curves to evaluate and compare the predictive distribution accuracy of proposed approach and alternative modeling approaches. We consider different simulation scenarios and explore both Log-normal and Weibull model specifications in the comparison study. We use the K-M curve calculated from the ground-truth data as benchmark. To investigate the importance of considering group-shared latent heterogeneity, we compare the proposed model with the model which neglects group-shared latent heterogeneity [15]. We train the models and compare the prediction performance of the reliability function evaluated at new values of observed covariates. As illustrated in Figure. Figure. 2. Overall, the proposed approach can achieve robust estimates of the group-shared latent heterogeneity and improved prediction performance of reliability function in different scenarios. Further, we illustrate the estimated group-shared latent heterogeneity in different simulation scenarios. Based on the results of posterior mean and 95% credible interval, the estimated group-shared latent heterogeneities are significantly different from zero, as shown in Figure. 3. This demonstrates that significant influence of group-shared unobserved covariates exists and cannot be neglected. We then use (a) GSLH-D Weibull model in S1 (b) GSLH-M Weibull model in S2 (c) GSLH-M Log-normal model in S2 (d) GSLH-C Log-normal model in S3 Figure 3: Posterior mean and 95% credible interval of estimated 's with different specifications in different simulation scenarios GSLH-C model with different simulation settings in scenario S3 as an illustration example to show the density of estimated group-shared latent heterogeneity as well as the ground-truth. As illustrated in Figure. 4, the ground-truth values are fully covered by the derived 95% credible intervals in all cases. When sample size per group M becomes larger, the posterior mode of estimated group-shared latent heterogeneity becomes closer to the true value (illustrated by lines with star symbol in Figure. 4). When number of groups n increases, the precision of estimated group-shared latent heterogeneity is also improved (illustrated by blue lines in Figure. Table. 1. The baseline model has worst goodness-of-fit performance in all simulation scenarios with different settings since it fully neglects the group-shared latent heterogeneity. The GSLH-C model among different scenarios achieves better goodness-of-fit performance when different subpopulations become less identifiable and the proportion of continuous component increases. When the continuous component is involved in the group-shared latent heterogeneity (e.g., S2 and S3), both the GSLH-C model and the GSLH-M model achieve improved performance as compared to the GSLH-D model. On the other side, when the group-shared latent heterogeneity is purely discrete, the GSLH-D model achieves the best performance among all models. The adequate model which properly specifies the type of group-shared latent heterogeneity in each scenario can achieve the best goodness-of-fit performance. Based on the performance evaluation results, the proposed approach is able to identify the structure of group-shared unobserved covariates, especially for the discrete type which is not comprehensively addressed in previous studies.
Further, we compare the ability of quantifying the group-shared latent heterogeneity among the GSLH-D, GSLH-C, and GSLH-M models. The posterior mean error, which refers to the average of absolute differences between posterior mean of 's and the true values, is calculated to evaluate the estimation (a) S1 (b) S2 (c) S3 The GLSH-D model achieves smallest posterior mean error in scenario S1 when the group-shared latent heterogeneity is purely discrete type. On the other side, the GSLH-C model results in largest posterior mean error in S1 since it fully mis-specifies the structure of group-shared latent heterogeneity. When the proportion of continuous component increases in the groupshared latent heterogeneity, the posterior mean error of the GLSH-D model becomes significantly larger due to the mis-specification of latent variables. Overall, the GSLH-D model is very sensitive to misspecification error of group-shared latent heterogeneity while the GSLH-C and GSLH-M models achieve more robust estimates of group-shared latent heterogeneity against the mis-specification error. The results of posterior mean errors based on proposed approach are helpful for identifying the structure of group-shared unobserved covariates and examining whether continuous component is involved in the groupshared latent heterogeneity. Moreover, the empirical results of Weibull specification are also presented. The model performance in different ground-truth settings of group-shared latent heterogeneity are summarized in Table. 2. Similar Table 2: DIC comparison results with Weibull model specification to the results of Log-normal specification, the GSLH-D model under Weibull specification exhibits better goodness-of-fit performance in scenario S1 where the group-shared latent heterogeneity is discrete type. When the group-shared latent heterogeneity involves continuous component, such as in scenarios S2 and S3, the performance improvements of the GSLH-D model are significantly affected due to the misspecification of the structure of latent variables. Further, the posterior mean errors of different models under (a) S1 (b) S2 (c) S3 Weibull specification are illustrated in Figure. 6. Based on the empirical results, the GSLH-D model achieves smallest posterior mean errors in S1 while the posterior mean errors of the GSLH-D model become significantly larger in scenarios S2 and S3 due to the mis-specification of the structures of latent variables. Similar to the results of Log-normal specification, the GSLH-D model is more sensitive to mis-specification error while the GSLH-C and GSLH-M models are more robust to the mis-specification of latent variables in different ground-truth scenarios.

4.2.Real Case Study
To further demonstrate the effectiveness and applicability of the proposed modeling approach, we present a real case study on the failure time analysis of test units in tensile tests. In total, 36 tensile tests are performed and the actual lifespans of test units are collected from stress-strain data. To examine the potential group-shared latent heterogeneity, we employ the proposed modeling approach with different types of group-shared latent heterogeneity under Log-normal specification to analyze the lifetime data. We use the model without considering group-shared latent heterogeneity as the baseline model. Without incorporating any covariates information and group-shared latent heterogeneity, the baseline model is simply reduced to Weibull distribution-based model. We then calculate the DIC improvement for each specific model with different types of group-shared latent heterogeneity as compared to the baseline model. Based on the evaluation results, as shown in Figure. 7 (a), there exists group-shared latent heterogeneity and accounting for such significant influence can improve the goodness-of-fit performance considerably. Further, based on the results of three models with different types of group-shared latent heterogeneity, the GSLH-D model achieves the best performance improvement. This indicates that the group-shared unobserved covariates will be highly likely qualitative. We further extract additional information from the tensile test on each batch based on domain knowledge. We find that different ordinal levels of environmental conditions, such as low and high tensile force as well as low and high temperature, are applied to different batches of test units. This is consistent with the premise that the group-shared unobserved covariates are qualitative. By incorporating such discrete group-shared unobserved covariates as observed covariates, the performance improvement of proposed approach becomes quite small, as shown in Figure. 7 (b). The baseline model becomes Weibull regression model with qualitative observed covariates.
With incorporating different types of group-shared latent heterogeneity, there is limited performance improvement of the proposed approach as compared to the baseline model. This indicates that most of the heterogeneity have been explained by the observed covariates and group-shared latent heterogeneity has limited contributions to the model performance improvement. Thus, the baseline model without considering group-shared latent heterogeneity becomes almost equivalent to the proposed approach which accounts for multi-type group-shared latent heterogeneity. This pilot real study demonstrates the abilities of proposed modeling approach in capturing the influences of group-shared unobserved covariates and identifying the structure of group-shared unobserved covariates. The estimation results based on proposed approach can guide the reliability engineer to incorporate the group-shared covariates with identified structure to improve reliability assessment and further to identify the most appropriate design changes with the quantified groupshared latent heterogeneity in design stage. With the improved modeling accuracy, the proposed approach can also facilitate the cost-effective maintenance decisions in the field operation stage.

Conclusion
In this work, we propose a generic modeling approach to account for the intra-group dependency in product lifetime analysis and to capture the group-shared latent heterogeneity due to the influences of group-shared unobserved covariates. Specifically, we propose a generic lifetime model with different specifications of AFT models to explore different structures of group-shared unobserved covariates and to capture multi-type group-shared latent heterogeneity. Bayesian estimation technique is further employed to jointly estimate the influence of observed covariates and multi-type group-shared latent heterogeneity. Further, we construct different simulation scenarios in numerical study and conduct comprehensive comparison study to demonstrate the improved estimation performance and prediction accuracy of proposed approach. With constructed simulation scenarios, we also present an empirical study to investigate the impacts of group number and sample size per group on estimating the group-shared latent heterogeneity. Moreover, we present real case study to demonstrate the effectiveness of proposed approach. The proposed approach is flexible to capture the different types of group-shared latent heterogeneity and is able to identify different structures of group-shared unobserved covariates.