{'fraction_non_alphanumeric': 0.028756315315938818, 'fraction_numerical': 0.012353796110457472, 'mean_word_length': 4.296737536656892, 'pattern_counts': {'":': 0, '<': 1, '<?xml version=': 0, '>': 3, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Estimating Software Reliability Using Size-biased Concepts

Software reliability estimation is one of most active area of research in software testing. Since recording of time between failures has often been too difficult to collect, software testing data now have commonly been recorded as test-case-wise in a discrete set up. Although there have many models that were developed to estimate software reliability, they were too restrictive in nature. We have developed a model using the size-biased concept which not only estimates the software reliability, but also estimates the total number of bugs present in the software. The model is highly flexible as it could provide the reliability estimates at the end of software testing and also at any future phase of software testing that could have been conducted in order to improve the software reliability. This flexibility is instrumental to find the stopping phase such that the software reliability achieves a desired optimum level (e.g., 95%). In addition, we also provide a model extension which could be applied on grouped bugs in different regions of a software. We assessed the performance of our model via simulation study and found that each of the key parameters could be estimated with satisfactory level of accuracy. We also applied our model to two different software testing data sets. In the first model application we found that the conducted software testing was inefficient and a considerable amount of further testing is required to achieve a optimum reliability level. On the other hand, the application to second empirical data set has shown that the respective software was highly reliable with software reliability estimate 99.8%. We anticipate that our novel modelling approach to estimate software reliability could be very useful for the users and can potentially be a key tool in the field of software reliability estimation.


Introduction
Software reliability estimation and its related problems have become very important since most of the modern day devices have software as one of its components. White goods, medical equipments, cars, space related equipments are just some examples of this kind of devices, where software plays an important role in the execution and output of the devices. Apart from these devices there are a lot of software which are used in the service sectors, like banking software. Some of these software are called critical, since the failure of such a software may create havoc.
Many researchers studied various aspects connected with software reliability (Pham, 2000;Yamada, 2014). Determining optimum time for software release remains an interesting field of research Chakraborty et al. (2019). Nayak (1988) proposed that the software testing data needs to be collected in a different way than what was being done by most researchers till then.
Time between failures data becomes difficult to collect as the complicacies in software and its testing increases. In most cases now, the information logged during software testing becomes test -case-wise and hence discrete in nature. Software reliability studies, particularly, research on optimum duration of software testing under a discrete set up has also been focus of many studies (Chakraborty and Arthanari, 1994;Chakraborty, 1996;Dewanji et al., 2011;Das et al., 2019). Most of the literature available on this tries to develop optimum testing strategy based on the number of remaining bugs in the software (Chakraborty et al., 2019;Eom et al., 2013).
However, it is true that if the remaining bugs are in such paths (of the software) which will be rarely traversed by any inputs to be used by the users, then the chances of software failure will also be rare, which in turn, reflects that software is reliable. This particular phenomenon has not been properly studied in the literature of software reliability, though it looks to be more reasonable and closure to reality.
We introduce here a concept of 'size of a bug' or more particularly, 'the eventual size of a bug' which will be able to capture the chances of a remaining bug to be the cause of failure of software. The size or eventual size of a bug is defined as the number of inputs that may eventually pass through the bug in the whole life of a software, irrespective of whether the bug is detected or not. It must also be noted that the eventual size of a bug sometimes may be referred to as simply the size of the bug. A software can be considered as a collection of several paths and each input to the software is expected to follow a particular path. If the same input is used several times, then also, it can only check whether that particular path has any bugs or not. Since it will not traverse any other path, it will not be able to check whether other paths have any bugs or not. Different inputs would be required to check existence of bugs in different paths. We may assume that an input can identify at most one bug which lies on the path that the input would traverse.
Although Chakraborty (1996) introduced the size-biased concept in software reliability, the concept had been used earlier by Patil and Rao (1978) in different fields of investigation.
It is interesting to note that one may come up with a more realistic and advanced model for software reliability using the size-biased concept in relation to detection and debugging process of a software. A similar concept can also be used in deciding when to stop drilling while looking for a source of oil or natural gas in a given search area (Chakraborty and Arthanari, 1994).

Size-biased concept in software testing
It is quite natural that a path (in the software) branches into several sub-paths at a later stage.
For all these sub-paths, a part of the path is common in the beginning. Imagine that a bug is sitting on the common path and another bug is sitting on one of the several subpaths associated with the common path. It is quite obvious that the size of the bug, present in the common path is much higher compared to that of the bug in the sub-paths, since all inputs collectively going through each of the sub-paths must be traversing through the common path before entering into a sub-path. The size of a bug also, thus, may give an indication of how quickly a bug could be identified. If a bigger bug is not detected it would create a potential threat to the functioning of the software, even if there is only one bug. It is simple to understand that the probability of detection of a bug (and hence its fixation) depends on the size of the bug.
Larger the size of the bug, larger will be the chance of detecting that bug earlier in the testing phase as has been indicated in Chakraborty and Arthanari (1994). In fact, Chakraborty and Arthanari (1994) also have shown that similar concepts are applicable in discovering fields with rich hydrocarbon contents in the field of producing oil and natural gas. It is also clear that a bug which exists in a path that will hardly be traversed by any input, will remain harmless as far as the running of the software is concerned. This brings us to the conclusion that reliability of the software does not depend on just the number of bugs remaining in the software, rather it depends on the positioning of the bugs, particularly the paths on which it exist and whether that path is frequently traversed by inputs which are random in nature as per the user (Littlewood, 1979). Hence in order to have a better model for software reliability, our attention would be to find out the total size of the bugs that will remain and not just the number of remaining bugs.
In a discrete software testing framework, when an input is being tested, it results in either a failure or a success (finding an error). Testing of software is carried out into many phases, where, in each phase a series of inputs are tested and results of each testing are recorded as either a success or a failure. After identifying the bugs at the end of testing within a phase, they are debugged at the end of the phase. This process of debugging is known as periodic debugging or interval debugging (Das et al., 2016).
For testing software, we need to keep in mind certain factors like when we should stop testing, or what will be the criteria of stopping testing etc. If after sufficient testing and debugging most of the bugs remain in the software, then it may result in improper functioning of the software after release in the market. Therefore, a decision to optimize software testing and debugging time is an important part of the development process of software. Even if the number of remaining bugs is smaller, but the total size of the remaining bugs is big, then also the software may fail frequently.
The input space, consisting of all possible inputs to the software can be broadly divided into two subsets of which one consists of all the inputs which will result in a failure (we call it as a success set) and the other subset namely, failure set, consisting of all the inputs which give expected output. Testing and debugging of bugs are carried out in several phases in most situations Dewanji et al. (2011). Unlike the assumptions in most software reliability models, in real life situations, it is quite difficult to debug every time a bug is found. It may happen that two inputs have a common path at the beginning1 due to the presence of some common factors and then each of the inputs branches off to complete the job.
Following Dewanji et al. (2011), the process of testing flags off as soon as a bug is found and the process is stopped culminating in recording or logging in an incidence of a success. It is easy to understand that the next bug in the path can be detected only after debugging the bug which is detected earlier. Therefore we can assume that the size of a bug which is present at the beginning of a path is much larger compared to the size of a bug present at the end of a sub-path or compared to the bugs present in a path that are hardly traversed by any input.
Hence, detecting a bug during software testing, can be thought to be a probabilistic sampling, where the chances of a bug being detected is proportional to the size of the bug. Patil and Rao (1978) used a similar concept, named as size-biased concept, for modeling identification of species.

General approach
We utilized the hierarchical modelling philosophy to formulate a model to address the problem of imperfect detection of the bugs in software testing procedure. The developed model can be used to estimate the total number of bugs present in the software, as well as the remaining eventual bug size. We also provided a new procedure to predict the stopping phase such that the estimated remaining bug size at that phase remains below a preassigned threshold. Later, we extended the model described above to also accommodate the possible groups of bugs who share the same bug size. We conducted a simulation study to assess the efficacy of the model. We assessed the robustness of the models using relative bias, coefficient of variation, and coverage probability of the 95% credible intervals of population size of the bugs. We applied our developed models to an empirical software bug data set and Indian Space Research Organisation (ISRO) mission data set. Finally, we estimated software reliability in each case.

Model description
We utilize the hierarchical modelling philosophy to formulate a model to address the problem of imperfect detection of the bugs in software testing procedure.

State process
Consider a number of distinct and independent bugs N are present in a particular software and size of each bug is denoted by S i . Let S denotes a vector of these latent variables S 1 , S 2 , . . . , S N defining the size of the N (unknown) bugs under study. For the ease of computation and other technical advantages (described later), we define N ∼ Binomial (M, ψ), where M represents the maximum possible number of bugs present in the software and ψ denotes the inclusion probability to indicate the proportion of M that represent the real population of bugs.

Observation process
We suppose that T j , j = 1, 2, . . . , Q inputs are used for each of the R testing phases. We consider the situation where a present bug can get detected in any of the T j inputs at the j-th phase.
Let y ij represent the binomial detection outcome for a bug i over the T j inputs on phase j. If y ij > 0, this subsequently implies that y il = 0, l = 1, 2, . . . , (j − 1). It should be noted that after the bug gets detected at the j-th phase, it is eliminated from the pool of bugs. For example, in a software testing, if bug 1 gets detected at phase m i = 4, we would have y 1l = 0, l = 1, 2, . . . , 3 and y 14 > 0.
We used the data augmentation approach to model the number N of bugs in the software by choosing a large integer M to bound N and introduced a vector of M latent binary variables z = (z 1 , z 2 , . . . , z M ) such that z i = 1 if individual i is a member of the population and z i = 0 otherwise. We assume that each z i is a realisation of a Bernoulli trial with parameter ψ, the inclusion probability.
A binomial model, conditional on z i , is assumed for each observation y ij : where p i denotes the detection probability of the i-th bug on a phase. The detection probability p i is modelled as a increasing function of the bug size S i since the detection probability directly depends on the size of a bug, more the bug size, higher the detectability.

Model for detection probability
From the definition of bug size, S i is higher if placement of i-th bug is on a common path near the origin and a number of subpaths follow subsequently. If r denotes the probability of bug detection in any one of the inputs that have passed through the i-th bug, then the probability of detecting i-th bug with one input is The parameter r plays the role of a shared parameter across all the bugs and critical for the dependence structure of the nodes in our joint probability model. In addition, the above formulation of p i comes naturally from our definition of bug size and accounts for individuallevel heterogeneity in detection probability of the bugs. Note that, p i is a monotonically increasing function of S i and when S i = 0, we have p i = 0.

Model for N
We used the data augmentation approach to model the number of bugs N in the software by choosing a large integer M to bound N and introduced a vector of M latent binary variables z = (z 1 , z 2 , . . . , z M ) such that z i = 1 if individual i is a member of the population and z i = 0 otherwise. We assume that each z i is a realisation of a Bernoulli trial with parameter ψ, the inclusion probability.
We assume that n bugs get detected over the R testing phases which is expected to be less than the total number of bugs N due to imperfect detection during testing. Consequently, as part of the data augmentation approach, the detection data set {y ij } i,j is supplemented with a large number of "all-zero" encounter histories Y rem , an array of "all-zero" detection histories with dimensions (M − n) × Q. We label the complete zero augmented complete detection data set as Y .

Estimating the remaining eventual bug size and the stopping phase
The above model is well suited to estimate the number of bugs N , the detection probability p i 's, and bug size S i 's. But to estimate the remaining eventual bug size at a later untested phase, we proceed as follows.
We denote f as the model for the detection observations for a bug with inputs T j , j = 1, 2, . . . , J, where J > Q), andỹ as future observation or alternative detection outcome that could have been obtained during the testing phase. Since the stopping phase (such that the remaining eventual size of the bugs is less than M ) is unknown to the software tester, we assign a sufficiently large value for J, considering the available RAM size of the computing device and and computing time. The posterior predictive model for a new detection dataỹ i for the i-th bug is then, where θ denotes the vector of all the parameters r, S , z , ψ and f (ỹ i | Y ) is the predictive density forỹ i induced by the posterior distribution π(θ | Y ).
In practice, we obtain a single posterior replicateỹ where {θ (l) : l = 1, 2, . . . , L} represents a set of MCMC draws from the posterior distribution of parameter θ.
We define a set of deterministic binary variables u ij which takes the value 1 if i-th bug is detected on or before j-th phase and 0 otherwise. Total size of the bugs that are detected up to the j-th phase is then computed as Consequently, we also compute the total eventual remaining size of the bugs that are not detected up to the We obtain the stopping phase k such that B k < (where is a preassigned threshold). We compute B j for each replicated data set

Estimation of software reliability
Software reliability at phase j is estimated as posterior probability that the remaining size is less than or equal to the prefixed small quantity given the obseved data Y ,

Modelling for grouped bugs
Often we come across situations where a few bugs are collocated on the same path or same part of the software in such a way that we can assume without loss of generality that each of them have them same bug size. For computational and notational simplicity, We make a transformation of the data set ((y ij )) to (y * g ) where the observed data y * g represents the number of bugs from the g-th group that are detected. Consequently, we have y * g ∼ Binomial(T j(g) , p * g ), p * g denotes the probability of detecting a bug in g-th bug-group with a single test case and j(g) denotes the corresponding phase to the g-th group.
Here, we consider a number of distinct group of bugs N G that are present in a software and each bug in a group (say, g-th) has size S * g . Each group of bugs comprises at least one bug. Following Section 2.2.1, we define N G ∼ Binomial (M G , ψ), where M G is a large positive integer that gives an upper bound to N G . The link between p g and the size S * g remains the same as in Section 2.2.3, p * g = 1 − (1 − r * ) S * g . We used the data augmentation approach to model the number of bug-groups N * (discussed in Section 2.2.4). The total number of bugs N * has the following expression: where n denotes the number of bugs detected during the testing period and a g denotes the number of bugs in the g-th group that went undetected. We utilized the posterior predictive distribution of new detection dataỹ * with density f (ỹ * g | Y * ) to estimate a g .
To compute the remaining eventual size, we introduce binary variables ((u gQ )), g = 1, 2, . . . , M G , where u gQ takes the value 1 if g-th bug-group is detected on or before Q-th phase and takes 0 otherwise. The remaining eventual size is calculated as where d g denotes the number of bugs in g-th bug-group .

Prior assignment
Bug sizes (S i 's) are usually latent and unobservable. We assign a Poisson-Gamma mixture prior for S i to capture required level of variability in the latent variable. Consequently, each S i is assumed to follow Poisson distribution with mean λ i , where the λ i is a random draw from Gamma distribution with shape parameter a s and rate b s . We assign bounded Uniform prior over the interval (0, 1) for detection probability r and the inclusion probability ψ 3 Model fitting

Model performance measures
We used relative bias, coefficient of variation and coverage probability to evaluate the effect of detection function misspecifications on population size and home range size estimators.
Relative bias. Relative bias (RB) is calculated as whereθ denotes the posterior mean 1 R R r=1 θ (r) and θ 0 gives the true value.
Coefficient of variation. Precision was measured by the coefficient of variation (CV): where SD(θ) = 1 R

Results from Simulation study
We fitted our Bayesian size-biased model to each of the 200 simulated data sets using MCMC and M is set to 400 for each model fitting. All MCMC samples of the parameters of interest (e.g., population size N , detection parameter r) were obtained after ensuring proper mixing and convergence, withR values below 1.1. The posterior estimates of different parameters were obtained using the MCMC chains. The posterior summaries of the total number of bugs N and detection parameter r for the simulation study are provided in Tables 2 and 3, respectively and also portrayed in Figure 1.
We estimated the reliability at each phase and also at different possible future phases (assuming a pre-specified number of test cases in each phases). It is important to mention that, the estimation of reliability heavily depends on the pre-specified threshold and the number of test cases used during the future phases (that would be conducted after the first 5 phases already conducted). Here we have assumed that the number of test cases in each future phase to be the same as the number of inputs in the respective scenario.
The reliability (i.e., posterior probability of the remaining size lying below a threshold) is a non-decreasing function of testing phase index, since remaining bug size gets reduced with more bugs being detected in subsequent testing phases. We found the reliability estimates to attain the optimum 95% level (with threshold 100) to be varying with respect to different simulation scenarios ( Figure 1). For instance, the reliability estimate attained the optimum 95% level (with threshold 100) at phase 30 in set 1, implying the developer would need to continue software testing for 25 more future phases (after the 5 testing phases already conducted) to attain optimum software reliability level. Hence, the stopping phase was estimated as 30. For other sets the estimates stopping phases were at phase 24 (set 2), phase 14 (set 3) and phase 10 (set 4).
5 Application to Software testing empirical data

Data description
The data set consists a total of 8757 test inputs detailed with build number, case id, severity, cycle, result of test, defect id etc. In this data, the severity of a path is broadly divided into three categories, namely, simple, medium and complex depending on the effect of the bug if it is not debugged before marketing the software. The data has four cycles namely Cycle 1, Cycle 2, Cycle 3 and Cycle 4, which is equivalent to the different phases of testing we have referred to Section 2. After each cycle, the bugs that are identified during the cycle are debugged as mentioned in the Section 2.

Results from Software bug data analysis
The posterior estimates of the main parameters N , ψ, r and B 4 are provided in Table 4  to be 3000 in order to resemble with the observed data set.
We found the reliability to attain the optimum 95% level at phase 16 if we would have continued with 3000 test cases in each phase, implying the developer would need to continue software testing for 12 more future phases (after the 4 testing phases already conducted) to attain optimum software reliability level. Hence, the stopping phase was estimated as 16. The reliability took much longer (40 phases) to reach optimum 95% level with 1000 test cases in each phase, and took only 12 phases with 5000 test cases in each phase (these results are provided in the appendix). This also revealed that it takes approximately 36000 future test cases to attain the optimum 95% level of reliability.
6 Application to ISRO mission empirical data Different bugs were detected during these thress primary stages: n CI = 33 bugs were detected during CI stage, n M T = 27 bugs were detected during MT and n ST = 34 bugs were detected during ST (where the phase specific segregation is as the following: n 1 = 9, n 2 = 7, n 3 = 7, n 4 = 8, n 5 = 1, n 6 = 2, n 7 = 0). There were also different number of test cases for each mission in each software and in each phase. For our analysis we consider the testing data from MT and seven phases of ST (i.e., Q = 8 testing phases) in total as observed data set.

Results from ISRO mission data analysis
We applied the grouped version of our size-biased model (Section 2.3) to ISRO mission data set which was perfectly suited for applying this model. The different missions, different softwares used in those missions and the different phases -all contributed to the variation of groups and number of bugs in a group. In the observed data set, any change in the mission, software or phase was considered as a different group formation. Here, it is not possible to extend the number of phases, hence instead of finding a stopping phase, we obtain the number of future test cases required to get the remaining bug size below a pre-specified threshold. This future test cases can be implemented before a future mission or after a software update.
The posterior mean of number of groups of bugs was estimated at 84 with a 95% credible interval (80, 89) (see Table 5). The posterior mean estimate of ψ is 0.257 with a 95% credible interval (0.195, 0.323). This also confirms our specified upper bound M G = 200 for the number of groups to be appropriate. The size-biased detection model parameter is estimated as 1.102× 10 −3 with a 95% credible interval (6.439×10 −4 , 1.807×10 −3 ). The total number of bugs present was estimated as 94 with 95% credible interval (94,95) which is highly precise.
The reliability of the softwares is estimated as 0.995 after the 8 testing phases (including module testing and seven phases of simulation testing) with threshold = 25. Since the testing phases had managed to detect almost all the bugs present in the softwares, this has led to such high reliability. We also show that reliability increases with increase in number of future test cases ( Figure 3).

Discussion
We described a Bayesian size-biased model that can be applied to software testing data set to explicitly model and estimate the population size, detection parameter and latent size of the bugs. The model also allows estimation of reliability at any given phase for any given threshold for the remaining bug size using posterior predictive distribution of the bug detection data.
Consequently, we could obtain an estimate of the stopping phase providing the number of additional phases of testing are required to achieve an optimum reliability level (e.g., 0.95).
We showed via a simulation study that the parameters of interest (e.g., N , r, reliability) can be accurately estimated by our model. Number of inputs plays a key role in software testing in general, as higher number of inputs boosts the probability of detecting of bugs (Table 1).
This also led to more accurate estimation of the model parameters, which can be observed in the lower magnitude of CV estimates of N and r with higher number of inputs (Tables 2   and 3). Further, we also noticed that, in such scenarios, optimum reliability level was attained comparatively quicker than the scenarios with lower number of inputs (Figure 1e).
Size biased model fitted to empirical software testing data of bugs yielded satisfactory estimates of the key parameters. However, we noticed that the software testing conducted were rather inefficient since the estimated software reliability was approximately near zero after the first four phases of testing ( Figure 2). We anticipate that some major bugs (with moderately large size) were still present. We receommend to continue testing for at least 36000-40000 more test cases (which could be broken down into multiple phases) to attain optimum software reliability level 95%.
On the contrary, software reliability estimates of ISRO mission softwares were found to be extremely high (i.e., 0.998) after the first 8 testing phases, demonstrating the advantage of efficient software testing. Our finding that the number of bugs detected were almost equal to the true number of bugs available to be detected also supports this.
Given the enormous amount of interest in software testing use in technology industry, our size-biased model could be very useful to provide accurate estimates of the number of present bugs as well as software reliability. Our model used the Bayesian paradigm which added the required flexibility to estimate a large number of model parameters. Although we found the parameter estimates to be moderately robust, we recommend to conduct a prior sensitivity study before application of the size-biased model.

Conflicts of interest
It is hereby declared that the authors do not have any conflict of interest.

Code availability
R codes for generating simulated data and data analysis are provided in the online supplementary material and also can be found in GitHub https://github.com/soumenstat89/size_ biased. , S. (2014). Software Reliability Modeling: Fundamentals and Applications, volume 5.

Yamada
Springer.      Barplot of a particular colour (e.g., pink) in panel (d) corresponds to a particular threshold (given along the x-axis).