text:
Using Hidden Semi-Markov Models for Effective Online Failure Prediction
- Claud Shelton
- 4 years ago
- Views:
1 Using Hidden Semi-Markov Models for Effective Online Failure Prediction Felix Salfner and Miroslaw Malek Institut für Informatik, Humboldt-Universität zu Berlin Unter den Linden 6, Berlin, Germany {salfner Abstract A proactive handling of faults requires that the risk of upcoming failures is continuously assessed. One of the promising approaches is online failure prediction, which means that the current state of the system is evaluated in order to predict the occurrence of failures in the near future. More specifically, we focus on methods that use event-driven sources such as errors. We use Hidden Semi- Markov Models (HSMMs) for this purpose and demonstrate effectiveness based on field data of a commercial telecommunication system. For comparative analysis we selected three well-known failure prediction techniques: a straightforward method that is based on a reliability model, Dispersion Frame Technique by Lin and Siewiorek and the eventset-based method introduced by Vilalta et al. We assess and compare the methods in terms of precision, recall, F-measure, false-positive rate, and computing time. The experiments suggest that our HSMM approach is very effective with respect to online failure prediction. Anticipating failures before they occur and applying preventive strategies to avoid them or reducing time-torepair by preparation for upcoming failures is a promising approach to further enhancement of system dependability. Preventive maintenance has become over the years an increasingly significant area in research on dependability, as can be seen from initiatives and research efforts on autonomic computing [15], trustworthy computing [19], recovery-oriented computing [5], rejuvenation (e.g., [10]) and various conferences on self-*properties (see, e.g., [2]). Proactive reaction to faults is at first glance closely coupled with fault detection: A fault needs to be detected before a system can react to it. However to be precise, not the fault but the failure is the kind of event that should be primarily avoided, which makes a big difference especially in the case of fault-tolerant systems. Hence, efficient proactive fault handling requires the prediction of failures, to judge whether a faulty situation bears the risk of a failure or not. Time-to-failure is naturally connected to reliability, since F(t) = P[T t] = 1 R(t) (1) where F(t) denotes the probability of a failure until time t and R(t) denotes reliability. Assessment of F(t) has been an issue in dependable computing research for several decades. The majority of work has analyzed F(t) based on failure rates estimated from static characteristics of the system or the development process, such as lines of code, number of bug-fixes, etc. These models are in most cases targeted at long-term predictions. In view of ever-increasing system complexity it has become necessary to turn to runtime monitoring-based methods which evaluate a current state of the system. Hence, the focus of online failure prediction is to perform short-term failure predictions based on the current runtime state of the system. 1. Introduction Figure 1. Time relations in online failure prediction: t present time; t l lead time; t w warning time; t p prediction period; t d data window size Online (short-term) prediction of eventually upcoming failures is shown in Figure 1. If a prediction is performed at time t we would like to know whether at time t + t l a failure will occur or not. We call t l the lead time. t l has a lower bound called warning time t w, which is determined by the time needed to perform some proactive action, e.g., the time needed to restart a component. On the
2 Figure 2. Classification of failure prediction techniques. The bottom line represents exemplary approaches. other hand, large values of t l will result in inaccurate predictions since the chance to anticipate an imminent failure early is less than shortly before the failure. We used a lead time of t l = 5 min. in our experiments. The prediction period t p describes the length of the time interval for which the prediction holds. If t p becomes large, the probability that a failure occurs within t p increases. 1 On the other hand, a large t p limits the use of predictions since it cannot be determined, when exactly the failure will occur. As online failure prediction takes into account the current state of the system, some prediction techniques use data from an interval preceding present time. The length of this data window is called data window size and is denoted by t d. Failure prediction is only the first prerequisite step in proactive fault handling. It goes without saying that prediction sometimes has to be coupled with root cause analysis, i.e., diagnosis, in order to determine what causes the imminent failure that has been predicted. However, not all proactive techniques require a detailed analysis: for example, setting up an additional checkpoint can be scheduled without knowing the root cause. This paper introduces a new method for online failure prediction that is based on Hidden Semi-Markov Models. The basic idea of our approach is to predict failures by recognizing patterns of error events that indicate an imminent failure. In order to assess its capability to predict failures, the method has been applied to field data of a commercial telecommunication system. Results are compared to three well-known failure prediction techniques: a straightforward method that is based on a reliability model, the Dispersion Frame Technique developed by Lin and Siewiorek [18] and the eventset-based method by Vilalta et al. [31]. The paper is organized as follows: A coarse classification of related work is provided in Section 2. Our HSMM based failure prediction approach is presented in Section 3 1 If t p, predicting a failure would always be true! while the models used for comparison are described in Section 4. The experiment conducted to compare the approaches is characterized in Section 5 and results are provided in Section Related Work and Approaches First approaches to failure prediction were developed in combination with reliability theory and preventive maintenance (See, e.g., [12] for an overview). A key input parameter of these models is the distribution of time to failure. Therefore, a lot of work exists providing methods to fit various reliability models to data with failure time occurrences. There were also approaches that tried to incorporate several factors into the distribution. For example, the MIL-book [21] classified, among others, the manufacturing process whereas several software reliability methods incorporated code complexity, fixing of bugs, etc. into their models (see, e.g., [9]). However, these methods are tailored to long-term predictions and do not work appropriately for short-term predictions needed for online failure prediction. Methods for short-term failure prediction are typically based on runtime monitoring as they take into account a current state of the system. A variety of runtime monitoringbased methods have been developed. A coarse classification is shown in Figure 2. Failure prediction approaches can be subdivided into methods that evaluate the occurrence of previous failures, methods that evaluate symptoms of faults, and methods that evaluate errors, which are manifestations of faults. The first group failure prediction based on the occurrence of previous failures is closely related to the methods known from reliability prediction. However, since they are runtime monitoring methods they build on failures that have occurred during runtime rather than failure rates determined from system properties or test-runs. Failure predictors of this category have been proposed in [6] and [22]. The majority of existing prediction techniques are symptom-based. Symptoms are side-effects of faults such
3 as increasing memory consumption, increasing number of running processes, etc. These methods evaluate periodically measured system parameters such as the amount of memory usage, number of processes, etc. in order to, e.g., estimate a trend or detect deviation from normal system behavior. Approaches such as Multivariate State Estimation Technique (MSET) [29], trend analysis techniques like the one developed in [11] or function approximation approaches like Universal Basis Functions (UBF) [14] belong to this category. With respect to the third category of runtime monitoringbased failure prediction approaches, which evaluate the occurrence of errors, Siewiorek and Swarz [28] state that the first approaches have been proposed by Nassar et al. [20]. These approaches rely on systematic changes in the distribution of error types and on significant increase of error generation rates between crashes. Another method has been developed by Lin and Siewiorek [17, 18] called Dispersion Frame Technique (DFT). Its original purpose was to predict failures of the Andrews distributed File System at Carnegie Mellon University. DFT analyzes (after some preprocessing) the time of error occurrence in the near past and applies heuristic rules in order to predict an imminent failure. As systems became more and more complex, the number of ways a system could fail continued to grow even more rapidly. This is reflected in recently developed failure prediction approaches: As it does not seem sufficient anymore just to investigate, when errors have occurred, these algorithms investigate what has happened in the system: predictions are based on the type of errors that have occurred. To our knowledge, the first data mining approach to failure prediction has been published by Hätönen et al. [13]. The authors propose to manually set up a rule miner specifying certain characteristics of episode rules. Weiss [34] introduced a failure prediction technique called timeweaver that is based on a genetic training algorithm. One of the most well-known prediction algorithm of this category has been developed at IBM T.J. Watson by Vilalta et al. (see, e.g., [31]). The authors introduce a concept called eventsets and apply data-mining techniques to identify sets of events that are indicative of the occurrence of failures. Further methods have briefly been described by Levy et al. in [16]. Please note that failure prediction does not intend to find the root cause of a problem. This implies that the focus is not on an identification whether the root cause is transient / intermittent or permanent. The goal is to evaluate the current system state with respect to the risk that a failure occurs within a short time interval in the future, regardless of the fault that causes the failure. However, a classification of the underlying fault might be helpful but is not considered in any of the presented approaches. Classification of our approach. Our work is focused on the development of an error-based failure prediction technique that incorporates both time of occurrence and type of error events. This combination leads to the notion of error patterns. By this approach, the task of failure prediction is turned into a pattern recognition problem. In 2006, we have published results for an approach called Similar Events Prediction (SEP) [26]. Although the results of SEP have been encouraging, it did not allow for predictions longer than one minute which is too short time for effective preventive actions and its computational complexity to compute the predictions is too high. Furthermore, SEP models are overfitted to the training data resulting in bad prediction results if system behavior changes only slightly. For this reason, we reverted to an idea presented by one of the authors at EDCC-5 [24]: the usage of hidden Markov models. In order to make this idea of practical value we had to extend the approach to Hidden Semi-Markov Models (HSMM), which is the approach presented in this paper. To the best of our knowledge, SEP and HSMM are the only error-based online failure prediction methods that apply pattern recognition techniques. Selection of approaches used for comparison. In order to evaluate quality of our HSMM approach we compare it with other well-known event-based techniques such as DFT by Lin and Siewiorek [18], which investigates the time of error occurrence, and the eventset-based method by Vilalta et al. [31], which investigates the type of error events that have occurred prior to failures. We also compared HSMM to a prediction approach that evaluates the occurrence of failures. However, since we neither had access to the source code nor expert knowledge of the commercial system from which we obtained our data, we could not apply a sophisticated reliability model. Hence, the reliability model based prediction allows only for a general grading of the approaches. All four models are described in more detail in the next sections. 3. HSMM-based Failure Prediction The basic assumption of our approach is that failureprone system behavior can be identified by characteristic patterns of errors. The rationale for this assumption is that due to dependencies within the software, a detected fault (error) in one system component leads under certain conditions to errors of dependent components. Since fault-tolerant systems can handle most of these situations but fail under some conditions, the goal is to identify and recognize those patterns that indicate an upcoming failure. Hidden Markov models have successfully been applied to various pattern recognition tasks such as speech recognition and genetic sequence analysis. In the area of dependable
4 Figure 3. Two HSMMs are trained: One for failure sequences and one for non-failure sequences. Sequences consist of errors A, B, or C that have occurred in previously recorded training data. Failure sequences consist of errors that occurred within a time window of length t d preceding a failure ( ) by lead time t l. Non-failure sequences consist of errors that occurred at times when no failure was imminent. computing, hidden Markov models have been applied for the detection of intrusion into computer systems [33], fault diagnosis [7], or network traffic modeling [27]. Hidden Markov models have been extended to hidden semi-markov models by several researchers. However, since most of these extensions have been developed for speech recognition, they are not appropriate for our purpose. An overview of previous extensions and a detailed description of our model are presented in [25]. Once the two models are trained, new unknown sequences can be evaluated if they indicate an upcoming failure or not. To achieve this, sequence likelihood is computed for both models and Bayes decision theory is applied in order to yield a classification (see Figure 4) Approach Error event timestamps and message IDs form an eventdriven temporal sequence, which will from now on be called error sequence. HSMM failure prediction applies machine learning techniques in order to algorithmically identify characteristic properties indicating whether a given error sequence is failure-prone or not. More specifically, we adjust the model s parameters using recorded error logs for which it is clear when failures have occurred. The trained hidden semi-markov models are then applied in order to classify new error sequences as failure-prone or not failureprone. In machine learning, such approach is called offline supervised learning. After some data preprocessing including techniques such as tupling [30], failure and non-failure error sequences are extracted. Failure sequences consist of errors that have occurred within a time window of length t d preceding a failure by lead time t l. Non-failure sequences consist of errors appearing in the log when no failures have occurred (see Figure 3). Two HSMMs are trained: One for failure sequences and the other for non-failure sequences. The main goal of training is that the failure model specifies characteristics of failure sequences the non-failure model stays rather unspecific and is only needed for sequence classification. Figure 4. Online failure prediction. At the occurrence of error A (present time), the sequence under investigation consists of all errors that have occurred within the preceding time window of length t d. Failure prediction is performed by computing sequence likelihood of the sequence using both the failure and non-failure model. Both likelihoods are evaluated using Bayes decision theory for classification as failure-prone or failure-free The key notion of this approach to failure prediction is as follows: by training, the failure model has learned how error sequences look like if they precede a failure by lead time t l. Hence, if a sequence under consideration is very similar to failure sequences in the training data, it is assumed
5 Figure 5. An exemplary HSMM as it is used for failure prediction. It consists of N-1 states that can generate error symbols A, B, or C, plus a final absorbing state that can generate failure symbol F. g ij (t) indicate state transition distributions. that a failure is looming which will occur at time t l ahead. Of course, error sequences are rarely identical. Therefore, a probabilistic approach is needed in order to handle similarity, which in our case is achieved by use of hidden semi- Markov models Hidden Semi-Markov Models Hidden Semi-Markov Models (HSMMs) are a continuous-time extension to hidden Markov models, which in turn are an extension to discrete time Markov chains (DTMC). This section sketches the basic principle of our model, a more detailed treatment can be found in [25]. HSMMs are determined by several quantities. Let S = {s i } denote the set of states, 1 i N. Let π = [π i ] denote the vector of initial state probabilities and G(t) = [g ij (t)] an N N matrix of cumulative state transition probability distributions over time with the following properties: i : π i = 1, and (2) i lim g ij(t) = 1 (3) t j π and G define a continuous-time semi-markov process of traversals through states S. Furthermore, let O = {o k }, 1 k M denote a finite set of discrete observation symbols. Each time the stochastic process enters a state, one such observation symbol is generated according to a state-dependent probability distribution b i (o k ). Observation probabilities for all states set up an N M matrix B = [b i (k)] = [b ik ]. In summary, an HSMM is completely defined by λ = (π, G(t), B). Our approach allows to use a great variety of cumulative, parametric, continuous distributions to be used for g ij (t). Moreover, our approach allows to use mixtures of such distributions such that g ij (t) = p ij s.t. i, j : R r=0 r=0 w ij,r κ ij,r (t θ ij,r ) (4) R N w ij,r = 1 and i : p ij = 1 (5) j=1 p ij is the limiting weight for transitions from state i to state j corresponding to transition probabilities in DTMCs. For each transition from i to j, w ij,r denotes the weight of the r-th kernel κ ij,r. Finally, κ ij,r (t θ ij,r ) denotes a cumulative, continuous, parametric distribution over time t with parameters θ ij,r such as mean and variance in case of normal distributions. HSMMs are called hidden since it is assumed that only the generated symbols can be observed and that the state of the stochastic process is hidden from the observer. In the case of failure prediction, this property can be mapped onto the fundamental concepts of faults and errors: Faults are by definition unobserved. Hence they correspond to the hidden states. Errors, which are the manifestation of faults, correspond to the manifestation of hidden states, which are observation symbols. Figure 5 shows an exemplary HSMM as used for failure prediction. It constitutes a left-to-right structure including shortcut transitions to increase flexibility of the model. The string of hidden states ends in a special hidden state representing a failure. We have adapted standard HMM algorithms such as Baum-Welch and Forward-Backward to train the models (i.e., adjust λ from a set of training sequences) and to efficiently compute sequence likelihood. The next two sections provide more details. Although the models need to be trained before they can be applied in order to perform online failure prediction, the prediction part is described first.
6 The reason for this is that training shares some theory with prediction but adds some more concepts Failure prediction Assume that both the failure and non-failure model have been trained, i.e., parameters λ F and λ F are set. The goal is to assess, whether a given observation temporal sequence (an error sequence) o = [o 0... o L ] is failure prone or not. As shown in Figure 4, the first step is to compute sequence likelihoods of o for both models, on which the sequence is subsequently classified as failure-free or failure-prone using Bayes decision theory. Sequence likelihood is defined as the probability that a given model λ can generate observation sequence o: P(o λ)= P s πs 0 bs 0 (o0) Q L k=1 P(S k=s k, d k =t k t k 1 S k 1 =s k 1 ) b sk (o k ) (6) where s = [s k ] denotes a sequence of states of length L+1 and t k are the timestamps of the sequence. The sum over s denotes that all possible state sequences are investigated. However as is the case for standard HMMs, due to the sum over all possible state sequences s Equation 6 suffers from unacceptable complexity. The so-called forward algorithm solves this issue for standard HMMs by applying a dynamic programming approach (see, e.g., [23]). A similar algorithm can be derived for our HSMM: By assuming g ii (t) = 0 and defining v ij (d k ) = P(S k = s j, d k = t k t k 1 S k 1 = s i ) g ij (d k ) if i j = N (7) 1 g ih (d k ) if i = j h=1 to incorporate the case of remaining in the same state, the so-called forward variable α becomes: α k (i) = P(O 0 O 1... O k, S k = s i λ) α 0 (i) = π i b i (O 0 ) N α k (j) = α k 1 (i) v ij (t k t k 1 ) b j (O k ) i=1 (1 k L) and sequence likelihood can be efficiently computed by N P(o λ) = α L (i) (9) i=1 (8) Sequence classification. Bayes decision theory proves that the error of misclassification is minimized if a sequence is assigned to the class of maximum posterior probability. However, in real failure prediction applications there are several things that have to be accounted for: 1. Likelihood P(o λ) gets small very quickly for long sequences, such that limits of double-precision floating point operations are reached. For this reason, a technique called scaling and log-likelihoods have to be used. 2. Misclassification error is not always a good metric to evaluate classification. Rather, in real applications, different costs may be associated with classification. For example, falsely classifying a non-failure sequence as failure-prone might be much worse than vice versa. These issues can also be addressed by Bayes decision theory and the resulting classification rule is: Classify sequence o as failure-prone, iff log [ P(o λ F ) ] log [ P(o λ F) ] > [ ] [ ] c log FF c F F P( F) + log c F F c FF P(F) } {{ } } {{ } ( ; ) const. (10) where c ta denotes the associated cost for assigning a sequence of type t to class a, e.g., c F F denotes the cost for falsely classifying a failure-prone sequence as failurefree. P(F) and P( F) denote class probabilities of failure and non-failure sequences, respectively. See, e.g., [8] for a derivation of the formula. The right hand side of Inequality 10 determines a constant threshold on the difference of sequence loglikelihoods. If the threshold is small more sequences will be classified as failure-prone increasing the chance of detecting failure-prone sequences. On the other hand, the risk of falsely classifying a failure-free sequence as failure-prone is also high. If the threshold increases, the behavior is inverse: more and more failure-prone sequences will not be detected at a lower risk of false classification for non-failure sequences Training We have adapted the Baum-Welch expectationmaximization algorithm [3, 23] for this purpose. The objective of the training algorithm is to optimize the HSMM parameters π, G(t), and B such that the overall training sequence likelihood is maximized. From this approach follows, that the basic model topology (number of states, available transitions between states, number of observation symbols, etc.) must be pre-specified.
7 In analogy with the standard HMM training algorithm, a backward variable β is defined as: β k (i) = P(O k+1... O L S k = s i, λ) β L (i) = 1 N β k (i) = v ij (d k )b j (O k+1 )β k+1 (j) j=1 Using α and β together, ξ can be computed: (11) ξ k (i, j) = P(S k = s i, S k+1 = s j o, λ) (12) ξ k (i, j) = α k (i) v ij (d k+1 ) b j (O k+1 ) β k+1 (j) P Ni=1 P Nj=1 (13) α k (i) v ij (d k+1 ) b j (O t+1 ) β k+1 (j) which is the probability that a transition from state i to state j takes place at time t k. α, β and ξ can be used to maximize the model parameters. Re-estimation of π, and B is equivalent to standard HMMs, whereas re-estimation of p ij, w ij,r, and θ ij,r is based on the following principle: ξ k (i, j) not only assigns a weight to the transition from i to j, as is the case for standard HMMs, but also assigns a weight to the duration of the transition. More precisely, each transition i j can in principle account for all transition durations t k+1 t k in the training sequences, each weighted by ξ k (i, j). Hence transition parameters can be optimized iteratively such that they best represent the weighted durations. The entire procedure computation of α, β and ξ and subsequent maximization of model parameters is iterated until convergence. It can be proven that the procedure converges at least to a local maximum. Since it starts from a randomly initialized HSMM, the algorithm is performed several times with different initializations and the best model is kept for prediction. In order to avoid overfitting, i.e., the models memorize the training data set very well but perform badly on new, unknown data, we use background distributions for both B and G(t). Since size and topology of the model are not optimized by the Baum-Welch algorithm, several configurations of the principle structure shown in Figure 5 have to be tried Computational complexity With respect to online prediction the forward algorithm has to be executed twice: one time for the model of failureprone sequences and one time for the non-failure sequence model. Although the standard HMM forward algorithm belongs to complexity class O(N 2 L) where N is the number of hidden states and L is the length of the symbol sequence, We use a left-to-right model which limits complexity to O(NL), although the constant factors are larger than for standard HMMs since cumulative distributions need to be evaluated. Training complexity is difficult to estimate since the number of iterations can hardly be estimated. One iteration has complexity O(N 2 L). However, training time is not as critical as it is not performed during system runtime. 4. Models Used for Comparison We have compared our HSMM failure prediction approach to three other approaches: Dispersion Frame Technique and eventset-based method, which also operate on the occurrence of errors, and a straightforward reliability-based prediction algorithm, which evaluates previous occurrences of failures Reliability-based prediction As can be seen from Equation 1, failures can be predicted by reliability assessment during runtime. Due to the lack of insight into the system, we had to use a rather simple reliability model that assumes a Poisson failure process and hence approximates reliability by an exponential distribution: F(t) = 1 e λt (14) The distribution is fit to the short-term behavior of the system by setting the failure rate to the inverse of mean-timeto-failure (MTTF). We have set MTTF to the mean value observed in the data set used to assess the predictive accuracy of the technique. This yields an optimistic assessment of predictive accuracy since in a real environment, MTTF must be estimated from the training data set. A failure is predicted according to the median of the distribution. After each failure that occurs in the test data set, the timer is reset and prediction starts again. This is why this simple failure prediction method is based on runtime monitoring. Please note that we included such simple failure prediction model only for a rough assessment of failure-based prediction approaches and to get an idea what level of failure prediction quality can be obtained with a very simple approach. On the other hand, Brocklehurst and Littlewood [4] argue that none of the reliability models are really accurate, which is especially true for online short-term predictions Dispersion Frame Technique (DFT) DFT is a well-known heuristic approach to analyze a trend in error occurrence frequency developed by Lin [17] and published later by Lin and Siewiorek [18]. In their paper, the authors have shown that DFT is superior to classic statistical approaches like fitting of Weibull distribution shape parameters. A Dispersion Frame (DF) is the interval time between successive error events. The Error Dispersion Index (EDI)
8 is defined to be the number of error occurrences in the later half of a DF. A failure is predicted if one of five heuristic rules fires, which account for several system behaviors. For example, one rule puts a threshold on error-occurrence frequencies, another one on window-averaged occurrence frequency. Yet another rule fires if the EDI is decreasing for four successive DFs and at least one is half the size of its previous frame. If the predicted failure is closer to present time than warning time t w, the prediction has been eliminated. In order to adjust the parameters we optimized each parameter separately. If two choices were almost equal in precision and recall, we took the one with fewer false positives Eventset-based Method Vilalta et al. from IBM Research at T.J. Watson proposed a prediction technique that is based on sets of events preceding a target event [31, 32]. In our case events correspond to errors and target events to failures. Since the authors did not consider any lead time, we had to alter the approach such that the events forming an eventset had to occur at least t l before the failure. An eventset is formed from all error types occurring within the time window (see Figure 6). The database DB of indicative eventsets is algorithmically built by procedures known from data mining. Going through the training dataset, two initial databases are formed: a failure database containing all eventsets preceding failures and a non-failure database with eventsets that occurred between the failures. The database also contains all subsets of eventsets that occurred in the training data, which has at least theoretically cardinality of the power set. However by minimum support thresholding together with branch and bound techniques, cardinality can be limited. The goal is to reduce the database in several steps such that only most indicative eventsets stay in it. To achieve this, all eventsets that have support less than a minimum threshold are filtered out. The remaining eventsets are checked for their confidence, which is in this case the ratio of number of times an eventset occurred in the failure database in comparison to its overall occurrence. This step removes eventsets that occur frequently before failures but also between failures. Additionally, a significance test is performed that removes an eventset if the probability of occurrence in the failure database is similar to the probability of occurrence in the non-failure database. In a last filtering step all remaining eventsets are sorted according to their confidence and all eventsets that are more general and have lower confidence are removed: an eventset Z i is said to be more general than Z j if Z i Z j. 5. Experiment Description Figure 6. Eventset-based method. An eventset is the set of error types that have occurred within a time window of length t d preceding a failure ( ) by lead time t l The goal of the method is to set up a rule-based failure prediction system containing a database of indicative eventsets. For prediction, each time an error occurs, the current set of events Z is formed from all errors that have occurred within t d before present time. The database DB of indicative eventsets is then checked whether Z is a subset of any D DB. If so, a failure warning is raised at time t l in the future. All prediction techniques presented in this paper have been applied to data of a commercial telecommunication system, which main purpose is to realize a so-called Service Control Point (SCP) in an Intelligent Network (IN). An SCP provides services 2 to handle communication related management data such as billing, number translations or prepaid functionality for various services of mobile communication: Mobile Originated Calls (MOC), Short Message Service (SMS), or General Packet Radio Service (GPRS). The fact that the system is an SCP implies that the system cooperates closely with other telecommunication systems in the Global System for Mobile Communication (GSM), but the system does not switch calls itself. It rather has to respond to a large variety of different service requests regarding accounts, billing, etc. submitted to the system over various protocols such as Remote Authentication Dial In User Interface (RADIUS), Signaling System Number 7 (SS7), or Internet Protocol (IP). The system s architecture is a multi-tier architecture employing a component based software design. At the time when measurements were taken the system consisted of more than 1.6 million lines of code, approximately so-called Service Control Functions (SCF)
9 Figure 7. Number of errors per 100 seconds in part of the test data. Diamonds indicate occurrence of failures. components realized by more than 2000 classes, running simultaneously in several containers, each replicated for fault tolerance Failure definition and input data According to [1], a failure is defined as the event when a system ceases to fulfill its specification. Specifications for the telecommunication system require that within successive, non-overlapping five minutes intervals, the fraction of calls having response time longer than 250ms must not exceed 0.01%. This definition is equivalent to a required four-nines interval service availability: srvc. req. w/i 5 min., tr 250ms A i = total no. of srvc. req. w/i 5 min.! 99.99% (15) where t r denotes response time. Hence the failures predicted in this work are performance failures that occur when Equation 15 does not hold. System error logs have been used as input data. The amount of log data per time unit varies heavily, as can be seen in Figure 7. Training data contained (untupled) errors and 27 failures whereas test data had (untupled) errors and 24 failures. Figure 9 shows a histogram of time-between-failures (TBF) and autocorrelation of failure occurrence for the test data Metrics The models ability to predict failures is assessed by four metrics that have an intuitive interpretation: precision, recall, false positive rate and F-measure. Although originally defined in information retrieval these metrics are frequently used for prediction assessment and have been used in similar studies, e.g., [34]. Precision: fraction of correctly predicted failures in comparison to all failure warnings Recall: fraction of correctly predicted failures in comparison to the total number of failures. F-Measure: harmonic mean of precision and recall False positive rate: fraction of false alarms in comparison to all non-failures. Consider the following two examples for clarification: (a) A perfect failure prediction would achieve a one-toone matching between predicted and actual failures which would result in precision = recall = 1 and false positive rate = 0. (b) If a prediction algorithm achieves precision of 0.8, the probability is 80% that any generated failure warning is correct (refers to a true failure) and 20% are false warnings. A recall of 0.9 expresses that 90% of all actual failures are predicted (and 10% are missed). A false positive rate of 0.1 indicates that 10% of predictions that should not result in a failure warning are falsely predicted as failure-prone. Predicted Truth Failure Non-failure Failure True positive False positive (TP) (FP) Non- False negative True negative Failure (FN) (TN) (a) Metric Formula precision p = TP TP+FP recall=true positive rate r = tpr = TP TP+FN false positive rate fpr = FP FP+TN F-measure F = 2 p r p+r (b) Figure 8. Contingency table (a) and definition of metrics (b) In order to estimate the metrics, predictions have been computed for the test data set which have then been compared to the true occurrence of failures and a contingency table has been created (c.f. Figure 8-a). The decision, whether a predicted failure co-incidents with a true failure is based
10 Density ACF TBF [min] Lag [min] Figure 9. Left: Histogram of time between failures (TBF). Right: autocorrelation of failure occurrence. on prediction period: if a true failure occurs within the timeinterval of length t p starting at the predicted time of failure occurrence, the prediction is counted as true positive. We used a prediction period of t p = 60s. All four metrics can then be computed from the contingency table using the formulas provided by Figure 8-b. There is often an inverse proportionality between high recall and high precision. Improving recall in most cases lowers precision and vice versa. The same holds for true positive rate (which is equal to recall) and false positive rate: Increasing true positive rate also increases false positive rate. One of the big advantages of the HSMM method in comparison to the other techniques presented here is that it involves a customizable threshold for classification (c.f., Equation 10). By varying this threshold, the inverse proportionality can be controlled. From this follows that the contingency table looks different for each threshold value and hence also the four metrics change. It is common in this case to visualize precision and recall by a precision/recall plot, and true positive vs. false positive rate by a so-called ROC plot. In order to compare computational complexity of the approaches, we measured computation times. This does not substitute a complexity analysis but since all algorithms have been run on the same machine processing the same data set, the numbers can be used to compare the different approaches. Specifically, we measured if appropriate training time and testing time. Testing time refers to the amount of time needed to process the entire test data in batch mode, which means that the next error is processed immediately after computations for the previous have finished, and not with the delays as they have occurred in the original log. 6. Results In this section, we present results based on applying the presented failure prediction methods to real data of the industrial telecommunication system. The section is structured along the approaches ordered by complexity of the approaches. In the following the term prediction performance is used to denote the overall ability to predict failures as expressed by precision, recall, F-measure and false positive rate (FPR) rather than the number of predictions per time unit Reliability model based prediction The reliability model-based failure prediction approach is rather simple and has been included to provide sort of a lower bound to show what prediction performance can be achieved with nearly no effort. Not surprisingly, prediction performance is low: Precision equals 0.214, recall yielding an F-measure of Since the approach only issues failure warnings (there are no nonfailure-predictions), false positive rate cannot be determined, here. The reason why this prediction method does not really work is that the prediction method is almost periodic: after the occurrence of a true failure, the next failure is predicted at constant time to occur at the median of a distribution that is not adapted during runtime. However, it can be seen from the histogram of time-between-failures that the distribution is wide-spread (Figure 9-a) and from the auto-correlation of failure occurrence that there is no periodicity evident in the data set (Figure 9-b). Since training only comprises computation of the mean of time-between-failures, we did not report training time. The same holds for prediction time.
11 Density number of symbols (a) (b) Figure 10. Histogram of length of sequences (a) and histogram of delays between errors (b) Dispersion Frame Technique Comparing the results of DFT with the original work by Lin and Siewiorek, prediction performance was worse. The main reason for that seems to be the difference of investigated systems: While the original paper investigated failures in the Andrews distributed File System based on the occurrence of host errors, our study applied the technique to errors that had been reported by software components in order to predict upcoming performance failures. In our study, intervals between errors are much shorter (c.f. Figure 10-b). Since DFT predicts a failure half of the length of the dispersion frame ahead, the method rather seldomly predicted a failure far ahead from present time. Since initial results for DFT were quite bad, we modified the thresholds for tupling and for the rules to yield better results. The best combination of parameters we could find resulted in a precision of 0.314, recall of 0.458, F-measure of and FPR of Since there is no real training, we have not reported training time. Running the algorithm on the entire test data set resulted in a computation time of four seconds Eventset-based method Construction of the rule database for the eventset-based method includes the choice of four parameters: length of the data window, level of minimum support, level of confidence, and significance level for the statistical test. We have run the training algorithm for all combinations of various values for the parameters and have selected the optimal combination with respect to F-measure. Best results have been achieved for a window length of 30s, confidence of 10%, support of 50% and significance level of 3σ yielding a recall of 0.917, precision of 0.242, F-measure of , and FPR of This means that the prediction algorithm was able to predict almost all failures of the test data set. However, the problem with the approach is precision: only 24.2% of failure warnings are correct and 75.8% are false alarms. We have also tried to set parameters such that precision and recall become more equal but with no success: recall has been reduced dramatically without substantial improvement in precision. Training took 25s for one specific parameter setting and training time was mainly determined by the a-priori algorithm used for initial database construction. Performing rule checking on the test database took 9s. Our explanation for the results of the eventset-based method is that for the data of the telecommunication system checking solely for the occurrence of error types is not sufficient. It seems as if our basic assumption holds that error patterns need to be considered in order to reliably identify imminent failures Hidden Semi-Markov Models For the experiments we used Hidden Semi-Markov Models with N = 100 states and shortcuts leaping up to five states. The number N has been determined by the length of sequences, of which a histogram is shown in Figure 10-a. Probabilities π have been initialized in a linearly decreasing way such that the first state receives maximum weight and state N receives no initial weight, and then π is adapted by the training algorithm. In order to determine transition duration distributions, we have investigated the distribution of time between errors.
12 data quantiles data quantiles exponential quantiles exponential mixed with uniform (a) (b) Figure 11. QQ-plot of delays between error symbols. (a) plotted against exponential distribution (b) plotted against exponential distribution mixed with uniform distribution Figure 10-b shows a histogram. Most interestingly, by first visual impression the distribution seems to resemble an exponential distribution. Nevertheless, we have investigated various distributions by QQ plots. It turned out that the exponential distribution really yields the best fit. However, the exponential does not fit well in the tail of the distribution. To accommodate for this we have used an exponential distribution with uniform background to account for the tail resulting in a sufficiently good fit. Due to space restrictions, we can only present QQ plots comparing delay distribution with the pure exponential and exponential with background distribution in Figure 11. Other distributions such as lognormal resulted in significant deviations. Regarding observation background distributions we have chosen a distribution reflecting occurrence probability within the entire data set. As mentioned before, one of the advantages of our HSMM approach is that it allows to employ a customizable threshold permitting to control the tradeoff between, e.g., precision and recall. For this reason, we do not report single values for the metrics but provide two plots: a Precision-Recall plot and a ROC curve (see Figure 12). For each point in the Precision-Recall plot, an F-value can be computed resulting in F-measures ranging up to As might have been expected from the description of our approach, computing times for our HSMM method are longer than for the other methods: training the two HSMM models took 47s and predicting the data set took 15s. 7. Conclusions We have introduced a new approach to online failure prediction that forecasts the occurrence of failures by recognition of failure-prone patterns of error events. The prediction model builds on Hidden Semi-Markov Models. In order to compare results, we selected Dispersion Frame Technique (DFT) by Lin and Siewiorek, which evaluates the time when errors have occurred and the eventset-based method introduced by Vilalta et al., which applies data mining techniques on types of errors. As third comparison we have used a reliability model-based failure prediction approach that operates on the occurrence of failures only. All four models have been applied to field data of a commercial telecommunication system and results have been reported in terms of precision, recall, F-measure, and false positive rate. Our experiments have shown a natural correlation of the complexity of prediction approaches and the results that can be achieved: The reliability model based approach is simplest since it analyzes the least amount of information requiring almost no computation power but it results in very poor prediction. DFT is a bit more complex and analyzes a little more data as only last few error messages are taken into account when checking several heuristic rules. This leads to better prediction results. The third method, the eventset-based method, applies a complex analysis using data mining techniques on a training data set. The method investigates sets of error types that have occurred within a time window. Hence the method builds on more detailed information resulting in an astonishing recall of However, the method tends to warn frequently about an upcoming failure even if this is not the case: approximately three fourth of warnings are false alarms. Therefore, precision is rather low. Our HSMM approach is more complex investigating both time and type of error messages, which turns them into temporal sequences of error events. We use hidden semi-markov models which require more complex computations. On the other hand, our HSMM approach out-
13 Figure 12. Prediction performance of the HSMM approach. precision-recall plot (left) and ROC curve (right). The various symbols denote different classification threshold values Prediction technique Precision Recall F-Measure FPR reliability-based n/a DFT Eventset HSMM (max. F-measure) Figure 13. Summary of prediction results performs the other approaches significantly. For example, for the maximum F-value of , it achieves precision of 0.85 and recall of Furthermore, our approach allows to employ a customizable threshold by which the tradeoff between, e.g., precision and recall can be controlled. Acknowledgments This work was supported by Intel Corporation and Deutsche Forschungsgemeinschaft DFG. We would also like to thank the authors and maintainers of GHMM at Max Planck Institute for Molecular Genetics for providing us with an efficient open source implementation of hidden Markov models and support. References [1] A. Avižienis, J.-C. Laprie, B. Randell, and C. Landwehr. Basic concepts and taxonomy of dependable and secure computing. IEEE Transactions on Dependable and Secure Computing, 1(1):11 33, [2] O. Babaoglu, M. Jelasity, A. Montresor, C. Fetzer, S. Leonardi, van Moorsel A., and M. van Steen, editors. Self-Star Properties in Complex Information Systems, volume 3460 of Lecture Notes in Computer Science. Springer- Verlag, [3] L. E. Baum, T. Petrie, G. Soules, and N. Weiss. A maximization technique occurring in the statistical analysis of probabilistic functions of Markov chains. Annals of Mathematical Statistics, 41(1): , [4] S. Brocklehurst and B. Littlewood. New ways to get accurate reliability measures. IEEE Software, 9(4):34 42, [5] A. Brown and D. A. Patterson. To err is human. In Proceedings of the First Workshop on Evaluating and Architecting System dependability (EASY 01), Göteborg, Sweden, Jul [6] A. Csenki. Bayes predictive analysis of a fundamental software reliability model. IEEE Transactions on Reliability, 39(2): , Jun [7] A. Daidone, F. Di Giandomenico, A. Bondavalli, and S. Chiaradonna. Hidden Markov models as a support for diagnosis: Formalization of the problem and synthesis of
14 the solution. In IEEE Proceedings of the 25th Symposium on Reliable Distributed Systems (SRDS 2006), Leeds, UK, Oct [8] R. O. Duda, P. E. Hart, and D. G. Stork. Pattern Classification. Wiley-Interscience, 2 edition, [9] W. Farr. Software reliability modeling survey. In M. R. Lyu, editor, Handbook of software reliability engineering, chapter 3, pages McGraw-Hill, [10] S. Garg, A. Puliafito, M. Telek, and K. Trivedi. Analysis of preventive maintenance in transactions based software systems. IEEE Trans. Comput., 47(1):96 107, [11] S. Garg, A. van Moorsel, K. Vaidyanathan, and K. S. Trivedi. A methodology for detection and estimation of software aging. In Proceedings of the 9th International Symposium on Software Reliability Engineering, ISSRE 1998, Nov [12] I. Gertsbakh. Reliability Theory: with Applications to Preventive Maintenance. Springer-Verlag, Berlin, Germany, [13] K. Hätönen, M. Klemettinen, H. Mannila, P. Ronkainen, and H. Toivonen. Tasa: Telecommunication alarm sequence analyzer, or: How to enjoy faults in your network. In IEEE Proceedings of Network Operations and Management Symposium, volume 2, pages , Kyoto, Japan, Apr [14] G. A. Hoffmann and M. Malek. Call availability prediction in a telecommunication system: A data driven empirical approach. In Proceedings of the 25th IEEE Symposium on Reliable Distributed Systems (SRDS 2006), Leeds, United Kingdom, Oct [15] P. Horn. Autonomic computing: IBM s perspective on the state of information technology, Oct [16] D. Levy and R. Chillarege. Early warning of failures through alarm analysis - a case study in telecom voice mail systems. In ISSRE 03: Proceedings of the 14th International Symposium on Software Reliability Engineering, Washington, DC, USA, IEEE Computer Society. [17] T.-T. Y. Lin. Design and evaluation of an on-line predictive diagnostic system. PhD thesis, Department of Electrical and Computer Engineering, Carnegie-Mellon University, Pittsburgh, PA, Apr [18] T.-T. Y. Lin and D. P. Siewiorek. Error log analysis: statistical modeling and heuristic trend analysis. IEEE Transactions on Reliability, 39(4): , Oct [19] C. Mundie, P. de Vries, P. Haynes, and M. Corwine. Trustworthy computing. Technical report, Microsoft Corp., Oct [20] F. A. Nassar and D. M. Andrews. A methodology for analysis of failure prediction data. In IEEE Real-Time Systems Symposium, pages , [21] D. of Defense. MIL-HDBK-217F Reliability Prediction of Electronic Equipment. Washington D.C., [22] J. Pfefferman and B. Cernuschi-Frias. A nonparametric nonstationary procedure for failure prediction. IEEE Transactions on Reliability, 51(4): , Dec [23] L. R. Rabiner. A tutorial on hidden Markov models and selected applications in speech recognition. Proceedings of the IEEE, 77(2): , Feb [24] F. Salfner. Predicting failures with hidden Markov models. In Proceedings of 5th European Dependable Computing Conference (EDCC-5), pages 41 46, Budapest, Hungary, Apr Student forum volume. [25] F. Salfner. Modeling event-driven time series with generalized hidden semi-markov models. Technical Report 208, Department of Computer Science, Humboldt-Universität zu Berlin, Germany, Available at abstract.php?id= [26] F. Salfner, M. Schieschke, and M. Malek. Predicting failures of computer systems: A case study for a telecommunication system. In Proceedings of IEEE International Parallel and Distributed Processing Symposium (IPDPS 2006), DPDNS workshop, Rhodes Island, Greece, Apr [27] P. Salvo Rossi, G. Romano, F. Palmieri, and G. Iannello. A hidden markov model for internet channels. In IEEE Proceedings of the 3rd International Symposium on Signal Processing and Information Technology (ISSPIT 2003), pages 50 53, Dec [28] D. P. Siewiorek and R. S. Swarz. Reliable Computer Systems. Digital Press, Bedford, MA, second edition, [29] R. M. Singer, K. C. Gross, J. P. Herzog, R. W. King, and S. Wegerich. Model-based nuclear power plant monitoring and fault detection: Theoretical foundations. In Proceedings of Intelligent System Application to Power Systems (ISAP 97), pages 60 65, Seoul, Korea, Jul [30] M. M. Tsao and D. P. Siewiorek. Trend analysis on system error files. In Proc. 13th International Symposium on Fault- Tolerant Computing, pages , Milano, Italy, [31] R. Vilalta, C. V. Apte, J. L. Hellerstein, S. Ma, and S. M. Weiss. Predictive algorithms in the management of computer systems. IBM Systems Journal, 41(3): , [32] R. Vilalta and S. Ma. Predicting rare events in temporal domains. In Proceedings of the 2002 IEEE International Conference on Data Mining (ICDM 02), pages , Washington, DC, USA, IEEE Computer Society. [33] C. Warrender, S. Forrest, and B. Pearlmutter. Detecting intrusions using system calls: alternative data models. In IEEE Proceedings of the 1999 Symposium on Security and Privacy, pages , [34] G. Weiss. Timeweaver: A genetic algorithm for identifying predictive patterns in sequences of events. In Proceedings of the Genetic and Evolutionary Computation Conference, pages , San Francisco, CA, Morgan Kaufmann.
Error Log Processing for Accurate Failure Prediction. Humboldt-Universität zu Berlin
Error Log Processing for Accurate Failure Prediction Felix Salfner ICSI Berkeley Steffen Tschirpke Humboldt-Universität zu Berlin Introduction Context of work: Error-based online failure prediction: errorMore information
Proactive Fault Management
Proactive Fault Management Felix Salfner 7.7.2010 www.rok.informatik.hu-berlin.de/members/salfner Contents Introduction Variable Selection Online Failure Prediction Overview Four Online Failure PredictionMore information
Error Log Processing for Accurate Failure Prediction
Error Log Processing for Accurate Failure Prediction Felix Salfner International Computer Science Institute, Berkeley email@example.com Steffen Tschirpke Humboldt-Universität zu Berlin firstname.lastname@example.com information
Using Data Mining for Mobile Communication Clustering and Characterization
Using Data Mining for Mobile Communication Clustering and Characterization A. Bascacov *, C. Cernazanu ** and M. Marcu ** * Lasting Software, Timisoara, Romania ** Politehnica University of Timisoara/ComputerMore information
Statistics in Retail Finance. Chapter 6: Behavioural models
Statistics in Retail Finance 1 Overview > So far we have focussed mainly on application scorecards. In this chapter we shall look at behavioural models. We shall cover the following topics:- BehaviouralMore information
Evaluation & Validation: Credibility: Evaluating what has been learned
Evaluation & Validation: Credibility: Evaluating what has been learned How predictive is a learned model? How can we evaluate a model Test the model Statistical tests Considerations in evaluating a ModelMore information
Online Failure Prediction in Cloud Datacenters
Online Failure Prediction in Cloud Datacenters Yukihiro Watanabe Yasuhide Matsumoto Once failures occur in a cloud datacenter accommodating a large number of virtual resources, they tend to spread rapidlyMore information
Overview. Evaluation Connectionist and Statistical Language Processing. Test and Validation Set. Training and Test Set
Overview Evaluation Connectionist and Statistical Language Processing Frank Keller firstname.lastname@example.com Computerlinguistik Universität des Saarlandes training set, validation set, test set holdout, stratificationMore information
Finding soon-to-fail disks in a haystack
Finding soon-to-fail disks in a haystack Moises Goldszmidt Microsoft Research Abstract This paper presents a detector of soon-to-fail disks based on a combination of statistical models. During operationMore information
An On-Line Algorithm for Checkpoint Placement
An On-Line Algorithm for Checkpoint Placement Avi Ziv IBM Israel, Science and Technology Center MATAM - Advanced Technology Center Haifa 3905, Israel firstname.lastname@example.com Jehoshua Bruck California InstituteMore information
The Basics of Graphical Models
The Basics of Graphical Models David M. Blei Columbia University October 3, 2015 Introduction These notes follow Chapter 2 of An Introduction to Probabilistic Graphical Models by Michael Jordan. Many figuresMore information
Gamma Distribution Fitting
Chapter 552 Gamma Distribution Fitting Introduction This module fits the gamma probability distributions to a complete or censored set of individual or grouped data values. It outputs various statisticsMore information
Practical Online Failure Prediction for Blue Gene/P: Period-based vs Event-driven
Practical Online Failure Prediction for Blue Gene/P: Period-based vs Event-driven Li Yu, Ziming Zheng, Zhiling Lan Department of Computer Science Illinois Institute of Technology {lyu17, zzheng11, firstname.lastname@example.com information
Azure Machine Learning, SQL Data Mining and R
Azure Machine Learning, SQL Data Mining and R Day-by-day Agenda Prerequisites No formal prerequisites. Basic knowledge of SQL Server Data Tools, Excel and any analytical experience helps. Best of all:More information
Video Affective Content Recognition Based on Genetic Algorithm Combined HMM
Video Affective Content Recognition Based on Genetic Algorithm Combined HMM Kai Sun and Junqing Yu Computer College of Science & Technology, Huazhong University of Science & Technology, Wuhan 430074, ChinaMore information
Practical Data Science with Azure Machine Learning, SQL Data Mining, and R
Practical Data Science with Azure Machine Learning, SQL Data Mining, and R Overview This 4-day class is the first of the two data science courses taught by Rafal Lukawiecki. Some of the topics will beMore information
Detection. Perspective. Network Anomaly. Bhattacharyya. Jugal. A Machine Learning »C) Dhruba Kumar. Kumar KaKta. CRC Press J Taylor & Francis Croup
Network Anomaly Detection A Machine Learning Perspective Dhruba Kumar Bhattacharyya Jugal Kumar KaKta»C) CRC Press J Taylor & Francis Croup Boca Raton London New York CRC Press is an imprint of the TaylorMore information
A Game Theoretical Framework for Adversarial Learning
A Game Theoretical Framework for Adversarial Learning Murat Kantarcioglu University of Texas at Dallas Richardson, TX 75083, USA muratk@utdallas Chris Clifton Purdue University West Lafayette, IN 47907,More information
!"!!"#$$%&'()*+$(,%!"#$%$&'()*""%(+,'-*&./#-$&'(-&(0*".$#-$1"(2&."3$'45"!"#"$%&#'()*+',$$-.&#',/"-0%.12'32./4'5,5'6/%&)$).2&'7./&)8'5,5'9/2%.%3%&8':")08';:More information
Advanced Signal Processing and Digital Noise Reduction
Advanced Signal Processing and Digital Noise Reduction Saeed V. Vaseghi Queen's University of Belfast UK WILEY HTEUBNER A Partnership between John Wiley & Sons and B. G. Teubner Publishers Chichester NewMore information
SIMPLIFIED PERFORMANCE MODEL FOR HYBRID WIND DIESEL SYSTEMS. J. F. MANWELL, J. G. McGOWAN and U. ABDULWAHID
SIMPLIFIED PERFORMANCE MODEL FOR HYBRID WIND DIESEL SYSTEMS J. F. MANWELL, J. G. McGOWAN and U. ABDULWAHID Renewable Energy Laboratory Department of Mechanical and Industrial Engineering University ofMore information
OBJECTIVE ASSESSMENT OF FORECASTING ASSIGNMENTS USING SOME FUNCTION OF PREDICTION ERRORS CLARKE, Stephen R. Swinburne University of Technology Australia One way of examining forecasting methods via assignmentsMore information
Mining the Software Change Repository of a Legacy Telephony System
Mining the Software Change Repository of a Legacy Telephony System Jelber Sayyad Shirabad, Timothy C. Lethbridge, Stan Matwin School of Information Technology and Engineering University of Ottawa, Ottawa,More information
A Tool for Generating Partition Schedules of Multiprocessor Systems
A Tool for Generating Partition Schedules of Multiprocessor Systems Hans-Joachim Goltz and Norbert Pieth Fraunhofer FIRST, Berlin, Germany {hans-joachim.goltz,firstname.lastname@example.com Abstract.More information
FRAUD DETECTION IN ELECTRIC POWER DISTRIBUTION NETWORKS USING AN ANN-BASED KNOWLEDGE-DISCOVERY PROCESS Breno C. Costa, Bruno. L. A. Alberto, André M. Portela, W. Maduro, Esdras O. Eler PDITec, Belo Horizonte,More information
Development of dynamically evolving and self-adaptive software. 1. Background
Development of dynamically evolving and self-adaptive software 1. Background LASER 2013 Isola d Elba, September 2013 Carlo Ghezzi Politecnico di Milano Deep-SE Group @ DEIB 1 Requirements Functional requirementsMore information
Supply chain management by means of FLM-rules
Supply chain management by means of FLM-rules Nicolas Le Normand, Julien Boissière, Nicolas Méger, Lionel Valet LISTIC Laboratory - Polytech Savoie Université de Savoie B.P. 80439 F-74944 Annecy-Le-Vieux,More information
Data Quality Mining: Employing Classifiers for Assuring consistent Datasets
Data Quality Mining: Employing Classifiers for Assuring consistent Datasets Fabian Grüning Carl von Ossietzky Universität Oldenburg, Germany, email@example.com Abstract: IndependentMore information
CS 2750 Machine Learning. Lecture 1. Machine Learning. http://www.cs.pitt.edu/~milos/courses/cs2750/ CS 2750 Machine Learning.
Lecture Machine Learning Milos Hauskrecht firstname.lastname@example.com 539 Sennott Square, x5 http://www.cs.pitt.edu/~milos/courses/cs75/ Administration Instructor: Milos Hauskrecht email@example.com 539 SennottMore information
Monitoring of Internet traffic and applications
Monitoring of Internet traffic and applications Chadi BARAKAT INRIA Sophia Antipolis, France Planète research group ETH Zurich October 2009 Email: email@example.com WEB: http://www.inria.fr/planete/chadiMore information
C. Wohlin, "Managing Software Quality through Incremental Development and Certification", In Building Quality into Software, pp. 187-202, edited by
C. Wohlin, "Managing Software Quality through Incremental Development and Certification", In Building Quality into Software, pp. 187-202, edited by M. Ross, C. A. Brebbia, G. Staples and J. Stapleton,More information
Performance Measures for Machine Learning
Performance Measures for Machine Learning 1 Performance Measures Accuracy Weighted (Cost-Sensitive) Accuracy Lift Precision/Recall F Break Even Point ROC ROC Area 2 Accuracy Target: 0/1, -1/+1, True/False,More information
On Correlating Performance Metrics
On Correlating Performance Metrics Yiping Ding and Chris Thornley BMC Software, Inc. Kenneth Newman BMC Software, Inc. University of Massachusetts, Boston Performance metrics and their measurements areMore information
INTELLIGENT ENERGY MANAGEMENT OF ELECTRICAL POWER SYSTEMS WITH DISTRIBUTED FEEDING ON THE BASIS OF FORECASTS OF DEMAND AND GENERATION Chr.
INTELLIGENT ENERGY MANAGEMENT OF ELECTRICAL POWER SYSTEMS WITH DISTRIBUTED FEEDING ON THE BASIS OF FORECASTS OF DEMAND AND GENERATION Chr. Meisenbach M. Hable G. Winkler P. Meier Technology, LaboratoryMore information
E3: PROBABILITY AND STATISTICS lecture notes
E3: PROBABILITY AND STATISTICS lecture notes 2 Contents 1 PROBABILITY THEORY 7 1.1 Experiments and random events............................ 7 1.2 Certain event. Impossible event............................More information
Private Record Linkage with Bloom Filters
To appear in: Proceedings of Statistics Canada Symposium 2010 Social Statistics: The Interplay among Censuses, Surveys and Administrative Data Private Record Linkage with Bloom Filters Rainer Schnell,More information
Broadband Networks. Prof. Dr. Abhay Karandikar. Electrical Engineering Department. Indian Institute of Technology, Bombay. Lecture - 29.
Broadband Networks Prof. Dr. Abhay Karandikar Electrical Engineering Department Indian Institute of Technology, Bombay Lecture - 29 Voice over IP So, today we will discuss about voice over IP and internetMore information
Data Clustering. Dec 2nd, 2013 Kyrylo Bessonov
Data Clustering Dec 2nd, 2013 Kyrylo Bessonov Talk outline Introduction to clustering Types of clustering Supervised Unsupervised Similarity measures Main clustering algorithms k-means Hierarchical MainMore information
3. Dataset size reduction. 4. BGP-4 patterns. Detection of inter-domain routing problems using BGP-4 protocol patterns P.A.
Newsletter Inter-domain QoS, Issue 8, March 2004 Online monthly journal of INTERMON consortia Dynamic information concerning research, standardisation and practical issues of inter-domain QoS --------------------------------------------------------------------More information
The Optimality of Naive Bayes
The Optimality of Naive Bayes Harry Zhang Faculty of Computer Science University of New Brunswick Fredericton, New Brunswick, Canada email: hzhang@unbca E3B 5A3 Abstract Naive Bayes is one of the mostMore information
Gerard Mc Nulty Systems Optimisation Ltd email@example.com/0876697867 BA.,B.A.I.,C.Eng.,F.I.E.I
Gerard Mc Nulty Systems Optimisation Ltd email@example.com/0876697867 BA.,B.A.I.,C.Eng.,F.I.E.I Data is Important because it: Helps in Corporate Aims Basis of Business Decisions Engineering Decisions EnergyMore information
Chapter 14 Managing Operational Risks with Bayesian Networks
Chapter 14 Managing Operational Risks with Bayesian Networks Carol Alexander This chapter introduces Bayesian belief and decision networks as quantitative management tools for operational risks. BayesianMore information
Less naive Bayes spam detection
Less naive Bayes spam detection Hongming Yang Eindhoven University of Technology Dept. EE, Rm PT 3.27, P.O.Box 53, 5600MB Eindhoven The Netherlands. E-mail:email@example.com also CoSiNe Connectivity SystemsMore information
Health Care and Life Sciences
Sensitivity, Specificity, Accuracy, Associated Confidence Interval and ROC Analysis with Practical SAS Implementations Wen Zhu 1, Nancy Zeng 2, Ning Wang 2 1 K&L consulting services, Inc, Fort Washington,More information
Published in: Proceedings of 6th International Computer Engineering Conference (ICENCO)
Aalborg Universitet Characterization and Modeling of Network Shawky, Ahmed Sherif Mahmoud; Bergheim, Hans ; Ragnarsson, Olafur ; Wranty, Andrzej ; Pedersen, Jens Myrup Published in: Proceedings of 6thMore information
Why Taking This Course? Course Introduction, Descriptive Statistics and Data Visualization. Learning Goals. GENOME 560, Spring 2012
Why Taking This Course? Course Introduction, Descriptive Statistics and Data Visualization GENOME 560, Spring 2012 Data are interesting because they help us understand the world Genomics: Massive AmountsMore information
Intrusion Detection via Machine Learning for SCADA System Protection
Intrusion Detection via Machine Learning for SCADA System Protection S.L.P. Yasakethu Department of Computing, University of Surrey, Guildford, GU2 7XH, UK. firstname.lastname@example.com J. Jiang DepartmentMore information
Failure Prediction in IBM BlueGene/L Event Logs
Seventh IEEE International Conference on Data Mining Failure Prediction in IBM BlueGene/L Event Logs Yinglung Liang, Yanyong Zhang ECE Department, Rutgers University {ylliang, email@example.com information
CHARACTERISTICS IN FLIGHT DATA ESTIMATION WITH LOGISTIC REGRESSION AND SUPPORT VECTOR MACHINES Claus Gwiggner, Ecole Polytechnique, LIX, Palaiseau, France Gert Lanckriet, University of Berkeley, EECS,More information
Nonparametric adaptive age replacement with a one-cycle criterion
Nonparametric adaptive age replacement with a one-cycle criterion P. Coolen-Schrijner, F.P.A. Coolen Department of Mathematical Sciences University of Durham, Durham, DH1 3LE, UK e-mail: email@example.com information
Machine Learning Final Project Spam Email Filtering
Machine Learning Final Project Spam Email Filtering March 2013 Shahar Yifrah Guy Lev Table of Content 1. OVERVIEW... 3 2. DATASET... 3 2.1 SOURCE... 3 2.2 CREATION OF TRAINING AND TEST SETS... 4 2.3 FEATUREMore information
Supplement to Call Centers with Delay Information: Models and Insights
Supplement to Call Centers with Delay Information: Models and Insights Oualid Jouini 1 Zeynep Akşin 2 Yves Dallery 1 1 Laboratoire Genie Industriel, Ecole Centrale Paris, Grande Voie des Vignes, 92290More information
Tail-Dependence an Essential Factor for Correctly Measuring the Benefits of Diversification
Tail-Dependence an Essential Factor for Correctly Measuring the Benefits of Diversification Presented by Work done with Roland Bürgi and Roger Iles New Views on Extreme Events: Coupled Networks, DragonMore information
Simple and efficient online algorithms for real world applications
Simple and efficient online algorithms for real world applications Università degli Studi di Milano Milano, Italy Talk @ Centro de Visión por Computador Something about me PhD in Robotics at LIRA-Lab,More information
Keywords: Dynamic Load Balancing, Process Migration, Load Indices, Threshold Level, Response Time, Process Age.
Volume 3, Issue 10, October 2013 ISSN: 2277 128X International Journal of Advanced Research in Computer Science and Software Engineering Research Paper Available online at: www.ijarcsse.com Load MeasurementMore information
Bootstrapping Big Data
Bootstrapping Big Data Ariel Kleiner Ameet Talwalkar Purnamrita Sarkar Michael I. Jordan Computer Science Division University of California, Berkeley {akleiner, ameet, psarkar, firstname.lastname@example.com information
131-1. Adding New Level in KDD to Make the Web Usage Mining More Efficient. Abstract. 1. Introduction [1]. 1/10
1/10 131-1 Adding New Level in KDD to Make the Web Usage Mining More Efficient Mohammad Ala a AL_Hamami PHD Student, Lecturer m_ah_1@yahoocom Soukaena Hassan Hashem PHD Student, Lecturer soukaena_hassan@yahoocomMore information
Schedule Risk Analysis Simulator using Beta Distribution
Schedule Risk Analysis Simulator using Beta Distribution Isha Sharma Department of Computer Science and Applications, Kurukshetra University, Kurukshetra, Haryana (INDIA) firstname.lastname@example.com Dr. P.K.More information
Protein Protein Interaction Networks
Functional Pattern Mining from Genome Scale Protein Protein Interaction Networks Young-Rae Cho, Ph.D. Assistant Professor Department of Computer Science Baylor University it My Definition of BioinformaticsMore information
Information Theory and Coding Prof. S. N. Merchant Department of Electrical Engineering Indian Institute of Technology, Bombay
Information Theory and Coding Prof. S. N. Merchant Department of Electrical Engineering Indian Institute of Technology, Bombay Lecture - 17 Shannon-Fano-Elias Coding and Introduction to Arithmetic CodingMore information
On the Traffic Capacity of Cellular Data Networks. 1 Introduction. T. Bonald 1,2, A. Proutière 1,2
On the Traffic Capacity of Cellular Data Networks T. Bonald 1,2, A. Proutière 1,2 1 France Telecom Division R&D, 38-40 rue du Général Leclerc, 92794 Issy-les-Moulineaux, France {thomas.bonald, email@example.com information
Vilnius University. Faculty of Mathematics and Informatics. Gintautas Bareikis
Vilnius University Faculty of Mathematics and Informatics Gintautas Bareikis CONTENT Chapter 1. SIMPLE AND COMPOUND INTEREST 1.1 Simple interest......................................................................More information
Failures of software have been identified as the single largest source of unplanned downtime and
Advanced Failure Prediction in Complex Software Systems Günther A. Hoffmann, Felix Salfner, Miroslaw Malek Humboldt University Berlin, Department of Computer Science, Computer Architecture and CommunicationMore information
Hardware Implementation of Probabilistic State Machine for Word Recognition
IJECT Vo l. 4, Is s u e Sp l - 5, Ju l y - Se p t 2013 ISSN : 2230-7109 (Online) ISSN : 2230-9543 (Print) Hardware Implementation of Probabilistic State Machine for Word Recognition 1 Soorya Asokan, 2More information
MINITAB ASSISTANT WHITE PAPER This paper explains the research conducted by Minitab statisticians to develop the methods and data checks used in the Assistant in Minitab 17 Statistical Software. One-WayMore information
I.J.E.M.S., VOL.3(2) 2012: 99-104 ISSN 2229-6425 SOFTWARE PERFORMANCE EVALUATION ALGORITHM EXPERIMENT FOR IN-HOUSE SOFTWARE USING INTER-FAILURE DATA *Jimoh, R. G. & Abikoye, O. C. Computer Science Department,More information
Multiple Linear Regression in Data Mining
Multiple Linear Regression in Data Mining Contents 2.1. A Review of Multiple Linear Regression 2.2. Illustration of the Regression Process 2.3. Subset Selection in Linear Regression 1 2 Chap. 2 MultipleMore information
TIME DEPENDENT PRIORITIES IN CALL CENTERS LASSAAD ESSAFI and GUNTER BOLCH Institute of Computer Science University of Erlangen-Nuremberg Martensstrasse 3, D-91058 Erlangen, Germany E-mail : firstname.lastname@example.com,More information
STATISTICA Formula Guide: Logistic Regression. Table of Contents
: Table of Contents... 1 Overview of Model... 1 Dispersion... 2 Parameterization... 3 Sigma-Restricted Model... 3 Overparameterized Model... 4 Reference Coding... 4 Model Summary (Summary Tab)... 5 SummaryMore information
Assessing Schedule Performance with Earned Value
FEATURED PAPER OCTOBER 2007 Time Is Up: Assessing Schedule Performance with Earned Value By Robert Van De Velde Time drives projects. Quick and accurate evaluation of schedule performance is crucial. QuantitativeMore information
A C T R esearcli R e p o rt S eries 2 0 0 5. Using ACT Assessment Scores to Set Benchmarks for College Readiness. IJeff Allen.
A C T R esearcli R e p o rt S eries 2 0 0 5 Using ACT Assessment Scores to Set Benchmarks for College Readiness IJeff Allen Jim Sconing ACT August 2005 For additional copies write: ACT Research ReportMore information
SureSense Software Suite Overview
SureSense Software Overview Eliminate Failures, Increase Reliability and Safety, Reduce Costs and Predict Remaining Useful Life for Critical Assets Using SureSense and Health Monitoring Software What SureSenseMore information
Local outlier detection in data forensics: data mining approach to flag unusual schools
Local outlier detection in data forensics: data mining approach to flag unusual schools Mayuko Simon Data Recognition Corporation Paper presented at the 2012 Conference on Statistical Detection of PotentialMore information
Auxiliary Variables in Mixture Modeling: 3-Step Approaches Using Mplus
Auxiliary Variables in Mixture Modeling: 3-Step Approaches Using Mplus Tihomir Asparouhov and Bengt Muthén Mplus Web Notes: No. 15 Version 8, August 5, 2014 1 Abstract This paper discusses alternativesMore information
Towards better accuracy for Spam predictions
Towards better accuracy for Spam predictions Chengyan Zhao Department of Computer Science University of Toronto Toronto, Ontario, Canada M5S 2E4 firstname.lastname@example.com Abstract Spam identification is crucialMore information
1. Classification problems
Neural and Evolutionary Computing. Lab 1: Classification problems Machine Learning test data repository Weka data mining platform Introduction Scilab 1. Classification problems The main aim of a classificationMore information
Comparing the Results of Support Vector Machines with Traditional Data Mining Algorithms
Comparing the Results of Support Vector Machines with Traditional Data Mining Algorithms Scott Pion and Lutz Hamel Abstract This paper presents the results of a series of analyses performed on direct mailMore information
Measuring the Performance of an Agent
25 Measuring the Performance of an Agent The rational agent that we are aiming at should be successful in the task it is performing To assess the success we need to have a performance measure What is rationalMore information
CANADIAN APPLIED MATHEMATICS QUARTERLY Volume 12, Number 4, Winter 2004 ANALYZING NETWORK TRAFFIC FOR MALICIOUS ACTIVITY SURREY KIM, 1 SONG LI, 2 HONGWEI LONG 3 AND RANDALL PYKE Based on work carried outMore information
Social Media Mining. Data Mining Essentials
Introduction Data production rate has been increased dramatically (Big Data) and we are able store much more data than before E.g., purchase data, social media data, mobile phone data Businesses and customersMore information
Learning outcomes. Knowledge and understanding. Competence and skills
Syllabus Master s Programme in Statistics and Data Mining 120 ECTS Credits Aim The rapid growth of databases provides scientists and business people with vast new resources. This programme meets the challengesMore information
Information Leakage in Encrypted Network Traffic
Information Leakage in Encrypted Network Traffic Attacks and Countermeasures Scott Coull RedJack Joint work with: Charles Wright (MIT LL) Lucas Ballard (Google) Fabian Monrose (UNC) Gerald Masson (JHU)More information
CS229 Project Report Automated Stock Trading Using Machine Learning Algorithms
CS229 roject Report Automated Stock Trading Using Machine Learning Algorithms Tianxin Dai email@example.com Arpan Shah firstname.lastname@example.com Hongxia Zhong firstname.lastname@example.com 1. IntroductionMore information
Neural Networks and Back Propagation Algorithm
Neural Networks and Back Propagation Algorithm Mirza Cilimkovic Institute of Technology Blanchardstown Blanchardstown Road North Dublin 15 Ireland firstname.lastname@example.com Abstract Neural Networks (NN) are importantMore information
Enhancing Quality of Data using Data Mining Method
JOURNAL OF COMPUTING, VOLUME 2, ISSUE 9, SEPTEMBER 2, ISSN 25-967 WWW.JOURNALOFCOMPUTING.ORG 9 Enhancing Quality of Data using Data Mining Method Fatemeh Ghorbanpour A., Mir M. Pedram, Kambiz Badie, MohammadMore information
Examining Self-Similarity Network Traffic intervals
Examining Self-Similarity Network Traffic intervals Hengky Susanto Byung-Guk Kim Computer Science Department University of Massachusetts at Lowell {hsusanto, firstname.lastname@example.com Abstract Many studies haveMore information
Analysis of Load Frequency Control Performance Assessment Criteria
520 IEEE TRANSACTIONS ON POWER SYSTEMS, VOL. 16, NO. 3, AUGUST 2001 Analysis of Load Frequency Control Performance Assessment Criteria George Gross, Fellow, IEEE and Jeong Woo Lee Abstract This paper presentsMore information
Segmentation and Classification of Online Chats
Segmentation and Classification of Online Chats Justin Weisz Computer Science Department Carnegie Mellon University Pittsburgh, PA 15213 firstname.lastname@example.com Abstract One method for analyzing textual chatMore information
Modelling, Extraction and Description of Intrinsic Cues of High Resolution Satellite Images: Independent Component Analysis based approaches
Modelling, Extraction and Description of Intrinsic Cues of High Resolution Satellite Images: Independent Component Analysis based approaches PhD Thesis by Payam Birjandi Director: Prof. Mihai Datcu ProblematicMore information
Penalized regression: Introduction
Penalized regression: Introduction Patrick Breheny August 30 Patrick Breheny BST 764: Applied Statistical Modeling 1/19 Maximum likelihood Much of 20th-century statistics dealt with maximum likelihoodMore information
Predict the Popularity of YouTube Videos Using Early View Data
000 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050More information
Process Intelligence: An Exciting New Frontier for Business Intelligence
February/2014 Process Intelligence: An Exciting New Frontier for Business Intelligence Claudia Imhoff, Ph.D. Sponsored by Altosoft, A Kofax Company Table of Contents Introduction... 1 Use Cases... 2 BusinessMore information
Example G Cost of construction of nuclear power plants
1 Example G Cost of construction of nuclear power plants Description of data Table G.1 gives data, reproduced by permission of the Rand Corporation, from a report (Mooz, 1978) on 32 light water reactorMore information
Module 1: Introduction to Computer System and Network Validation
Module 1: Introduction to Computer System and Network Validation Module 1, Slide 1 What is Validation? Definition: Valid (Webster s Third New International Dictionary) Able to effect or accomplish whatMore information
Example: Credit card default, we may be more interested in predicting the probabilty of a default than classifying individuals as default or not.
Statistical Learning: Chapter 4 Classification 4.1 Introduction Supervised learning with a categorical (Qualitative) response Notation: - Feature vector X, - qualitative response Y, taking values in CMore information
How I won the Chess Ratings: Elo vs the rest of the world Competition
How I won the Chess Ratings: Elo vs the rest of the world Competition Yannis Sismanis November 2010 Abstract This article discusses in detail the rating system that won the kaggle competition Chess Ratings:More information
Support Vector Machines for Dynamic Biometric Handwriting Classification
Support Vector Machines for Dynamic Biometric Handwriting Classification Tobias Scheidat, Marcus Leich, Mark Alexander, and Claus Vielhauer Abstract Biometric user authentication is a recent topic in theMore information
Exploratory Data Analysis
Exploratory Data Analysis Johannes Schauer email@example.com Institute of Statistics Graz University of Technology Steyrergasse 17/IV, 8010 Graz www.statistics.tugraz.at February 12, 2008 IntroductionMore information
Applying Machine Learning to Stock Market Trading Bryce Taylor
Applying Machine Learning to Stock Market Trading Bryce Taylor Abstract: In an effort to emulate human investors who read publicly available materials in order to make decisions about their investments,More information
Statistical Models in R
Statistical Models in R Some Examples Steven Buechler Department of Mathematics 276B Hurley Hall; 1-6233 Fall, 2007 Outline Statistical Models Structure of models in R Model Assessment (Part IA) AnovaMore information


text:
New York to LA
So if you were following our Instagram posts over February and March you will have noticed us all over America. We took it upon ourselves to devote 2 months to 'market research' and drive from New York to LA, eating our way along the southern route across the states. We ate Pizza in NY, Ribs in Memphis, Jambalaya in New Orleans, Tacos in LA and tried a handful of BBQ joints across Texas. We started off with snow in New York, and made it through desert conditions all in the name of food sampling. If you're heading to the states and want any recommendations about places to go, places to eat or what to see give us a shout!

subset: commoncrawl


text:
Nearby Cities
Sat Nov 23 2019 at 07:30 pm to 09:30 pm
University of Memphis Rudi E. Scheidt School of Music, 3775 Central Avenue 129 Music Building, Memphis, Tennessee 38152, Tennessee, United States
Get Weekly Email Updates. No Spam, Promise!
E.g. Jack is first name and Mandanka is last name.


text:
Kathryn Rock is a director within the consumer finance practice at Navigant. Kate is a certified public accountant with more than 13 years of experience advising banking and government organizations and auditing various asset management clients, including hedge funds and private equity companies.
Kate is a proven engagement leader who provides solutions to clients managing major financial, operational, regulatory, and strategy changes. She has successfully managed large, complex projects and has developed significant expertise through her experience in providing risk management, regulatory compliance, finance transformation, accounting, financial reporting, and project management consulting services.
Kate has recently managed large-scale regulatory change and compliance-driven projects for a large U.S. mortgage servicer and a government organization.
Kate's areas of expertise include:
Prior to Navigant, Kate was an accomplished audit professional at a Big Four professional services firm. Kate performed external audits for multiple financial services clients.


text:
You Are Here: Home > Samuel Windsor > Cashmere Mix
Order before 3pm for
same day despatch.
We despatch 7 days a week
Customer Favourites
Cashmere Mix Jumper - Crew Neck - Garnet
Cashmere Mix Polo Jumper - Grey
Cashmere Mix Polo Jumper - Navy
Cashmere Mix Jumper - Crew Neck - Purple
Cashmere Mix Polo Jumper - Garnet
Cashmere Mix Jumper - Crew Neck - Sky
Cashmere Mix Tank Top - Garnet
Cashmere Mix Polo Jumper - Garnet
£27.50 Was £550 reviews
Whether wearing to the office as part of a formal outfit or simply smartening up your casual weekend attire, our long-sleeved polo jumpers are a wardrobe essential that can be worn all year round. 95 per cent cotton and 5 per cent cashmere. Machine washable at 30°.
Sizes for jumpers (to fit chest): Small (up to 36-38"), Medium (39-42"), Large (43-44"), X Large (45-48"), XX Large (49-50"), XXX Large (51-52").
Standard Delivery - From only £3.95
Please note standard delivery on all items is only £3.95 (for orders of £20 or less) and £4.95 (for orders over £20), delivery charge applies per order regardless of number of items purchased. Delivery to Northern Ireland, Isle of Man, Scottish Highlands and Islands is £6.95. Delivery to the Channel Islands, Scilly Isles and Eire is £10.95.
Express Delivery - From only £5.95
We offer a comprehensive range of express delivery options for a small additional charge. Simply choose the Express Delivery option on the checkout to view the delivery services available for your area.
Satisfaction Guaranteed
We want you to be 100% happy with your purchase from us, If you're not entirely satisfied with anything, for whatever reason, we will exchange or give you a full refund for shoe and clothing products returned to us, in mint condition, within 30 days.
Cashmere / Wool Mix Jumper
Units: Inches
Sleeve Length: This is the measurement from the shoulder seam to the end of the cuff
Shirt Length: Shirt length is taken from the base of the collar in the centre of the shirt to the bottom hem at the back
Fisherman's Jumpers
Chunky Knit Jumpers
Sleeve Length: Sleeve length is taken from the underarm seam to end of the cuff.
Shirt Length: Shirt length is taken from the base of the collar in the centre of the shirt to the bottom hem at the back.
Need Help?
Useful information
Product Information
Customer Reviews
|Contact Us Delivery FAQ
|Privacy Policy Cookies Popular Searches Terms & Conditions Offers Terms & Conditions Disclaimer Mobile Site
|Returns Policy Sitemap Customer Reviews
Our current customer satisfaction rating is an excellent...
Call us on: 0844 249 8096*


text:
1- When should we leave for the honeymoon?
The anticipation of the honeymoon is sometimes becomes greater than the anticipations of the wedding. The idea of going away with each other to a wonderful destination makes you want to go as soon as possible. Keep in mind several things when planning your departure date for your honeymoon. If your wedding is a Friday night wedding you are more than likely going to be up late at the reception celebrating with friends and family. Most flights depart early in the morning add on to the early arrival time that you have calculate for check in you may be lucky to get 3 hours of sleep in. Why not plan on having a relaxing day after the wedding either at home or at a hotel in the city near the airport you will depart from. Leaving 2 after the honeymoon keeps you from exhaustion this way you can arrive at your destination refreshed and ready to relax and enjoy the honeymoon.
2- Who pays and plans the honeymoon?
Traditionally the groom is responsible for paying for and planning the honeymoon. The bride and her family are typically the ones paying for and planning the wedding. By the groom planning the honeymoons this frees up the bride from another task on her ever so long to do list. There are lots of couples that do pay for the honeymoon together as a couple however the planning of the honeymoon should still be on the responsibility of the groom. The groom can keep the honeymoon destination a surprise or the details a surprise. The couple may also do the planning together however one person should be in charge of the details. The groom or the couple does not have to do the planning alone. Just like the bride will hire a wedding planner there are travel professionals that do specialize in planning honeymoons. Travel professionals bring a wealth of knowledge and expertise which will make the honeymoon planning process effortless.
3- What is a honeymoon registry and is it okay to have one?
Just like the registry you have for your china and household goods there is also now something called a honeymoon registry. This is all done virtually through an online host. There are several honeymoon registries to choose from. Honeyfund.com is a great one that has a free and a paid version. Guest give you money to go towards your honeymoon. The funds are deposited into an account that you collect on a specific date using a direct transfer to your bank or pay-pal. Some resorts such as Sandals have a direct honeymoon registry for guests staying at any of their resorts. You can use the money to go directly towards the honeymoon cost or use it for your excursions, spa treatments or specialty dining.
There are some different opinions on having a honeymoon registry. You know there will be that Aunt that has to get you a place setting or silverware. If you happen to be getting married a second time then a honeymoon registry is a great way for loved ones to give you a gift that you and your fiancé will love and use since you are more than likely already combining two household of "stuff".
4- How do we decide on a place to go that we both will enjoy?
This is a very important question. One that should take time on both parties part to discuss, contemplate and answer. Think about the places that you have been individually. Have you already traveled together? Each of you should write out your top 5 list of places that you want to go without the other one seeing it, then compare notes. Do you have any destinations that match? Do you have all completely different answers? You will need to discuss why you want to visit those destinations.
You will also need to think about how you want to spend your honeymoon. Do you want to spend it in Europe where a lot of walking and touring are entailed? Do you want to spend it in the snow being active on the slopes? Or do you want to relax and rest on the beach? Consider that you will both possibly be somewhat exhausted from all of the wedding festivities that have taken place over the previous months.
The key point to deciding on a place to go is communication with each other. Makes sure you both want to go so you will both enjoy the honeymoon.
5- Is it ok to go off the grid?
Absolutely! You do not even have to tell your friends and family where you are going. You do want to leave information with an emergency contact but that is all. This is your time. Even if the resort you will be staying at offers WiFi do your best to avoid being active on social media, texting and checking emails. This is time for you to be in the moment. This is the time to create memories that will last a lifetime. You can upload pictures at a later time to show friends and family what a great time you had and the beautiful scenery. Phone calls, texts and emails can all wait until you return. Friends and family will respect your wishes. Be in the present moment on your honeymoon. Enjoy the company of one another leave the electronics in the safe. Listen to the sounds that surround you. Look at the beauty that you are immersed in. Smell the aromas of flowers, the sea, the mountains or wherever you may be. Allow yourself to feel and enjoy the sensations of the emotions that you are feeling with each other.
Leave a Reply.
Vertical Divider
"Southern celebrations hold a special place in our hearts. From the Spanish moss on old oak trees and beautiful magnolia blooms, to celebrating time old traditions with family and friends. Southern Celebrations Magazine is a lifestyle publication celebrating every special occasion that we will encounter throughout our life!"


text:
Avatrex™ – Award Winning Technology
Imagine printing one material to produce all kinds of specialty graphics for events, trade shows, retail environments, promotional materials, labels and packaging, crafts, book covers, and much more.
Imagine minimizing downtime and maximizing output.
Imagine being able to produce applications you couldn't before because you didn't have the right equipment. Avatrex Transportable Imaging Technology makes all of this a reality. Avatrex is not a paper and it's not a film. You could say it's a printable adhesive. Ultimately, it's a technology that adapts to whatever you need it to do. And, since Avatrex Transportable Imaging takes on the color and texture of the substrate, heavily textured or embossed surfaces can now be decorated with full-color, photographic images and graphics.
Avatrex is currently available in both glossy and matte versions for aqueous, solvent, low-solvent, UV-curable and latex wide-format inkjet printers, and HP Indigo presses.
Avatrex 6500 Matte for HP Indigo presses was awarded a 3 Star rating by RIT HP Indigo WS 6000, WS6600, WS2000, WS4000, WS4600 and WS4050/4500 presses in February 2013. In May, 2013, Avatrex was the first substrate to be certified on the HP Indigo 5600 single-shot and with a 3-Star rating.
RIT Labels & Packaging Substrate Evaluation Report: Available here
RIT HP 5600 Single-Shot Certification: Available here
RIT HP 5600 Single-Shot Report: Available here
Avatrex technology was also named a recipient of a Print Industries of America 2012 InterTech Technology Award that judges selected for being truly innovative and for the significant impact the technology is expected to have on the graphic communications industry.


text:
USD U.S. dollar
In the centre of the most prestigious commercial area of Barcelona, in one of its most peaceful street, you will find the Hotel Derby, which links business and pleasure, offering a great comfort and the best services.
The hotel reserves the right to apply special cancellation policies to bookings for more than 5 rooms for the same nights.
Epson Restaurant
Ideal for your banquettes and private meetings, the restaurant will satisfy the most refined palates in an elegant setting where you will find some lithographic works by Miró.
The times
Enjoy a good coffee or a tea in our coffee-house whenever you prefer, or enjoy a lunch or a dinner in our elegant atmosphere.
Epson bar
An authentic Scottish bar, where you can end you day having a drink, a cocktail, some snacks…
There are three saloons of different capacity in the entrepreneurial centre of Barcelona, which create an ideal environment for business, with works of contemporary art and all the services at your disposal.
A delicious and relaxing location, with services and quality that will make you love one of the most dynamic city of Europe. Our guests will have at their disposal the access to the swimming-pool of the Gran Derby (at only 10 metres), rooms and public spaces for smokers, Internet wireless access, business centre, car park, 3 meeting rooms, bar-garden next to the swimming-pool, buffet breakfast and free national and international press service. Enjoy the room service, washing-machine and iron. Guests will have the free access to the Egyptian Art Museum of Barcelona. We have also an anti-fire system. Dogs are accepted.
Give us your Feedback


text:
England remain on course to qualify for the UEFA European Under-21 Championship play-offs after Nathan Redmond's second-half strike secured a narrow 1-0 win over Wales.
Both sides had chances to take the lead during an even opening 45 minutes in Derby, but Redmond's powerful effort in the 56th minute sealed victory to send Gareth Southgate's men six points clear of Moldova in qualifying Group 1.
Ravel Morrison fired narrowly wide from just outside the area on 25 minutes for the hosts but five minutes later the lively Wesley Burns beat England defender John Stones for pace and broke clear, only to drag his low attempt off target. The breakthrough arrived 11 minutes after the restart when Redmond, who had been a threat down both wings all night, cut inside from the left and unleashed a low shot that crept inside the near post.
Despite continuing to dominate for long periods, England failed to find a second goal to kill the game and Wales went close to nicking an equaliser but Lee Evans saw his long-range drive superbly tipped away by Jack Butland. "That was the toughest test we've had to come through," said manager Southgate. "Nathan Redmond has earned his chance to start. He has been playing well for his club and it's pleasing for him to get the goal. It will give him a big confidence boost."
©UEFA.com 1998-2011. All rights reserved.


text:
Kaatsbaan is a nonprofit, professional creative residence and performance facility situated on a 153-acre historic site in Tivoli, NY. Founded in 1990, it provides dance companies, choreographers, composers, set designers and all dance artists with a setting where they can create and showcase new work, rehearse, perform and develop new productions. It serves dance communities across the US and around the world.
Kaatsbaan International Dance Center will present nøa dance, an organization that unites diverse artists to create compelling contemporary dance performance. Since it's founding in 2004, nøa dance has developed its activities as an international contemporary dance project with its current home base in NYC.
nøa dance
Tuesday, October 6-Sunday, October 11, 2009
Open Rehearsal – Thursday, October 8, 2009 at 2 pm – Free open to the public.
Performances Saturday, October 10, 2009 at 7:30 p.m. and Sunday, October 11, 2009 at 2:30 p.m.
TICKETS: $25 Reserved Seating
$10 Student Rush at the Door
WHERE: Kaatsbaan International Dance Center
120 Broadway, Tivoli, NY
HOW: Reservations 845-757-5106