text:
Support Vector Data Description
- August Banks
- 3 years ago
- Views:
1 Machine Learning, 54, 45 66, 2004 c 2004 Kluwer Academic Publishers. Manufactured in The Netherlands. Support Vector Data Description DAVID M.J. TAX ROBERT P.W. DUIN Pattern Recognition Group, Faculty of Applied Sciences, Delft University of Technology, Lorentzweg 1, 2628 CJ Delft, The Netherlands Editor: Douglas Fisher Abstract. Data domain description concerns the characterization of a data set. A good description covers all target data but includes no superfluous space. The boundary of a dataset can be used to detect novel data or outliers. We will present the Support Vector Data Description (SVDD) which is inspired by the Support Vector Classifier. It obtains a spherically shaped boundary around a dataset and analogous to the Support Vector Classifier it can be made flexible by using other kernel functions. The method is made robust against outliers in the training set and is capable of tightening the description by using negative examples. We show characteristics of the Support Vector Data Descriptions using artificial and real data. Keywords: outlier detection, novelty detection, one-class classification, support vector classifier, support vector data description 1. Introduction Much effort has been expended to solve classification and regression tasks. In these problems a mapping between objects represented by a feature vector and outputs (a class label or real valued outputs) is inferred on the basis of a set of training examples. In practice, another type of problem is of interest too: the problem of data description or One-Class Classification (Moya, Koch, & Hostetler, 1993). Here the problem is to make a description of a training set of objects and to detect which (new) objects resemble this training set. Obviously data description can be used for outlier detection to detect uncharacteristic objects from a data set. Often these outliers in the data show an exceptionally large or small feature value in comparison with other training objects. In general, trained classifiers or regressors only provide reliable estimates for input data close to the training set. Extrapolations to unknown and remote regions in feature space are very uncertain (Roberts & Penny, 1996). Neural networks, for instance, can be trained to estimate posterior probabilities (Richard & Lippmann, 1991; Bishop, 1995; Ripley, 1996) and tend to give high confidence outputs for objects which are remote from the training set. In these cases outlier detection should be used first to detect and reject outliers to avoid unfounded confident classifications. Secondly, data description can be used for a classification problem where one of the classes is sampled very well, while the other class is severely undersampled. An example is a machine monitoring system in which the current condition of a machine is examined. An alarm is raised when the machine shows a problem. Measurements on the normal working conditions of a machine are very cheap and easy to obtain. Measurements of outliers, on
2 46 D.M.J. TAX AND R.P.W. DUIN the other hand, would require the destruction of the machine in all possible ways. It is very expensive, if not impossible, to generate all faulty situations (Japkowicz, Myers, & Gluck, 1995). Only a method which uses mainly the target data, and does not require representative outlier data, can solve this monitoring problem. The last possible use of outlier detection is the comparison of two data sets. Assume a classifier has been trained (by a long and difficult optimization) on some (possibly expensive) data. When a similar problem has to be solved, the new data set can be compared with the old training set. In case of incomparable data, the training of a new classifier will be needed. In many one-class classification problems an extra complication occurs, namely that it is beforehand not clear what the specific distribution of the data will be in practice. The operator of the machine can easily run the machine in different, but legal modes, covering the complete operational area of the machine. Although it defines the normal working area, the distribution over this area is not to be trusted. In practice the machine may stay much longer in one mode than in another, and this mode might have been sampled sparsely during the training phase. A data description for this type of data should therefore model the boundary of the normal class, instead of modeling the complete density distribution. Several solutions for solving the data description problem have been proposed. Most often the methods focus on outlier detection. Conceptually the simplest solution for outlier detection is to generate outlier data around the target set. An ordinary classifier is then trained to distinguish between the target data and outliers (Roberts et al., 1994). Koch et al. (1995) used ART-2A and a Multi-Layered Perceptron for the detection of (partially obscured) objects in an automatic target recognition system. Unfortunately this method requires the availability of a set of near-target objects (possibly artificial) belonging to the outlier class. The methods scale very poorly in high dimensional problems, especially when the near-target data has to be created and is not readily available. In classification or regression problems a more advanced Bayesian approach can be used for detecting outliers (Bishop, 1995; MacKay, 1992; Roberts & Penny, 1996). Instead of using the most probable weight configuration of a classifier (or regressor) to compute the output, the output is weighted by the probability that the weight configuration is correct given the data. This method can then provide an estimate of the probability for a certain object given the model family. Low probabilities will then indicate a possible outlier. The classifier outputs are moderated automatically for objects remote from the training domain. These methods are not optimized for outlier detection though; they require a classification (or regression) task to be solved and can be computationally expensive. Most often the task of data description is solved by estimating a probability density of the target data (Barnett & Lewis, 1978). For instance in Bishop (1994) and Tarassenko, Hayton, and Brady (1995) the density is estimated by a Parzen density estimator, whereas in Parra, Deco, and Miesbach (1996) one Gaussian distribution is used. In (Ritter & Gallegos, 1997) not only the target density is estimated, but also the outlier density. The first drawback is that in general (in higher dimensional feature spaces) a large number of samples is required. The second drawback is that they will not be very resistant to training data which only defines the area of the target data, and which does not represent the complete density distribution. It will mainly focus on modeling the high density areas, and reject low density areas, although they may define legal target data.
3 SUPPORT VECTOR DATA DESCRIPTION 47 Vapnik argued that in order to solve a problem, one should not try to solve a more general problem as an intermediate step (Vapnik, 1998). The estimation of the complete density instead of computing the boundary around a data set might require too much data and could result in bad descriptions. An attempt to train just the boundaries of a data set is made in Moya and Hush (1996) and Moya, Koch, and Hostetler (1993). Here neural networks are trained with extra constraints to give closed boundaries. Unfortunately this method inherits the weak points in neural network training, i.e. the choice of the size of the network, weight initialization, the stopping criterion, etc. Rosen (1965) made a data description as a by-product of a classification problem. He shows how the classification problem can be formulated as a convex programming problem. When the classification problem is not linearly separable, an ellipsoidal separation can be applied, where one of the classes is enclosed by an ellipsoid of minimum size. We will use a similar method for the one-class classification problem. Finally Schölkopf et al. used an hyperplane (Schölkopf et al., 1999b) to separate the target objects from the origin with maximal margin. This formulation is comparable with the Support Vector Classifier by Vapnik (1998) and it is possible to define implicit mappings to obtain more flexible descriptions. In this paper we discuss a method which obtains a spherically shaped boundary around the complete target set with the same flexibility. To minimize the chance of accepting outliers, the volume of this description is minimized. We will show how the outlier sensitivity can be controlled by changing the ball-shaped boundary into a more flexible boundary and how example outliers can be included into the training procedure (when they are available) to find a more efficient description. In Section 2 we present the basic theory, which is presented in part in Tax and Duin (1999). The normal data description, and also the description using negative examples will be explained and we will compare the method with the hyperplane approach in Schölkopf et al. (1999b). In Section 3 some basic characteristics of this data description will be shown, and in Section 4 the method will be applied to a real life problem and compared with some other density estimation methods. Section 5 contains some conclusions. 2. Theory To begin, we fix some notation. We assume vectors x are column vectors and x 2 = x x. We have a training set {x i }, i = 1,...,N for which we want to obtain a description. We further assume that the data shows variances in all feature directions Normal data description To start with the normal data description, we define a model which gives a closed boundary around the data: an hypersphere. The sphere is characterized by center a and radius R > 0. We minimize the volume of the sphere by minimizing R 2, and demand that the sphere contains all training objects x i. This is identical to the approach which is used in Schölkopf, Burges, and Vapnik (1995) to estimate the VC-dimension of a classifier (which is bounded by the diameter of the smallest sphere enclosing the data). At the end of this section we will
4 48 D.M.J. TAX AND R.P.W. DUIN show that this approach gives solutions similar to the hyperplane approach of Schölkopf et al. (1999b). Analogous to the Support Vector Classifier (Vapnik, 1998) we define the error function to minimize: F(R, a) = R 2 (1) with the constraints: x i a 2 R 2, i (2) To allow the possibility of outliers in the training set, the distance from x i to the center a should not be strictly smaller than R 2,but larger distances should be penalized. Therefore we introduce slack variables ξ i 0 and the minimization problem changes into: F(R, a) = R 2 + C i ξ i (3) with constraints that almost all objects are within the sphere: x i a 2 R 2 + ξ i, ξ i 0 i (4) The parameter C controls the trade-off between the volume and the errors. Constraints (4) can be incorporated into Eq. (3) by using Lagrange multipliers: L(R, a,α i,γ i,ξ i ) = R 2 + C i ξ i i α i {R 2 + ξ i ( x i 2 2a x i + a 2 )} i γ i ξ i (5) with the Lagrange multipliers α i 0 and γ i 0. L should be minimized with respect to R, a,ξ i and maximized with respect to α i and γ i. Setting partial derivatives to zero gives the constraints: L R = 0: L a = 0: a = α i = 1 (6) i i α ix i i α = α i x i i i (7) L = 0: ξ i C α i γ i = 0 (8) From the last equation α i = C γ i and because α i 0,γ i 0, Lagrange multipliers γ i can be removed when we demand that 0 α i C (9)
5 SUPPORT VECTOR DATA DESCRIPTION 49 Resubstituting (6) (8) into (5) results in: L = i α i (x i x i ) i, j α i α j (x i x j ) (10) subject to constraints (9). Maximizing (10) gives a set α i. When an object x i satisfies the inequality x i a 2 < R 2 + ξ i, the constraint is satisfied and the corresponding Lagrange multiplier will be zero (α i = 0). For objects satisfying the equality x i a 2 = R 2 + ξ i the constraint has to be enforced and the Lagrange multiplier will become unequal zero (α i > 0). x i a 2 < R 2 α i = 0,γ i = 0 (11) x i a 2 = R 2 0 <α i < C,γ i = 0 (12) x i a 2 > R 2 α i = C,γ i > 0 (13) Equation (7) shows that the center of the sphere is a linear combination of the objects. Only objects x i with α i > 0 are needed in the description and these objects will therefore be called the support vectors of the description (SV s). To test an object z, the distance to the center of the sphere has to be calculated. A test object z is accepted when this distance is smaller or equal than the radius: z a 2 = (z z) 2 i α i (z x i ) + i, j α i α j (x i x j ) R 2 (14) By definition, R 2 is the distance from the center of the sphere a to (any of the support vectors on) the boundary. Support vectors which fall outside the description (α i = C) are excluded. Therefore: R 2 = (x k x k ) 2 i α i (x i x k ) + i, j α i α j (x i x j ) (15) for any x k SV <C, the set of support vectors which have α k < C. Note that in all formulae (Eqs. (10), (14) and (15)) objects x i only appear in the form of inner products with other objects (x i x j ). Analogous to Vapnik (1998) the inner products can be replaced by a kernel function to obtain more flexible methods. We will use this in Section 2.3. The left plot in figure 1 shows an example description for a small 2 dimensional bananashaped data set. For this description C = 1 is taken, indicating the hard-margin solution (by constraint (6) the upper bound constraint on α i in (9) will always be satisfied for C 1). The solid circles indicate the support vectors, the dashed line is the boundary of the data description. The gray value indicates the distance to the center of the sphere; dark is close, light is remote. Only three objects are required to describe the complete data set. Schölkopf gives an alternative approach for solving the data description problem (Schölkopf et al., 1999b). A hyperplane is placed such that it separates the dataset from
6 50 D.M.J. TAX AND R.P.W. DUIN Figure 1. Example of a data description without outliers (left) and with one outlier (right). the origin with maximal margin. Although this is not a closed boundary around the data, it gives comparable solutions when the data is preprocessed to have unit norm (see also Schölkopf et al., 1999a). Forhyperplane w which separates the data x i from the origin with margin ρ, the following holds: w x i ρ ξ i i ξ i 0 (16) where ξ i accounts for possible errors. Schölkopf minimizes the structural error of the hyperplane, measured by w. This results in the following minimization problem: 1 min w,ρ,ξ 2 w 2 ρ + 1 ν N ξ i (17) i with constraints (16). The regularization parameter ν (0, 1) is a user defined parameter indicating the fraction of the data that should be separated and can be compared with the parameter C in the SVDD. Here we will call this method the ν-svc. An equivalent formulation of (16) and (17) is max ρ 1 w,ρ,ξ ν N ξ i, with w x i ρ ξ i, ξ i 0, w =1 (18) i where a constraint on w is introduced. When all the data in the original SVDD formulation is transformed to unit norm (see also (3) and (4)), we obtain the following optimization problem: min R 2 + C i ξ i with x i a 2 R 2 + ξ i i (19)
7 SUPPORT VECTOR DATA DESCRIPTION 51 where x and a are normalized vectors. Rewriting gives: max R 2 C i ξ i with 2(a x i ) 2 R 2 ξ i (20) We define w = 2a,ρ = 2 R 2, 1 ν N = C,ξ i = ξ i and the following optimization problem is obtained: max 2 + ρ 1 ξ i with w x i ρ ξ i, ξ i 0, w =2 (21) ν N i which is a comparable solution to Eq. (18). It differs in the constraint on the norm of w and in an offset of 2 in the error function. For non-normalized data the solutions become incomparable due to the difference in model of the description (hyperplane or hypersphere). We will come back to the ν-svc in Section 2.3 in the discussion on kernels SVDD with negative examples When negative examples (objects which should be rejected) are available, they can be incorporated in the training to improve the description. In contrast with the training (target) examples which should be within the sphere, the negative examples should be outside it. This data description now differs from the normal Support Vector Classifier in the fact that the SVDD always obtains a closed boundary around one of the classes (the target class). The support vector classifier just distinguishes between two (or more) classes and cannot detect outliers which do not belong to any of the classes. In the following the target objects are enumerated by indices i, j and the negative examples by l, m. For further convenience assume that target objects are labeled y i = 1 and outlier objects are labeled y l = 1. Again we allow for errors in both the target and the outlier set and introduce slack variables ξ i and ξ l : F(R, a,ξ i,ξ l ) = R 2 + C 1 ξ i + C 2 ξ l (22) and the constraints i l x i a 2 R 2 + ξ i, x l a 2 R 2 ξ l, ξ i 0, ξ l 0 i, l (23) These constraints are again incorporated in Eq. (22) and the Lagrange multipliers α i,α l, γ i,γ l are introduced: L(R, a,ξ i,ξ l,α i,α l,γ i,γ l ) = R 2 + C 1 ξ i + C 2 ξ l γ i ξ i γ l ξ l i l i l i α i [R 2 + ξ i (x i a) 2 ] l α l [(x l a) 2 R 2 + ξ l ] (24) with α i 0,α l 0,γ i 0,γ l 0.
8 52 D.M.J. TAX AND R.P.W. DUIN Setting the partial derivatives of L with respect to R, a, ξ i and ξ l to zero gives the constraints: α i α l = 1 (25) i l a = α i x i α l x l (26) i l 0 α i C 1, 0 α l C 2 i, l (27) When Eqs. (25) (27) are resubstituted into Eq. (24) we obtain L = α i (x i x i ) l α l (x l x l ) i, j α i α j (x i x j ) (28) + 2 l, j α l α j (x l x j ) l,m α l α m (x l x m ) (29) When we finally define new variables α i = y i α i (index i now enumerates both target and outlier objects), the SVDD with negative examples is identical to the normal SVDD. The constraints given in Eqs. (25) and (26) change into i α i = 1 and a = i α i x i and again the testing function Eq. (14) can be used. Therefore, when outlier examples are available, we will replace Eq. (10) by (29) and we will use α i instead of α i. In the right plot in figure 1 the same data set is shown as in the left plot, extended with one outlier object (indicated by the arrow). The outlier lies within the original description on the left. A new description has to be computed to reject this outlier. With a minimal adjustment to the old description, the outlier is placed on the boundary of the description. It becomes a support vector for the outlier class and cannot be distinguished from the support vectors from the target class on the basis of Eq. (14). Although the description is adjusted to reject the outlier object, it does not fit tightly around the rest of the target set anymore. A more flexible description is required Flexible descriptions When instead of the rigid hypersphere a more flexible data description is required, another choice for the inner product (x i x j ) can be considered. Replacing the new inner product by akernel function K (x i, x j ) = ( (x i ) (x j )) an implicit mapping of the data into another (possibly high dimensional) feature space is defined. An ideal kernel function would map the target data onto a bounded, spherically shaped area in the feature space and outlier objects outside this area. Then the hypersphere model would fit the data again (this is comparable with replacing the inner product in the Support Vector classifier when the classes are not linearly separable). Several kernel functions have been proposed for the Support Vector Classifier (Vapnik, 1998; Smola, Schölkopf, & Müller, 1998). It appears that not all these kernel functions map the target set in a bounded region in feature space. To show this, we first investigate the polynomial kernel.
9 SUPPORT VECTOR DATA DESCRIPTION 53 Figure 2. Data description trained on a banana shaped data set. Kernel is a polynomial kernel with varying degrees. Support vectors are indicated by the solid circles, the dashed line is the description boundary. The polynomial kernel is given by K (x i, x j ) = (x i x j ) d (30) where the free parameter d N + gives the degree of the polynomial kernel. The testing function of the SVDD (Eq. (14)) shows that only the second term accounts for the interaction between the test object z and the support objects x i. Recall that x i x j = cos(θ ij ) x i x j where θ ij is the angle between object vector x i and x j. When data is not centered around the origin, object vectors can become large. And when the θ ij is small, cos(θ ij ) 1 will stay almost constant. Then for larger degrees d, the polynomial kernel is approximated by: (x i x j ) d = cos d (θ ij ) x i d x j d x i d x j d (31) Equation (31) looses the sensitivity to θ ij in the neighborhood of the training data (where θ becomes small). The objects with the largest norm in the training set will overwhelm all other terms in the polynomial kernel. This effect can be suppressed by centering the data around the origin and rescaling the data to unit standard variance. Unfortunately rescaling to unit variance might only magnify the noise in directions with small variance, and the influence of the norms is not avoided. Finally, centering the data in the feature space by subtracting the averaged x (as explained in Schölkopf, 1997) does not resolve the problem of the large differences in vector norms. It can be shown that the centered SVDD is equivalent to the original SVDD. The influence of large norm objects is visible in figure 2. For a simple 2 dimensional data set, descriptions are obtained using a polynomial kernel with different degrees, ranging from d = 1.0 (left) to d = 6.0 (right). Again the solid circles indicate the support vectors, the dashed line is the description boundary mapped in the input space. The rigid spherical description is obtained for d = 1.0. For degree d = 6.0 the description is a sixth order polynomial. Here the training objects most remote from the origin (the objects on the right) become support objects and the data description only distinguishes on the basis of the norm of the vectors. Large regions in the input space without target objects will be accepted by the description.
10 54 D.M.J. TAX AND R.P.W. DUIN Next we investigate the Gaussian kernel with K (x i, x j ) = exp( x i x j 2 /s 2 ) (32) This kernel is independent of the position of the data set with respect to the origin, it only utilizes the distances between objects. In the testing function (Eq. (14)) the first term equals 1.0 and the testing function boils down to a weighted sum of Gaussians. Test object z is accepted when: ( z xi 2 ) α i exp R 2 /2 + C R (33) i s 2 where C R depends only on the Support Vectors x i and not on z. Using the Gaussian kernel the influence of the norms of the objects is avoided. The objects are mapped to unit norm vectors (the norm of the mapped objects (x i ) (x i ) = 1) and only the angles between object vectors count. When Gaussian kernels are used, both the SVDD and the ν-svc give comparable solutions. For small values of s, exp( x i x j 2 ) 0, i j and Eq. (10) is optimized when all s 2 objects become support objects with equal α i = 1. This is identical to the Parzen density N estimation with a small kernel width. For very large s the solution approximates the original spherically shaped solution. This can be shown by a Taylor expansion of the Gaussian kernel: K (x i, x j ) = 1 x i 2 /s 2 x j 2 /s 2 + 2(x i x j )/s 2 + (34) Substituting Eq. (34) into Eq. (10) we obtain: L = i α i (1 x i 2 /s 2 x i 2 /s 2 + 2(x i x i )/s 2 + ) i, j α i α j (1 x i 2 /s 2 x j 2 /s 2 + 2(x i x j )/s 2 + ) = α i x i 2 /s 2 2 i, j α i α j (x i x j )/s 2 + (35) Ignoring higher orders this is equivalent to Eq. (10) (with an extra scaling factor 2/s 2 ). When a new regularization parameter C = (2C)/s 2 is used, an identical solution is found. For intermediate values of s a weighted Parzen density estimation is obtained. Both the weights of the kernels and the choice of which training objects become support vectors are obtained automatically by the optimization procedure. These situations, ranging from a Parzen density estimation to the rigid hyper sphere, can be observed in figure 3, in combination with varying values of C. Inthe left column of pictures a small s = 1.0 width parameter is used, and in the right column a large s = 15 is used. In all cases, except for the limiting case where s becomes huge, the description is tighter than the normal spherically shaped description or the description with the polynomial kernel. Note that with increasing s the number of support vectors decreases. Secondly,
11 SUPPORT VECTOR DATA DESCRIPTION 55 Figure 3. Data description trained on a banana shaped data set. A Gaussian kernel with different widths (s = 1, 5, 15) and different values for C (C = 40, almost the hard margin case, and C = 0.1) are used. Support vectors are indicated by the solid circles, the solid white line is the description boundary. decreasing the parameter C constraints the values for α i more, and more objects become support vector. The error on the target class increases, but the covered volume of the data description decreases with decreasing C Target error estimate When an object drawn from the target distribution is rejected by the description, it is called an error. By applying Leave-One-Out estimation (Vapnik, 1998; Bishop, 1995), it can be shown that the number of support vectors is an indication of the expected error made on the target set. For that the notion of essential support vectors has to be introduced (Vapnik, 1998). The expansion of the center of the description a = i α ix i is not unique. It might be that more objects are on the boundary of the sphere than is necessary for the description (for instance when four objects are on a circle in a 2 dimensional feature space where 3 are sufficient 1 ). The essential support vectors are these objects which appear in all possible expansions. When one of the internal points (α i = 0) is left out of the training and the data description is computed, the same solution is obtained as with the training set including this training object. During testing this object will therefore be accepted by the description. When a nonessential support vector is left out during training, the solution including this object is still
12 56 D.M.J. TAX AND R.P.W. DUIN obtained, using the remaining support vectors in the expansion. When an essential support object on the boundary (a support vector with α i < C) isleft out, a smaller description is found. This support point will be rejected by the new solution. The errors, the support objects with α i = C are already outside the sphere. When they are left out of the training set, they will again be rejected. The fraction of objects which become (essential) support objects and outlier, is thus an leave-one-out error estimate on the target set: Ẽ LOO # SVs + # errors N (36) (where the inequality holds when not all support vectors are essential support vectors). When the Support Vector Data Description is trained on a set with both target and outlier objects present, only the fraction of support vectors on the target set should be considered for the estimation of the leave-one out error. The error of the outlier class can also be estimated, but only when a representative set of outliers is drawn from the true outlier distribution. Because usually in the one-class classification problem the outlier class is sampled very sparsely, a good representative data set is often not available and a reliable estimate of the error is not possible. This target error estimate opens the possibility to optimize the regularization parameter C and the width s of the Gaussian kernel. The value of the regularization parameter C can be chosen by using the fact that for errors α i = C. Recall the constraint that i α i = 1 (or i α i when negative examples are available) and therefore # errors 1. When in a N NC training set no errors are expected C can be set to 1.0, indicating that all target data should be accepted and all negative examples should be rejected. When no negative examples 1 are available and some outliers in the training set are expected, set C N (fraction outlier) beforehand. In Schölkopf et al. (1999a) the parameter (fraction outlier) is called ν and the fact that this ν sets the error (here on the target class) is called the ν-property. Secondly the kernel width s can be optimized on the basis of the required target acceptance rate. Comparing two width values s 1 > s 2,weget exp( x i x j 2 /s1 2) > exp( x i x j 2 /s2 2). For large enough s 1 Eq. (10) is maximized when the α i,α j corresponding to the larger terms K (x i, x j ) become zero. This means that the most separate objects become support vector and that the number of support vectors tends to decrease with increasing s. Using Eq. (36) this also means a decrease of the expected error on the target set (this should be no surprise, because the covered area in feature space tends to increase when the data model changes from a Parzen density estimate with small σ to the rigid hypersphere, see again figure 3). Unfortunately the minimization of Eq. (10) with Gaussian kernel requires s beforehand, so the optimization requires an iterative scheme to minimize the difference between the fraction SV s and the user defined error fraction. When a small fraction of the target data is to be rejected, the boundary is located in the tails of the target distribution. In order to have reliable estimates in these boundary in the tails, the required sample sizes become large. Consider a dataset with 10 objects in a 2 dimensional data set. In a typical training of an SVDD, it appears that the minimum number of support vectors is 2 to 3. To obtain a target error lower than 20% on this data, more training data is required. The accuracy of the target error estimate, the required number of training objects and some other characteristics will be investigated in the next section.
13 SUPPORT VECTOR DATA DESCRIPTION SVDD characteristics In this section several characteristics of the SVDD will be investigated. In all experiments the Gaussian kernel is used and an upper bound on the error on the target set is set a priori and the width parameter s is optimized Number of support vectors The question of how much data is required for finding a sufficiently accurate description (in terms of classification error) of a target class in some feature space cannot be answered a priori. Not only does it depend on the shape of the target data set, it also depends on the distribution of the (unknown) outlier set. However, a lower limit can be obtained, when a rigid spherically shaped data description is considered. Here just the coordinates of the center of the sphere and the radius of the sphere are required. Therefore in theory only two objects are sufficient to determine the sphere (independent of dimensionality). The weights α i are positive and constrained to sum up to 1 (Eq. (6)), and therefore the sphere center has to be within the convex hull of the support vectors. The center can therefore only be described by two objects when it is located on the line connecting these objects. For d-dimensional data which has variation in all directions, the required number of objects can increase up to d + 1. For data sets in a subspace, the number becomes less (and down to two if the data is on a one dimensional line). In figure 4 the fraction of outliers which is accepted, the fraction of training objects which become support vectors and the error on the target set is shown for data with different dimensionalities, 2D and 5D. The data is drawn from the (artificially generated) banana shaped distribution. For varying s an SVDD is trained. The error on the target set is then estimated by drawing a new independent test set containing 200 objects outliers are drawn from a square block around the data. Figure 4. The fraction of (independent test) outliers which is accepted, the fraction of the (training) target data which becomes support vector and the fraction of (independent test) target data rejected vs. the width parameter s for (left) 2 dimensional and (right) 5 dimensional data. The training set size is 50.
14 58 D.M.J. TAX AND R.P.W. DUIN By increasing s the number of support vectors decrease, closely followed by the error on the target set. For large s the error is bounded by the minimum number of support vectors which is needed for the description, in this case about 3 support vectors for 2D data and about 5 for 5D data. For a training set size of 50 this results in a minimum error of about 6% or 10%. The maximal fraction of outliers accepted is the quotient between the volume of the hypersphere and the volume of the outlier data block. For s > 20 in the 2D data and s > 40 in the 6D data both fractions stay constant, indicating that the maximum scale in the data is reached. Note that for increasing dimensionality of the feature space the volume of the outlier block tends to grow faster than the volume of the target class. This means that the overlap between the target and outlier data decreases and the classification problem becomes easier. In practice, this phenomenon is less apparent, although still present, because outliers are often not completely randomly scattered in all feature directions. Increasing the dimensionality to improve classification performance does have it s limits, because with increasing the dimensionality also the target class boundary increases, such that for reliable estimation of the boundary, more data is required. In Table 1 the minimum number of support vectors is shown for different datasets and varying dimensionalities (from 2D to the full dimensionality, using all available features). The first three datasets, gauss, banana and ellipse, are artificially generated. The Gauss and Table 1. Minimum number of support vectors, for different datasets (with dimensionalities from 2D to the full dimensionality) and different target classes. Dataset Full 2D 5D 10D 25D Gauss 2.94 (0.64) 5.63 (0.97) 7.81 (1.69) (2.06) Banana 3.23 (0.71) 5.50 (1.48) 7.76 (2.61) 8.71 (5.36) Ellipse 2.08 (0.04) 2.16 (0.08) 2.17 (0.07) 2.34 (0.19) Iris 2.00 (0.00) 2.00 (0.00) (4D) 2.20 (0.63) 2.10 (0.32) 2.30 (0.67) 2.10 (0.32) Sonar 6.20 (1.23) 2.40 (0.97) 2.70 (1.34) 5.60 (0.97) 6.00 (1.33) (60D) 7.30 (1.70) 3.10 (0.32) 4.40 (0.97) 4.90 (1.37) 7.20 (1.40) Dataset Full 2D 5D 10D 25D Glass 3.90 (0.32) 3.00 (0.67) 3.90 (0.32) (9D) 3.00 (0.47) 2.10 (0.32) 2.30 (0.67) 2.30 (0.48) 2.20 (0.42) 2.20 (0.42) 4.20 (0.42) 3.70 (0.48) 4.30 (0.48) Imox 4.00 (0.47) 3.80 (0.42) 4.00 (0.47) (8D) 2.60 (1.07) 2.30 (0.67) 2.60 (1.07) 3.70 (0.67) 2.20 (0.42) 3.10 (0.57) 5.00 (0.82) 3.30 (0.67) 5.00 (0.82) Values are averaged of 10 runs, the standard deviation is given between brackets.
15 SUPPORT VECTOR DATA DESCRIPTION 59 banana set have variance in all feature directions, the ellipse mainly in the first direction. The other datasets are normal classification problems, taken from the UCI repository (Blake, Keogh, & Merz, 1998). A data description is trained on each of the classes separately (thus the three class Iris dataset shows three lines of results). In the first column the number of SV s in the original feature space is shown (except for the artificial datasets). In the next columns it is shown for datasets which are reduced to the first few principal components. For the Gauss and banana datasets the number of SV s increases with increasing dimensionality, but stays in the order of d/2. In the ellipse dataset the single large variance direction causes this number to stay very low (two support vectors on the extremities of the dataset often support the whole data description). For the other data sets the number of support vectors is mainly determined by the dimensionality of the subspace in which the data is distributed. For the sonar database the number of SV s hardly changes after 25D, for the other datasets this already happens with 5D data. Finally note that the number of SV s for different classes can differ, e.g. in imox this number ranges from 2 to almost 5. When this limited amount of data is available, this minimum number of support vectors immediately gives an indication of the target error which can minimally be achieved (by Eq. (36)). Only using more training data can reduce the error on the target set further Training with outliers To compare data descriptions trained with outlier data with standard two-class classifiers, results on the classification problems mentioned in the previous section are shown in Table 2. One class is the target class, and all other data is outlier data. Tenfold cross validation is used to find the classification error. The reported results are the classification errors on an independent (labeled) test set. The classifiers are Gaussian-density based linear classifier (called Bayes), Parzen classifier and the Support Vector Classifier with polynomial kernel, degree 3. These are compared with the data descriptions, without and including example outliers (SVDD and SVDD-neg respectively). In both these cases the Gaussian kernel was used. The parameter σ was optimized such that approximately 10% of the data becomes support vector. When example outliers are used, the parameters C 1 and C 2 are set such, that the fraction of rejected objects on the target set should be less then 1%, while the fraction of accepted outlier objects can be up to 50%. To investigate the influence of the norms of the vectors in real world applications, not only the Gaussian kernel but also the polynomial kernel (with degree 3, where the data is rescaled to unit variance) is used. These results are given in the columns SVDD,p3 and SVDD-neg, p3. In most cases the classifiers, especially the Parzen classifier and the SVC with polynomial degree 3, outperform the data descriptions. This is not surprising, because they are constructed to give the best separation between the two classes, with equal focus on each of the classes. They are not limited by the extra constraint to have a closed boundary around on the target class. The performance of the one-class classifiers which only use information from the target set, perform worse, but in some cases still comparable to the classifiers which use information of both the target and outlier data. The data descriptions using outliers perform
16 60 D.M.J. TAX AND R.P.W. DUIN Table 2. the rest. Cross-validation error (in %) of different classifiers for distinguishing between one of the classes and cl SVC SVDD SVDD SVDD nr Bayes Parzen p3 SVDD p3 neg neg, p3 Iris (0.0) 0.0 (0.0) 0.0 (0.0) 4.7 (6.3) 33.3 (0.0) 8.0 (6.1) 4.7 (4.5) (9.4) 3.3 (3.5) 8.0 (6.1) 8.0 (8.8) 23.3 (8.5) 9.3 (4.7) 10.0 (3.5) (4.9) 3.3 (3.5) 6.0 (3.8) 7.3 (3.8) 38.7 (8.2) 9.3 (5.6) 20.0 (4.4) Glass (8.6) 19.2 (7.8) 22.4 (8.7) 33.6 (7.8) 36.5 (9.3) 29.0 (5.7) 32.7 (8.6) (10.2) 19.2 (6.9) 20.5 (11.2) 51.5 (5.1) 50.6 (6.1) 30.4 (7.9) 51.9 (7.0) (3.0) 8.4 (3.7) 10.3 (4.4) 30.0 (10.9) 54.7 (6.9) 9.3 (4.3) 46.8 (5.3) (4.7) 5.2 (3.5) 7.5 (6.2) 69.7 (4.6) 76.2 (4.1) 14.9 (3.6) 72.5 (6.0) Sonar (11.9) 14.5 (6.4) 11.0 (7.5) 50.6 (8.5) 44.3 (8.7) 35.2 (10.6) 45.2 (8.1) (11.9) 14.5 (6.4) 11.0 (7.5) 41.3 (6.3) 53.8 (7.0) 30.3 (11.6) 45.2 (9.2) Imox (4.8) 4.1 (4.7) 4.6 (4.5) 17.6 (8.4) 74.5 (10.6) 13.4 (8.6) 48.8 (8.6) (5.7) 0.5 (1.7) 1.6 (2.5) 6.8 (7.5) 66.7 (7.9) 8.3 (5.1) 22.4 (6.1) (4.8) 0.5 (1.7) 6.2 (6.8) 4.2 (4.8) 17.7 (8.5) 6.7 (5.9) 4.2 (3.3) (7.1) 4.1 (4.7) 10.4 (5.0) 18.8 (9.0) 49.1 (15.2) 13.1 (10.9) 40.2 (12.8) somewhat better than without example outliers, as should be expected, but in some cases it requires careful optimization of the C 1, C 2 parameters (for instance for the imox dataset which has much overlap between the classes). In most cases the data descriptions with the polynomial kernel perform worse than with the Gaussian kernel, except for a few cases. Note that when the polynomial kernel is used, the data is preprocessed to have zero mean and unit variance along each of the feature directions. When this preprocessing is not applied, the classification performance becomes extremely poor (worse than random guessing). 4. Experiments Finally in this section we investigate how the SVDD works in a real one-class classification problem. We focus on a machine diagnostics problem: the characterization of a submersible water pump (Tax & Duin, 1999). Here the normal operation of the pump has to be characterized, to distinguish it from all other faulty operation conditions. Both target objects (measurements on a normal operating pump) and negative examples (measurements on a damaged pump) are available. In a test bed several normal and outlier situations can be simulated. The normal situations consist of working conditions with different loads and speeds of the pump. The outlier data contain pumping situations with loose foundation, imbalance and a failure in the outer race of the uppermost ball bearing.
17 SUPPORT VECTOR DATA DESCRIPTION 61 Table 3. The training and testing sample sizes for the different vibration datasets with their settings. A speed setting of 46, 50, 54 Hz and a load setting of 29, 33 (0.1kW) means that all measurements with these settings are used (i.e. 6 situations in total). Settings Dataset number Speeds (Hz) Loads (0.1 kw) Testing set Training set target obj. Target obj. Outlier obj. 1 46, 48, 50, 52, 54 25, 29, , 50, 54 29, , 54 25, , 29, , 48, 50, 52, To characterize the normal operation condition of the pump, the operator varies the speed (with possible values of 46, 48, 50, 52, 54 Hz) and the load (25, 29, kw) of the pump. For each setting, the pump is run for some time. On the pump vibration sensors are mounted. From the recorded time series subsamples are taken and the power spectrum is calculated (containing 64 bins, resulting in a 64 dimensional dataset). By this procedure the normal operation conditions can be simulated and recorded. Thus the target area in the feature space can be indicated. It is not clear, though, what the typical working conditions will be in practice. The pump should be operating somewhere in the target area, but it will probably not uniformly over the whole area. These characteristics are simulated using this dataset by sampling the training data in different ways. In this way the personal choices of the operator vary, although they should in all cases define the normal operation area in the feature space. In Table 3 five different datasets are listed with their speed and load settings, and the number of training and testing objects available. The first dataset is considered to be the most complete, that it covers the whole working area of the pump. The next datasets are approximations to this dataset. To see how well the SVDD and the SVDD with negative examples (SVDD neg) perform, we compare them with a number of other methods. The first method is a normal density where just the mean and the covariance matrix of the data has to be estimated. This will result in a ellipsoidal boundary around the data. In high dimensional feature spaces the covariance matrix of the target set can become singular. The covariance matrix is regularized by adding a small diagonal term to the covariance matrix: = + λi. The regularization parameter is optimized for the problem at hand and is in most cases between 10 3 and The second method is the Parzen density where the width of the Parzen kernel is estimated using leaveone-out optimization (Duin, 1976). The third method is a Mixture of Gaussians, optimized using EM (Bishop, 1995). The number of clusters is set to 5 beforehand, but varying this value does not improve performance very much. The last method is a nearest neighbor method, which compares the local density of an object x with the density of the nearest neighbor in the target set (Tax & Duin, 2000). The distance between the test object x and its nearest neighbor in the training set NN tr (x) is compared with the distance between this nearest neighbor NN tr (x) and its nearest neighbor in the training set NN tr (NN tr (x)). When the first distance is much larger than the second distance, the object will be regarded as an outlier. We use the quotient between the first and
18 62 D.M.J. TAX AND R.P.W. DUIN Figure 5. The ROC curves for dataset 1 (see Table 3). Left for the data in the original 64D feature space, right the data in 15D (retaining 80% of the variance). the second distance as indication of the validity of the object: ρ(x) = x NN tr (x) NN tr (x) NN tr (NN tr (x)) (37) where NN tr x is the nearest neighbor of x in the training set. It is to be expected that when just the target class area is covered, and not sampled according to what will happen in practice, the density methods will fail and the methods which just focus on modeling the data will still work. In the case of a good distribution representation in the training set, the density and the boundary methods should perform about equal. For the comparison, the trade-off between the fraction of the target class rejected (error of the first kind, E I )versus the fraction of the outlier accepted (error of the second kind, E II )isinvestigated. The methods will be compared over a range of threshold values, from 1% target class rejection up to 50% rejection. The fraction of outliers rejected and target objects accepted is measured afterwards on the test set. In figure 5 the Receiver-Operating Characteristic curves are shown (ROC-curve) (Metz, 1978) for the methods trained on the power spectrum features. In the left plot the results on the full 64 dimensional feature space is shown. A good discrimination between target and outlier objects means both a small fraction of outlier accepted and a large fraction of target objects accepted. The optimal performing classifier will be in the upper left corner of the ROC curve. The ROC curves in the left plot show, that for this data overlap between the classes occur, and this optimal performance cannot be obtained. The sample size is too low to obtain a good density estimate and the Parzen density estimator is not able to estimate the density well. It rejects all target and all outlier data. The Gaussian density and the SVDD obtain about the same solution, indicating that in this representation with these amounts of data, no more than an ellipse can be fitted. The Mixture of Gaussians and the knn overfit and show poor performance. In the right subplot the data dimensionality is reduced by PCA to retain 80% of the variance. In this case, the sample size is sufficient to follow the boundaries of the target set more closely. The poor performance of the Gaussian shows, that the target distribution
19 SUPPORT VECTOR DATA DESCRIPTION 63 Table 4. Integrated errors for the five outlier detection methods on the original power spectrum data and versions with only the first few principal components. Number of features Method Normal dens Mix.o.Gauss Parzen dens knn SVDD Best performances from the methods are shown in bold. boundary is more complex than a ellipsoid. The Parzen density estimator has still some problems with the sample size, but both the SVDD and the knn give nice results. To make a more quantitative comparison, an error measure is derived from the ROCcurves. The fraction of the outliers which is accepted (E II ), is averaged over varying E I (Bradley, 1997): E M = B A E II (E I ) de I (38) In Table 4 the results on the power spectrum data are shown. The methods are applied to both the original 64 dimensional data set and to the first principal components of that data, ranging from 3 up to 30 features. For higher dimensionalities the normal densities and the SVDD perform well. This indicates that for higher dimensions, approximating the target class by one ellipse is sufficient. For the Normal density, the Mixture of Gaussians and the SVDD the error decreases with increasing dimensionality while for the Parzen estimator it decreases. On low dimensionalities the density can be estimated relatively well by the Parzen estimator, but it suffers from the fact that it is not capable of accepting all target objects in the test set. This means that in order to compute the AUC error from the ROC curve, the ROC-curve should be extended. Here it is chosen to extend it in the worst case sense (when a certain fraction target acceptance cannot be reached, it is assumed that the fraction of outliers accepted will be 1). Therefore the errors for the Parzen density estimator are high. The use of the negative examples in the SVDD is problematic in this example, because there are many outliers and some show complete overlap with the target class. Choosing a set of good outliers improves performance significantly, but when some poor outliers are chosen (outliers which are actually inside the target class) performance may deteriorate to random guessing. In these cases a manual optimization of C 1 and C 2 is required, by judging what fraction of the outliers are genuine outliers and which are actually not. When randomly outliers are drawn, the variance of the different outcomes is too large to be useful. In figures 6 and 7 the ROC curves for the one-class classifiers are given for the other datasets listed in Table 3. By the (sub)sampling of the training set, the distribution of the training target data does not completely reflect the target distribution in the test data, although it should define the same area in feature space. In almost all cases the SVDD,
20 64 D.M.J. TAX AND R.P.W. DUIN Figure 6. The ROC curves for dataset 2 and 3 (see Table 3). The data dimensionality was reduced by PCA to retain 90% of the variance. Figure 7. The ROC curves for datasets 4 (on the left) and 5 on the right). (see Table 3). The data dimensionality was reduced by PCA to retain 90% of the variance. which focuses on modeling the boundary, obtains a better performance than the density estimates (this is very clear in datasets 3 and 5). The performance of the density estimators can be improved with respect to the SVDD by reducing the dimensionality, but in these cases the absolute performance is so poor, that it is not useful to apply in practice. The worst classification performances are obtained for dataset 4, in which measurements on just one speed (50 Hz.) was included. Using just one speed setting is insufficient to define the whole feature area, in contrast to dataset 5, where a single load is used, but where the performance of the SVDD is still very good. These examples show that, when an area in feature space should be characterized (and not per se the complete target probability density), the SVDD gives very good data descriptions. The density estimators on the other hand, focus much more on the high density areas, which are not representative for the practical situation. 5. Conclusions In this paper we discussed a new method to solve the multidimensional outlier detection problem. Instead of estimating a probability density, it obtains a boundary around the data
Support Vector Machine (SVM)
Support Vector Machine (SVM) CE-725: Statistical Pattern Recognition Sharif University of Technology Spring 2013 Soleymani Outline Margin concept Hard-Margin SVM Soft-Margin SVM Dual Problems of Hard-MarginMore information
Support Vector Machines with Clustering for Training with Very Large Datasets
Support Vector Machines with Clustering for Training with Very Large Datasets Theodoros Evgeniou Technology Management INSEAD Bd de Constance, Fontainebleau 77300, France email@example.com MassimilianoMore information
Introduction to Support Vector Machines. Colin Campbell, Bristol University
Introduction to Support Vector Machines Colin Campbell, Bristol University 1 Outline of talk. Part 1. An Introduction to SVMs 1.1. SVMs for binary classification. 1.2. Soft margins and multi-class classification.More information
A Linear Programming Approach to Novelty Detection
A Linear Programming Approach to Novelty Detection Colin Campbell Dept. of Engineering Mathematics, Bristol University, Bristol Bristol, BS8 1 TR, United Kingdon C. email@example.com Kristin P. BennettMore information
Component Ordering in Independent Component Analysis Based on Data Power
Component Ordering in Independent Component Analysis Based on Data Power Anne Hendrikse Raymond Veldhuis University of Twente University of Twente Fac. EEMCS, Signals and Systems Group Fac. EEMCS, SignalsMore information
PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 4: LINEAR MODELS FOR CLASSIFICATION Introduction In the previous chapter, we explored a class of regression models having particularly simple analyticalMore information
Visualization of large data sets using MDS combined with LVQ.
Visualization of large data sets using MDS combined with LVQ. Antoine Naud and Włodzisław Duch Department of Informatics, Nicholas Copernicus University, Grudziądzka 5, 87-100 Toruń, Poland. www.phys.uni.torun.pl/kmkMore information
Knowledge Discovery from patents using KMX Text Analytics
Knowledge Discovery from patents using KMX Text Analytics Dr. Anton Heijs email@example.com Treparel Abstract In this white paper we discuss how the KMX technology of Treparel can help searchersMore information
A Simple Introduction to Support Vector Machines
A Simple Introduction to Support Vector Machines Martin Law Lecture for CSE 802 Department of Computer Science and Engineering Michigan State University Outline A brief history of SVM Large-margin linearMore information
Regression Using Support Vector Machines: Basic Foundations
Regression Using Support Vector Machines: Basic Foundations Technical Report December 2004 Aly Farag and Refaat M Mohamed Computer Vision and Image Processing Laboratory Electrical and Computer EngineeringMore information
An Introduction to Machine Learning
An Introduction to Machine Learning L5: Novelty Detection and Regression Alexander J. Smola Statistical Machine Learning Program Canberra, ACT 0200 Australia firstname.lastname@example.com Tata Institute, Pune,More information
Support Vector Clustering
Journal of Machine Learning Research 2 (2) 25-37 Submitted 3/4; Published 2/ Support Vector Clustering Asa Ben-Hur BIOwulf Technologies 23 Addison st. suite 2, Berkeley, CA 9474, USA David Horn SchoolMore information
Statistical Machine Learning
Statistical Machine Learning UoC Stats 37700, Winter quarter Lecture 4: classical linear and quadratic discriminants. 1 / 25 Linear separation For two classes in R d : simple idea: separate the classesMore information
Comparison of Non-linear Dimensionality Reduction Techniques for Classification with Gene Expression Microarray Data
CMPE 59H Comparison of Non-linear Dimensionality Reduction Techniques for Classification with Gene Expression Microarray Data Term Project Report Fatma Güney, Kübra Kalkan 1/15/2013 Keywords: Non-linearMore information
Modelling, Extraction and Description of Intrinsic Cues of High Resolution Satellite Images: Independent Component Analysis based approaches
Modelling, Extraction and Description of Intrinsic Cues of High Resolution Satellite Images: Independent Component Analysis based approaches PhD Thesis by Payam Birjandi Director: Prof. Mihai Datcu ProblematicMore information
STA 4273H: Statistical Machine Learning
STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Statistics! firstname.lastname@example.com! http://www.cs.toronto.edu/~rsalakhu/ Lecture 6 Three Approaches to Classification ConstructMore information
One-class classification
One-class classification Concept-learning in the absence of counter-examples Proefschrift ter verkrijging van de graad van doctor aan de Technische Universiteit Delft, op gezag van de Rector MagnificusMore information
High-dimensional labeled data analysis with Gabriel graphs
High-dimensional labeled data analysis with Gabriel graphs Michaël Aupetit CEA - DAM Département Analyse Surveillance Environnement BP 12-91680 - Bruyères-Le-Châtel, France Abstract. We propose the useMore information
Data Mining - Evaluation of Classifiers
Data Mining - Evaluation of Classifiers Lecturer: JERZY STEFANOWSKI Institute of Computing Sciences Poznan University of Technology Poznan, Poland Lecture 4 SE Master Course 2008/2009 revised for 2010More information
Dissimilarity representations allow for building good classifiers
Pattern Recognition Letters 23 (2002) 943 956 www.elsevier.com/locate/patrec Dissimilarity representations allow for building good classifiers El_zbieta Pezkalska *, Robert P.W. Duin Pattern RecognitionMore information
Example: Credit card default, we may be more interested in predicting the probabilty of a default than classifying individuals as default or not.
Statistical Learning: Chapter 4 Classification 4.1 Introduction Supervised learning with a categorical (Qualitative) response Notation: - Feature vector X, - qualitative response Y, taking values in CMore information
D-optimal plans in observational studies
D-optimal plans in observational studies Constanze Pumplün Stefan Rüping Katharina Morik Claus Weihs October 11, 2005 Abstract This paper investigates the use of Design of Experiments in observationalMore information
Machine Learning and Pattern Recognition Logistic Regression
Machine Learning and Pattern Recognition Logistic Regression Course Lecturer:Amos J Storkey Institute for Adaptive and Neural Computation School of Informatics University of Edinburgh Crichton Street,More information
Support Vector Machines Explained
March 1, 2009 Support Vector Machines Explained Tristan Fletcher www.cs.ucl.ac.uk/staff/t.fletcher/ Introduction This document has been written in an attempt to make the Support Vector Machines (SVM),More information
Linear Threshold Units
Linear Threshold Units w x hx (... w n x n w We assume that each feature x j and each weight w j is a real number (we will relax this later) We will study three different algorithms for learning linearMore information
Chapter 6. The stacking ensemble approach
82 This chapter proposes the stacking ensemble approach for combining different data mining classifiers to get better performance. Other combination techniques like voting, bagging etc are also describedMore information
Classification of Bad Accounts in Credit Card Industry
Classification of Bad Accounts in Credit Card Industry Chengwei Yuan December 12, 2014 Introduction Risk management is critical for a credit card company to survive in such competing industry. In additionMore information
Introduction to Machine Learning NPFL 054
Introduction to Machine Learning NPFL 054 http://ufal.mff.cuni.cz/course/npfl054 Barbora Hladká firstname.lastname@example.com Martin Holub email@example.com Charles University, Faculty of Mathematics andMore information
MACHINE LEARNING IN HIGH ENERGY PHYSICS LECTURE #1 Alex Rogozhnikov, 2015 INTRO NOTES 4 days two lectures, two practice seminars every day this is introductory track to machine learning kaggle competition!More information
Least Squares Estimation
Least Squares Estimation SARA A VAN DE GEER Volume 2, pp 1041 1045 in Encyclopedia of Statistics in Behavioral Science ISBN-13: 978-0-470-86080-9 ISBN-10: 0-470-86080-4 Editors Brian S Everitt & DavidMore information
Comparing the Results of Support Vector Machines with Traditional Data Mining Algorithms
Comparing the Results of Support Vector Machines with Traditional Data Mining Algorithms Scott Pion and Lutz Hamel Abstract This paper presents the results of a series of analyses performed on direct mailMore information
EM Clustering Approach for Multi-Dimensional Analysis of Big Data Set
EM Clustering Approach for Multi-Dimensional Analysis of Big Data Set Amhmed A. Bhih School of Electrical and Electronic Engineering Princy Johnson School of Electrical and Electronic Engineering MartinMore information
CS 2750 Machine Learning. Lecture 1. Machine Learning. http://www.cs.pitt.edu/~milos/courses/cs2750/ CS 2750 Machine Learning.
Lecture Machine Learning Milos Hauskrecht email@example.com 539 Sennott Square, x5 http://www.cs.pitt.edu/~milos/courses/cs75/ Administration Instructor: Milos Hauskrecht firstname.lastname@example.com 539 SennottMore information
Introduction to Machine Learning. Speaker: Harry Chao Advisor: J.J. Ding Date: 1/27/2011
Introduction to Machine Learning Speaker: Harry Chao Advisor: J.J. Ding Date: 1/27/2011 1 Outline 1. What is machine learning? 2. The basic of machine learning 3. Principles and effects of machine learningMore information
Classification algorithm in Data mining: An Overview
Classification algorithm in Data mining: An Overview S.Neelamegam #1, Dr.E.Ramaraj *2 #1 M.phil Scholar, Department of Computer Science and Engineering, Alagappa University, Karaikudi. *2 Professor, DepartmentMore information
Face Recognition using Principle Component Analysis
Face Recognition using Principle Component Analysis Kyungnam Kim Department of Computer Science University of Maryland, College Park MD 20742, USA Summary This is the summary of the basic idea about PCAMore information
Artificial Neural Networks and Support Vector Machines. CS 486/686: Introduction to Artificial Intelligence
Artificial Neural Networks and Support Vector Machines CS 486/686: Introduction to Artificial Intelligence 1 Outline What is a Neural Network? - Perceptron learners - Multi-layer networks What is a SupportMore information
REVSTAT Statistical Journal Volume 7, Number 1, April 2009, 23 36 THE SVM APPROACH FOR BOX JENKINS MODELS Authors: Saeid Amiri Dep. of Energy and Technology, Swedish Univ. of Agriculture Sciences, P.O.BoxMore information
Towards better accuracy for Spam predictions
Towards better accuracy for Spam predictions Chengyan Zhao Department of Computer Science University of Toronto Toronto, Ontario, Canada M5S 2E4 email@example.com Abstract Spam identification is crucialMore information
A fast multi-class SVM learning method for huge databases
www.ijcsi.org 544 A fast multi-class SVM learning method for huge databases Djeffal Abdelhamid 1, Babahenini Mohamed Chaouki 2 and Taleb-Ahmed Abdelmalik 3 1,2 Computer science department, LESIA Laboratory,More information
These slides follow closely the (English) course textbook Pattern Recognition and Machine Learning by Christopher Bishop
Music and Machine Learning (IFT6080 Winter 08) Prof. Douglas Eck, Université de Montréal These slides follow closely the (English) course textbook Pattern Recognition and Machine Learning by ChristopherMore information
Lecture 9: Continuous
CSC2515 Fall 2007 Introduction to Machine Learning Lecture 9: Continuous Latent Variable Models 1 Example: continuous underlying variables What are the intrinsic latent dimensions in these two datasets?More information
Using artificial intelligence for data reduction in mechanical engineering
Using artificial intelligence for data reduction in mechanical engineering L. Mdlazi 1, C.J. Stander 1, P.S. Heyns 1, T. Marwala 2 1 Dynamic Systems Group Department of Mechanical and Aeronautical Engineering,More information
Analysis of kiva.com Microlending Service! Hoda Eydgahi Julia Ma Andy Bardagjy December 9, 2010 MAS.622j
Analysis of kiva.com Microlending Service! Hoda Eydgahi Julia Ma Andy Bardagjy December 9, 2010 MAS.622j What is Kiva? An organization that allows people to lend small amounts of money via the InternetMore information
Cluster Analysis: Advanced Concepts
Cluster Analysis: Advanced Concepts and dalgorithms Dr. Hui Xiong Rutgers University Introduction to Data Mining 08/06/2006 1 Introduction to Data Mining 08/06/2006 1 Outline Prototype-based Fuzzy c-meansMore information
An Adaptive Index Structure for High-Dimensional Similarity Search
An Adaptive Index Structure for High-Dimensional Similarity Search Abstract A practical method for creating a high dimensional index structure that adapts to the data distribution and scales well withMore information
Predicting the Risk of Heart Attacks using Neural Network and Decision Tree
Predicting the Risk of Heart Attacks using Neural Network and Decision Tree S.Florence 1, N.G.Bhuvaneswari Amma 2, G.Annapoorani 3, K.Malathi 4 PG Scholar, Indian Institute of Information Technology, Srirangam,More information
Statistics in Retail Finance. Chapter 7: Fraud Detection in Retail Credit
Statistics in Retail Finance Chapter 7: Fraud Detection in Retail Credit 1 Overview > Detection of fraud remains an important issue in retail credit. Methods similar to scorecard development may be employed,More information
A Content based Spam Filtering Using Optical Back Propagation Technique
A Content based Spam Filtering Using Optical Back Propagation Technique Sarab M. Hameed 1, Noor Alhuda J. Mohammed 2 Department of Computer Science, College of Science, University of Baghdad - Iraq ABSTRACTMore information
Introduction to machine learning and pattern recognition Lecture 1 Coryn Bailer-Jones
Introduction to machine learning and pattern recognition Lecture 1 Coryn Bailer-Jones http://www.mpia.de/homes/calj/mlpr_mpia2008.html 1 1 What is machine learning? Data description and interpretationMore information
Social Media Mining. Data Mining Essentials
Introduction Data production rate has been increased dramatically (Big Data) and we are able store much more data than before E.g., purchase data, social media data, mobile phone data Businesses and customersMore information
A Learning Based Method for Super-Resolution of Low Resolution Images
A Learning Based Method for Super-Resolution of Low Resolution Images Emre Ugur June 1, 2004 firstname.lastname@example.com Abstract The main objective of this project is the study of a learning based methodMore information
Statistical Models in Data Mining
Statistical Models in Data Mining Sargur N. Srihari University at Buffalo The State University of New York Department of Computer Science and Engineering Department of Biostatistics 1 Srihari Flood ofMore information
Triangular Visualization
Triangular Visualization Tomasz Maszczyk and W lodzis law Duch Department of Informatics, Nicolaus Copernicus University, Toruń, Poland firstname.lastname@example.com;google:w.duch http://www.is.umk.pl Abstract.More information
Christfried Webers. Canberra February June 2015
c Statistical Group and College of Engineering and Computer Science Canberra February June (Many figures from C. M. Bishop, "Pattern Recognition and ") 1of 829 c Part VIII Linear Classification 2 LogisticMore information
An Overview of Knowledge Discovery Database and Data mining Techniques
An Overview of Knowledge Discovery Database and Data mining Techniques Priyadharsini.C 1, Dr. Antony Selvadoss Thanamani 2 M.Phil, Department of Computer Science, NGM College, Pollachi, Coimbatore, Tamilnadu,More information
Neural Networks Lesson 5 - Cluster Analysis
Neural Networks Lesson 5 - Cluster Analysis Prof. Michele Scarpiniti INFOCOM Dpt. - Sapienza University of Rome http://ispac.ing.uniroma1.it/scarpiniti/index.htm email@example.com Rome, 29More information
Robust Outlier Detection Technique in Data Mining: A Univariate Approach
Robust Outlier Detection Technique in Data Mining: A Univariate Approach Singh Vijendra and Pathak Shivani Faculty of Engineering and Technology Mody Institute of Technology and Science Lakshmangarh, Sikar,More information
Determining optimal window size for texture feature extraction methods
IX Spanish Symposium on Pattern Recognition and Image Analysis, Castellon, Spain, May 2001, vol.2, 237-242, ISBN: 84-8021-351-5. Determining optimal window size for texture feature extraction methods DomènecMore information
Nonlinear Iterative Partial Least Squares Method
Numerical Methods for Determining Principal Component Analysis Abstract Factors Béchu, S., Richard-Plouet, M., Fernandez, V., Walton, J., and Fairley, N. (2016) Developments in numerical treatments forMore information
A Comparative Study of the Pickup Method and its Variations Using a Simulated Hotel Reservation Data
A Comparative Study of the Pickup Method and its Variations Using a Simulated Hotel Reservation Data Athanasius Zakhary, Neamat El Gayar Faculty of Computers and Information Cairo University, Giza, EgyptMore information
Classifiers & Classification
Classifiers & Classification Forsyth & Ponce Computer Vision A Modern Approach chapter 22 Pattern Classification Duda, Hart and Stork School of Computer Science & Statistics Trinity College Dublin DublinMore information
Review of some concepts in predictive modeling
Review of some concepts in predictive modeling Brigham and Women s Hospital Harvard-MIT Division of Health Sciences and Technology HST.951J: Medical Decision Support A disjoint list of topics? Naïve BayesMore information
Section 1.1. Introduction to R n
The Calculus of Functions of Several Variables Section. Introduction to R n Calculus is the study of functional relationships and how related quantities change with each other. In your first exposure toMore information
Logistic Regression. Jia Li. Department of Statistics The Pennsylvania State University. Logistic Regression
Logistic Regression Department of Statistics The Pennsylvania State University Email: firstname.lastname@example.com Logistic Regression Preserve linear classification boundaries. By the Bayes rule: Ĝ(x) = arg maxMore information
The Scientific Data Mining Process
Chapter 4 The Scientific Data Mining Process When I use a word, Humpty Dumpty said, in rather a scornful tone, it means just what I choose it to mean neither more nor less. Lewis Carroll [87, p. 214] InMore information
Linear Classification. Volker Tresp Summer 2015
Linear Classification Volker Tresp Summer 2015 1 Classification Classification is the central task of pattern recognition Sensors supply information about an object: to which class do the object belongMore information
15.062 Data Mining: Algorithms and Applications Matrix Math Review
.6 Data Mining: Algorithms and Applications Matrix Math Review The purpose of this document is to give a brief review of selected linear algebra concepts that will be useful for the course and to developMore information
Feature Extraction by Neural Network Nonlinear Mapping for Pattern Classification
Lerner et al.:feature Extraction by NN Nonlinear Mapping 1 Feature Extraction by Neural Network Nonlinear Mapping for Pattern Classification B. Lerner, H. Guterman, M. Aladjem, and I. Dinstein DepartmentMore information
Classification Techniques for Remote Sensing
Classification Techniques for Remote Sensing Selim Aksoy Department of Computer Engineering Bilkent University Bilkent, 06800, Ankara email@example.com http://www.cs.bilkent.edu.tr/ saksoy/courses/cs551More information
11 Linear and Quadratic Discriminant Analysis, Logistic Regression, and Partial Least Squares Regression
Frank C Porter and Ilya Narsky: Statistical Analysis Techniques in Particle Physics Chap. c11 2013/9/9 page 221 le-tex 221 11 Linear and Quadratic Discriminant Analysis, Logistic Regression, and PartialMore information
Data Mining Chapter 6: Models and Patterns Fall 2011 Ming Li Department of Computer Science and Technology Nanjing University
Data Mining Chapter 6: Models and Patterns Fall 2011 Ming Li Department of Computer Science and Technology Nanjing University Models vs. Patterns Models A model is a high level, global description of aMore information
E-commerce Transaction Anomaly Classification
E-commerce Transaction Anomaly Classification Minyong Lee firstname.lastname@example.com Seunghee Ham firstname.lastname@example.com Qiyi Jiang email@example.com I. INTRODUCTION Due to the increasing popularity of e-commerceMore information
Concepts in Machine Learning, Unsupervised Learning & Astronomy Applications
Data Mining In Modern Astronomy Sky Surveys: Concepts in Machine Learning, Unsupervised Learning & Astronomy Applications Ching-Wa Yip email@example.com; Bloomberg 518 Human are Great Pattern RecognizersMore information
AAS 07-228 SPECIAL PERTURBATIONS UNCORRELATED TRACK PROCESSING INTRODUCTION James G. Miller * Two historical uncorrelated track (UCT) processing approaches have been employed using general perturbationsMore information
Support Vector Machines
Support Vector Machines Here we approach the two-class classification problem in a direct way: We try and find a plane that separates the classes in feature space. If we cannot, we get creative in twoMore information
Classification: Basic Concepts, Decision Trees, and Model Evaluation. General Approach for Building Classification Model
10 10 Classification: Basic Concepts, Decision Trees, and Model Evaluation Dr. Hui Xiong Rutgers University Introduction to Data Mining 1//009 1 General Approach for Building Classification Model Tid Attrib1More information
Breaking SVM Complexity with Cross-Training
Breaking SVM Complexity with Cross-Training Gökhan H. Bakır Max Planck Institute for Biological Cybernetics, Tübingen, Germany email@example.com Léon Bottou NEC Labs America Princeton NJ, USA email@example.com information
Lecture 3: Linear methods for classification
Lecture 3: Linear methods for classification Rafael A. Irizarry and Hector Corrada Bravo February, 2010 Today we describe four specific algorithms useful for classification problems: linear regression,More information
AUTOMATION OF ENERGY DEMAND FORECASTING. Sanzad Siddique, B.S.
AUTOMATION OF ENERGY DEMAND FORECASTING by Sanzad Siddique, B.S. A Thesis submitted to the Faculty of the Graduate School, Marquette University, in Partial Fulfillment of the Requirements for the DegreeMore information
Class-specific Sparse Coding for Learning of Object Representations
Class-specific Sparse Coding for Learning of Object Representations Stephan Hasler, Heiko Wersing, and Edgar Körner Honda Research Institute Europe GmbH Carl-Legien-Str. 30, 63073 Offenbach am Main, GermanyMore information
By W.E. Diewert. July, Linear programming problems are important for a number of reasons:
APPLIED ECONOMICS By W.E. Diewert. July, 3. Chapter : Linear Programming. Introduction The theory of linear programming provides a good introduction to the study of constrained maximization (and minimization)More information
Data Clustering. Dec 2nd, 2013 Kyrylo Bessonov
Data Clustering Dec 2nd, 2013 Kyrylo Bessonov Talk outline Introduction to clustering Types of clustering Supervised Unsupervised Similarity measures Main clustering algorithms k-means Hierarchical MainMore information
A Survey on Outlier Detection Techniques for Credit Card Fraud Detection
IOSR Journal of Computer Engineering (IOSR-JCE) e-issn: 2278-0661, p- ISSN: 2278-8727Volume 16, Issue 2, Ver. VI (Mar-Apr. 2014), PP 44-48 A Survey on Outlier Detection Techniques for Credit Card FraudMore information
THREE DIMENSIONAL REPRESENTATION OF AMINO ACID CHARAC- TERISTICS O.U. Sezerman 1, R. Islamaj 2, E. Alpaydin 2 1 Laborotory of Computational Biology, Sabancı University, Istanbul, Turkey. 2 Computer EngineeringMore information
Automatic Web Page Classification
Automatic Web Page Classification Yasser Ganjisaffar 84802416 firstname.lastname@example.com 1 Introduction To facilitate user browsing of Web, some websites such as Yahoo! (http://dir.yahoo.com) and Open DirectoryMore information
SUPPORT VECTOR MACHINE (SVM) is the optimal
130 IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 19, NO. 1, JANUARY 2008 Multiclass Posterior Probability Support Vector Machines Mehmet Gönen, Ayşe Gönül Tanuğur, and Ethem Alpaydın, Senior Member, IEEEMore information
Monotonicity Hints. Abstract
Monotonicity Hints Joseph Sill Computation and Neural Systems program California Institute of Technology email: email@example.com Yaser S. Abu-Mostafa EE and CS Deptartments California Institute of TechnologyMore information
Modelling Handwitten Digit Data using Probabilistic Principal Component Analysis
Modelling Handwitten Digit Data using Probabilistic Principal Component Analysis Mohamed E.M. Musa, Robert P.W. Duin and Dick de Ridder Pattern Recognition Group, Department of Applied Physics Delft UniversityMore information
Large Margin DAGs for Multiclass Classification
S.A. Solla, T.K. Leen and K.-R. Müller (eds.), 57 55, MIT Press (000) Large Margin DAGs for Multiclass Classification John C. Platt Microsoft Research Microsoft Way Redmond, WA 9805 email@example.com information
Categorical Data Visualization and Clustering Using Subjective Factors
Categorical Data Visualization and Clustering Using Subjective Factors Chia-Hui Chang and Zhi-Kai Ding Department of Computer Science and Information Engineering, National Central University, Chung-Li,More information
Dimensionality Reduction: Principal Components Analysis
Dimensionality Reduction: Principal Components Analysis In data mining one often encounters situations where there are a large number of variables in the database. In such situations it is very likelyMore information
Neural Networks. CAP5610 Machine Learning Instructor: Guo-Jun Qi
Neural Networks CAP5610 Machine Learning Instructor: Guo-Jun Qi Recap: linear classifier Logistic regression Maximizing the posterior distribution of class Y conditional on the input vector X Support vectorMore information
Improved Fuzzy C-means Clustering Algorithm Based on Cluster Density
Journal of Computational Information Systems 8: 2 (2012) 727 737 Available at http://www.jofcis.com Improved Fuzzy C-means Clustering Algorithm Based on Cluster Density Xiaojun LOU, Junying LI, HaitaoMore information
C19 Machine Learning
C9 Machine Learning 8 Lectures Hilary Term 25 2 Tutorial Sheets A. Zisserman Overview: Supervised classification perceptron, support vector machine, loss functions, kernels, random forests, neural networksMore information
A K-means-like Algorithm for K-medoids Clustering and Its Performance
A K-means-like Algorithm for K-medoids Clustering and Its Performance Hae-Sang Park*, Jong-Seok Lee and Chi-Hyuck Jun Department of Industrial and Management Engineering, POSTECH San 31 Hyoja-dong, PohangMore information
Support Vector Machine. Tutorial. (and Statistical Learning Theory)
Support Vector Machine (and Statistical Learning Theory) Tutorial Jason Weston NEC Labs America 4 Independence Way, Princeton, USA. email@example.com 1 Support Vector Machines: history SVMs introducedMore information
Lecture 6. Artificial Neural Networks
Lecture 6 Artificial Neural Networks 1 1 Artificial Neural Networks In this note we provide an overview of the key concepts that have led to the emergence of Artificial Neural Networks as a major paradigmMore information
Face Recognition using SIFT Features
Face Recognition using SIFT Features Mohamed Aly CNS186 Term Project Winter 2006 Abstract Face recognition has many important practical applications, like surveillance and access control.More information
Acknowledgments. Data Mining with Regression. Data Mining Context. Overview. Colleagues
Data Mining with Regression Teaching an old dog some new tricks Acknowledgments Colleagues Dean Foster in Statistics Lyle Ungar in Computer Science Bob Stine Department of Statistics The School of theMore information


text:
Sterling, CO, January 13, 2017 – Ten members of the Northeastern Junior College faculty and staff were recognized with service awards at the start of the 2017 Spring Semester. Combined, the group represents 145 years of service at Northeastern.
Receiving recognition for 30 years of service was Lyle Rose, Physical Plant, Structural Trades.
Twenty-five years of service award was given to Cynthia Vandenbark, Director of Academic Technology.
Twenty years of service award was given to Melissa Bornhoft, Assistant Director – CLC.
Awards for 15 years of service went to Martha Gareis, Coordinator Mail/Document Center and Deb Walker, Accounting/Business Instructor.
Ten year awards when to Dale Brekel, Custodian I; Amanda Kerker, Associate Professor of Business; Justina McCracken, Assistant Professor of Biology.
Engraved pins for five years of service in a permanent position went to Jeff Schiel, General Social Science Instructor and Jody Kind, Assistant Professor of Nursing.
Northeastern Junior College • 100 College Avenue • Sterling, Colorado 80751 • 1-800-626-4637
The college is working on fully conforming the njc.edu website to the WCAG 2.0 AA standards for accessibility. During this transition, an alternative medium for accessing information about our programs and services can be provided by contacting Steven Smith at firstname.lastname@example.com or 970-521-6657. If you have any suggestions for improving the accessibility features of our website, please contact the webmaster at email@example.com.

subset: commoncrawl


text:
|"Yesterday, today and tomorrow" bush
|This orchid lives in the old frangipani tree
A compendium of snippets for family and friends
|"Yesterday, today and tomorrow" bush
|This orchid lives in the old frangipani tree
I grew up with a yesterday, today and tomorrow plant in the yard. I always have warm childhood memories when I see one. Did you see Bondi Vet recently when they had it on? (Lachlan is addicted to it, I have to watch it you know). Anyway they had a story on about a dog that ate the seeds of YTandT and started fitting and was quite unwell. Turns out it's poisonous for dogs.
Gee, thanks Louise! No, I had no idea it was poisonous for dogs. It's good to know these things because I have a dog who will eat many things ("Mmmm, interesting... concrete! Who knows, might *not* be poisonous!").
As we don't have a TV set up in the house I miss watching vet shows ... but in our busy lives, something *had* to give, and that thing was the TV.
Maybe Lachlan will become a vet? Does he know yet?


text:
The new FaceID recognition system from Apple and that comes with the iPhone X unlocks an iPhone more slowly than did Touch ID.
Tom's Guide is one of the sites that has done the tests with an iPhone X for the evaluation this week, and its reviewer Mark Spoonauer measured the comparison between the Face ID and Touch ID with a stopwatch .
" I have been using Face ID on the iPhone X for more than 24 hours, and I do not need a stopwatch to tell you that the phone is unlocked slower than when I used Touch ID on my old iPhone 7 Plus. I used a stopwatch application anyway to test it . "
It took 1.2 seconds for Spoonauer to unlock the iPhone X until it pressed the side button on the side of the device and another 0.4 seconds to slide until it reached the lock screen, while to get to the start screen on an iPhone 7 Plus by Touch ID took 0.91 seconds.
When using Elevate to activate and slide on the screen as the iPhone is recognizing a face, the unlock was faster in a total of 1.16 seconds but Spoonaeur has found it to be slower than the Touch ID.
In a raw comparison like that, the Touch ID seems to be the fastest unlocking method, but as Matthew Panzarino of TechCrunch points out, in real day-to-day use, Face ID has advantages over Touch ID because it is a more agile interaction .
With Touch ID, to do something like open a notification, you need to take advantage of the notification and then use Touch ID to open the notification, a two-step gesture. With Face ID, recognize your face as the phone rises while the notification is being played.
Panzarino says that Face ID is slower when placed face to face with Touch ID, which is "much more fluid and faster to really" do things ".
Face ID is a first generation technology, since in a Touch ID principles it was also much slower than it is today. In the future, Face ID will improve and get much faster , but it seems to be a more natural biometric authentication method that does not require thought or attention to users once they get used to the way it works.
Via | macrumors


text:
And the moth season continues with the Dotted Border - Agriopis marginaria (guess where its name comes from):
Sunday, 21 March 2010
A few sunny days....
the first hoverfly of the year appears:
I would have expected Eristalis pertinax, since I usually find these are the first large hoverflies to appear, but this turned out to be Eristalis tenax, the Drone Fly (so called because it resembles the male honey bee). The front face shows a very wide black stripe which is virtually diagnostic at this time of year:
And the moth season continues with the Dotted Border - Agriopis marginaria (guess where its name comes from):
4 comments:
- Gill said...
Interesting - my very early hovers usually are E. tenax rather than pertinax - and the 'marmalade hover' Episyrphus balteatus of course.
It's wonderful how a few hours' sun brings these insects out isn't it? Do they hibernate as adults? Where I wonder? You never seem to disturb them, unlike say peacock butterflies or queen wasps.
- 22 March 2010 at 12:37
- Stuart said...
Mated female Eristalis tenax overwinter in cracks in bark on trees, although I've never found them. The one in my photograph is certainly female, given the large gap between the eyes.
- 22 March 2010 at 17:28
- ScienceWriter said...
Great to see Spring has finally sprung!
- 26 March 2010 at 14:54
- Neil said...
Some nice finds and photos. Not seen a hoverfly yet, but Im sure it won't be long - if the rain ever stops....
- 1 April 2010 at 20:10
Subscribe to: Post Comments (Atom)


text: Welcome to Apple Lane Apartments! Apple Lane Apartments provide off-campus housing to for students of the University of Kansas, Haskell Indian Nations University, and those throughout the Lawrence area. Spacious studio floor plans are available. With ample free parking, nearby trails, close proximity to highways, and surrounded by a variety of retail stores and restaurants, it's no wonder so many people choose Apple Lane Apartments as their home!


text:
We have a vision for making a difference above and below the surface.
In a world, oftentimes, overshadowed with doom and gloom about the future we face due to climate change, sea level rise, ocean acidification, and many disasters, we bring experiential art and interdisciplinary collaboration into the equation to solve the world's complex challenges.
"Zoa" means animals of a specific kind, living beings.
We're passionate and dedicated to designing unique marine habitats, combining traditional and digital fabrication to regenerate biodiverse coral reef ecosystems.
Through curiosity and dedication to our wild ancestral architects, we aim to develop a new aesthetic + functional vocabulary for them to colonize and co-construct.
Zoaistas cultivate the conditions for interspecies life to flourish.
There are many ways to join us and help support the movement.
Lemuel Vega planting Agaricia Tenuifolia February 4, 2018.


text:
Description: Cindy Crawford and Alex Sanders finish up our movie by having a slower more romantic romp that shows sexual intensity isn't always about speed and noise. He doesn't nail her with shock and awe? he goes long and strong because he knows that's how he can
Duration: 20:39 min
Views: 54,593
Added: October 3, 2014 by robert5543


text: Each day Cuba Trade produces a Daily Briefing newsletter. It is a round-up of the current news around the web on doing business and investment in Cuba. If you would like to receive our newsletter Monday through Friday please send us your email address in the form below. Thank you.


text:
- Item Type: Necklaces
- Fine or Fashion: Fashion
- Style: Bohemia
- Necklace Type: Torques
- Gender: Women
- Material: Acrylic
- Chain Type: Twisted Singapore Chain
- Length: around 39cm
- Metals Type: Zinc Alloy
- Shape\pattern: Geometric
- Model Number: 6708-8
Welcom to our store!
4:We promise all of the products we sold is 100% brand new and in high quality.
A:Aliexpress does not accept Paypal.Please contact us if Paypal is your only choice.
Thank you for your time.
Shipping Terms
Standard Service is China post airmail(can be tracked) .
Usually it takes about 15~45days business days. It\'s trackable by your country post office website (China EMS website).
The shipping time during Holiday Season (Christmas, Chinese New Year etc) could be longer than usual time. Please add another 5-10 days as a delay.
Return Policy
1.For any reason,if you are not satisfied with the product,you can return it within 7 days after received.You will undertake the shipping fee.
2.If the product is defective,please contact us immediately for repairing or replacement.If a repairing and replacement is not available,we will offer the refund to you.
The five star feedback is highly appreciated .
- Unit Type: piece
- Package Weight: 0.04kg (0.09lb.)
- Package Size: 10cm x 10cm x 10cm (3.94in x 3.94in x 3.94in)
- Unit Type: piece
- Package Weight: 0.04kg (0.09lb.)
- Package Size: 10cm x 10cm x 10cm (3.94in x 3.94in x 3.94in)


text:
Member Since: February 7, 2011
Country: United States
I'm under the impression that fire does not emit much UV, but maybe in an otherwise closed environment? Try it! $13 to learn something new! EDIT: So apparently UV is used for flame detection, but this specific sensor may/may not work. See other comments.
I'm glad to see PDO is being used and a good thing. Around here, university students, professors, web develops, etc. mostly have little or no knowledge about PDO, and thus shun it as if it is some kludgey basement hackery not meant for proper development.
512 Bytes DRAM? Is this missing a prefix, since it can apparently run Ubuntu still?
These are typically meant to hook up to female blade connectors. If you want to solder to them, scuff the metal with sandpaper until it is all copper - solder will now stick easily :) They're not copper on the surface because copper is too soft for connectors and will also oxidize, so there's some sort of harder (and harder-to-solder) metal over the copper.
The LED will use all the current it can, that's why LEDs need a current-limiting resistor. This design has a 470ohm resistor. If I make a very wild guess (knowing it's a green LED) that the voltage drop through the LED will be about 2.4V, and we know I (current) = V (voltage) / R (resistance), we get I = (5.0V-2.4V)/470ohm = ~5.5mA, which isn't far off the given current spec. Generally, the max current spec of an LED is an absolute limit. You can run it anywhere below that :)
A bit late on the reply, but common household hot water tanks hold water around 60C (140F). Basically all that is needed hot tap water that isn't super hard, and a brush wouldn't hurt. Deposits from hard water could cause problems, but unlikely for hobby use.
I didn't get these from SparkFun, but have what seems to be an identical one from Princess Auto, so keep in mind I'm speaking about potentially different ones. Also keep in mind it is heavily used every day in an open University electronics lab.
It works great, much better than the giant plastic one we had before. I don't think the body is metal, just shiny-painted plastic, but it really doesn't affect the performance. The button lets the spring do its work without getting caught up, the tip doesn't melt even with the iron cranked to max, and it's all still looking new after much punishment.
The threads holding it together look questionable, but we've never had to take it apart. They don't get loose by themselves either.
Honestly, it's a good product if it's the same as we have. For $4, this is seriously decent. I usually detest cheap tools, but these work well!
Oh, and it doesn't really clog. Once in a while if you're just dumping solder into the thing, it can be a push to get everything out, but that's abusing the tool a bit.
If you guys make raise a huge amount extra… stop in Canada too? :)
I have the "normal" (Arduino form-factor) Maple, and can offer a few opinions. Keep in mind I don't use shields so those aren't important to me. The IDE is the same, the language is the same, the library has some small (and very welcome) differences. For example, use of analogWrite() is replaced by pwmWrite() (much more accurate English, plus Maple Native will actually have a DAC on it (so I hear) to possibly use a real analogWrite()). The integer types are different of course, being a 32-bit architecture; keep typedefs or use the ones existing for uint8, int16 etc. to have to avoid porting integer types. There's some pin issues with certain shields; a Google search will help you determine shield compatibility, and so will LeafLab's site/forum.
That said, the ChipKit UNO (SF, please look into carrying?) runs a hair faster clock (80MHz PIC32) that seems to get a huge digital I/O speed boost over the Maple. I assume one does not largely excel in number crunching over the other though.
I'd do a full comparison of benefits between Arduino (Uno), Maple, Arduino Pro Mini/Micro, and Maple Mini, but I only have two of those devices. Maple is a great platform, and it's nice to see something Arduino-like in hardware/programming concept with more power, and yet breaking away from the restrictive Arduino form factor. Personally, I liked the multi-function pin arrangements so that I could have 14 analog lines easily :)
"Really Ma, the dog ate my parts and I need more!"
No public wish lists :(


text:
"I have purchased numerous pieces of jewelry from Diamonds on Main over the past 16, almost 17, years. Each time I entered the store, I was greeted like an old friend. And each time I received very personalized service while picking out my jewelry. Typically I worked with JoAnne. She would inquire about the reason for my purchase and get a sense of what I was looking for: type of stone, setting, jewelry type, and price range. She would then suggest a number of different pieces. In almost all situations, I was able to pick something from one of her suggestions.
In addition, when I needed service, such as cleaning or an exchange of a piece, JoAnne and staff at Diamonds on Main were very accommodating. In fact, I purchased one piece that required some special work on it and because of the desired time frame that I needed it (my 15th wedding anniversary), she was able to work some "magic" with her vendor to expedite the request, so that I received it in time.
For those with discriminating jewelry tastes and who desire unique settings, I would highly recommend Diamonds on Main."
Dick Bartlett
"Diamonds on Main has been our favorite jeweler for more than a decade. They are professional, caring, and courteous. Their beautiful high-quality original designs keep bringing us back. We know we can trust them to find us exactly what we are looking for. We are proud to recommend Diamonds on Main!"
Child, Inc.
"When we stumbled in the door of Diamonds on Main, little did we know we'd forge a lifelong friendship with JoAnne and her staff. Our custom wedding and anniversary rings still receive compliments ate 15 years! Each time we decide to celebrate a loved one or an event in our lives, we always return to Diamonds on Main."
Forever friends,
Wallis and Lynda Brooks
We're honored to have been voted "Best Jeweler of 2018" in the St. Croix Valley.


text:
Stylish Event Space in Nassau County
Host an Event at Fox Hollow
Our stunning Nassau County event space is perfect for your next social event. Whether you're celebrating an anniversary, birthday party or Bar Mitzvah, hosting a special occasion brunch or an elaborate dinner, or searching for the best Sweet 16 venues in Long Island, The Somerley will meet all your needs and more. From intimate get-togethers to large family reunions and holiday gatherings, plan an event with the gorgeous backdrop of our Gold Coast estate.
Popular On-Site Social Events Include
- Weddings
- Anniversaries
- Bar/Bat Mitzvah
- Sweet 16 Parties
- Elaborate Dinner Functions
- Corporate Retreats
- Holiday Gatherings
- Post Wedding Reception


text:
Powered by PornMD
Lola James is an Ebony beauty from the Dominican Republic who now resides in Los Angeles. If you like pretty faces and stunning bodies on your pornstars, you are in for a serious treat. Lola has a beatific face, and every inch of her 32D-25-37 frame is utterly delectable. Her shapely legs lead to a beautiful, round ass. Lola's titties are perfection, and you need to see that pussy on camera to really appreciate it. Her co-stars get a much more intimate sampling, but fans are all too happy to watch.
Her first year in porno was 2012. She was playing with herself in flicks like I Love Big Toys 35, proving herself a master of solo work. Lola James also does really well in group sex situations, as evidenced in Orgy Masters. She's in a big orgy scene with pornstars like Kristina Rose, and her work is no doubt part of the reason this flick was nominated for the 2013 AVN Award for Best Orgy/Gangbang Release.
She does well in all-girl scenes, too, as can be seen in lesbian flicks like Girls Massaging Girls 2 with Britney Amber. Maybe it's just easiest to say everyone loves Lola James. She's beautiful, horny, and willing to do anything on camera. What's not to love?


text:
Lindsay Levesque
CD (DONA), Placenta Encapsulator
Lindsay Levesque has been practicing as a doula in the Bay Area since 2012, after the birth of her wonderful daughter. Lindsay is certified through DONA (emphasizing hospital birth support) and Sacred Birth Angels (emphasizing home and birth center support). Lindsay helps each client and partner investigate and address their personal preferences, concerns, and goals for birth. Lindsay works collaboratively with the client, partner, and birth team offering strong support while allowing space for the intimacy and privacy of labor. The support Lindsay offers is rooted in prenatal education of the physiological and hormonal processes of labor, mindfulness, breath, movement, and supportive attention. Childbirth is challenging for a reason; it teaches us to be brave, open, patient, strong, and tender among other qualities essential to parenting. Lindsay supports you through this important process.
Lindsay offers birth doula services and placenta encapsulation.


text: New patient, trouble with CIC Jul 13, 2017 7:19:03 GMT -8 Quote Select PostDeselect PostLink to PostBack to Top Post by barbalot on Jul 13, 2017 7:19:03 GMT -8 My daughter, 20, broke T12 and chipped L1 in an accident a month ago. After surgery and 2 1/2 weeks in a rehab hospital she is ready to come home, except for one thing. She just cannot get the hang of cathing herself. This is completely understandable as even some nurses/techs can't do it because her meatus is very small and hidden. And because of her injury she is barely able to reach to do it.I just started watching as the nurses do it, and I can't see it either, so I'm not sure I'll be able to cath her at home either, let alone be there every 4 hours since I work.She has made tremendous improvements in walking and is fairly mobile, although with weakness in her legs, so she is lucky in that regard. She's doing the bowel program OK. It's just the cathing at this point and I just don't know how she will be able to cath herself and be at home. Until this week her bladder was unable to release any urine on its own, but is now doing so (although she can't control it) but a lot is left behind.Looking for any advice on this.


text:
Shareist Documentation
1. Sign up at http://www.shareist.com
2. On the next page, click the "Create a notebook" button
3. Personalize your notebook URL
4. Choose a title for your notebook.
5. Choose a username.
7. Click the "Create notebook!" button.
Your notebook will be live within several seconds and you will be redirected to your notebook's dashboard. You can start posting content by clicking the "+ Add page" button.