text:
Add Document
Sign In
Create An Account
Scalable Hierarchical Multitask Learning Algorithms for Conversion Optimization in Display Advertising Amr Ahmed ∗ Google
[email protected]
Abhimanyu Das ∗
Alexander J. Smola ∗
Microsoft Research
Carnegie Mellon University and Google
[email protected]
ABSTRACT Many estimation tasks come in groups and hierarchies of related problems. In this paper we propose a hierarchical model and a scalable algorithm to perform inference for multitask learning. It infers task correlation and subtask structure in a joint sparse setting. Implementation is achieved by a distributed subgradient oracle and the successive application of prox-operators pertaining to groups and subgroups of variables. We apply this algorithm to conversion optimization in display advertising. Experimental results on over 1TB data for up to 1 billion observations and 1 million attributes show that the algorithm provides significantly better prediction accuracy while simultaneously being efficiently scalable by distributed parameter synchronization.
Categories and Subject Descriptors G.3 [Mathematics of Computing]: Probability and Statistics; I.2.6 [Artificial Intelligence]: Learning
In many cases data inference problems do not arise in isolation. That is, we usually encounter a range of related problems and there is considerable benefit in solving them jointly. This insight has been exploited repeatedly and it has led to algorithms commonly known as multitask learning techniques [4, 22, 12, 7, 18]. Applications, e.g. to massively multitasked spam filtering [20] show its practical importance. The key idea is that by solving related tasks we are able to learn more about an individual task. In this paper we study the problem of conversion maximization in display advertising. That is, we focus on maximizing the occurrence of commercially relevant actions such as purchases, account creation, mailing list signups, etc. This involves estimating a user's propensity to perform such actions and to identify generally susceptible populations of users. The challenge here is that we have both a broad range ∗
work done while the authors were at Yahoo! Research
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from
[email protected]
WSDM'14, February 24–28, 2014, New York, New York, USA. Copyright 2014 ACM 978-1-4503-2351-2/14/02 ...$15.00. http://dx.doi.org/10.1145/2556195.2556264.
[email protected]
of different advertisers and also a range of subtasks (views, clicks, conversions) that we wish to maximize. As is to be expected in computational advertising, the amount of data can be quite significant. Moreover, the data is not necessarily homogeneous. Tasks have wildly varying sizes, commensurate with the financial stake of the advertisers and the popularity of their product. Likewise, attributes are sparse and many occur only in a small number of contexts. This requires an effective inference approach. Our work builds on well known multiple kernel learning [19] and collaborative filtering techniques namely that of effectively imposing a hyperprior on the regularization term. In doing so it is possible to cast multitask learning as a nontrivial convex optimization problem. See e.g. [23] for details. This strategy is then combined with a hierarchical model over task and subtask specific parameters. Furthermore, we impose structured sparsity along the lines of [5]. To solve the problem in practice we rely on a distributed subgradient oracle. Load-balancing is achieved by using consistent hashing for task distribution over processors and distributed variable aggregation to mitigate the latency and task restarts otherwise required in Hadoop MapReduce. That is, we use the variable distribution of [13] for storage. Subsequently we invoke a sequence of prox operators [5] to synchronize efficiently between local and global penalties. To summarize, our contributions are the following: • We formulate the joint conversion, click and unattributedconversion modeling problem in behavioral targeting as a large-scale hierarchical multitask learning problem and show that the convex multitask learning approach of [23] can be adapted to this setting. • We design an efficient distributed implementation of the above algorithm that scales to Terascale data. • Using a real-world, web-scale display advertising targeting data set and two smaller public datasets, we show the ability of our algorithm to significantly improve on the baseline modeling performance obtained by traditional single-task inference systems.
Challenges in Conversion Maximization
Recent trends in behavioral targeting and display advertising emphasize the importance of commercially relevant actions. That is, rather than user clicks, advertisers aim to maximize the number of conversions they receive. Conversions stand for purchases, account creation, or any other relevant action that an advertiser sees as particularly desirable. To obtain good performance, publishers tend to instrument
their websites with embedded code which allows third parties to capture user transactions and generate user segments that are of high value for a particular advertiser [1]. These segments primarily contain users that are actually inclined to perform a transaction as opposed to a casual visit to the web site through an accidental click. Conversions on an advertiser's web site are either "attributed" to their corresponding display ads based on advertiserspecific rules, such as the amount of time elapsed between the conversion time and the time that the ad was last shown to the user, or they are "unattributed" if they cannot be tied to a specific display ad. Past work [3] has shown the superiority of targeting platforms maximizing for attributed conversions to traditional solutions maximizing for clicks. For conversion-oriented behavioral targeting the traditional approach has been to only consider attributed conversions. The corresponding inference problem for each advertising campaign is then solved independently, for instance, by fitting a Support Vector Machine or Logistic Regression model. This generates separate models based on user data for each campaign. However, a typical behavioral targeting platform optimizing for attributed conversions, henceforth simply referred as conversions, faces two core issues: • There is a large volumes of user histories that need to be processed in a periodic fashion in order to perform inference over attributed conversions for a large number of ad campaigns. Processing activities of billions of users on a daily basis imposes many challenges such as how to build user profiles in an efficient way, and how to optimize multiple campaigns at the same time; • When optimizing for each campaign separately we are likely to do poorly for infrequent campaigns. We therefore need to design algorithms that can deal with sparseness of attributed conversions in many campaigns. The absence of a sufficient number of labeled data for the inference tasks creates a major bottleneck against achieving tangible targeting performance improvement.
Multitask Learning
For each advertising campaign, we can formulate several other related inference tasks apart from conversion modeling: we can attempt to infer the likelihood of a click (which is typically a prerequisite for conversion) and to model the likelihood of unattributed conversions (the latter helps to identify similar users). These different inference problems of each campaign are likely to be correlated. Hence modeling them jointly should improve estimation performance. Additionally, it is quite likely that there is significant correlations between inference tasks across different advertising campaigns. For example, if there exists advertising campaigns corresponding to different brands of cars, the conversion or click models for all these campaigns might be quite similar to each other. Thus, performing a joint inference over all these potentially-correlated tasks together might lead to better performance than solving these inference tasks separately. This is the basic premise of multitask learning. The key difference to conventional multitask learning is that in our case there exists a hierarchy between tasks. That is, we expect that all tasks for a given campaign (e.g. selling car insurance for a particular company) have related specificity in terms of their user demographic. Hence it is only reasonable that the sets of features and preferences are
shared between them. It is to be expected that joint feature selection should improve the performance of each estimator. Strictly speaking, we have two somewhat related goals — one is to do well for all tasks, i.e. click, conversion, and unattributed conversion estimation. This is a symmetric setting where the goal is to use the task correlations to simultaneously improve the prediction performance of all the tasks. The other task is to perform well for conversion prediction while using the remaining data as side information. This is an asymmetric setting. While both tasks are rather related, they are subtly different in their performance criteria and in terms of the estimation problem. In this paper, we formulate two different hierarchical multitask models for these settings: a hierarchical model for the symmetric setting, and an attachment model for the asymmetric setting. In the hierarchical model, we first define an inter-campaign correlation matrix on a root-level set of feature weights for each campaign. This is then used to derive feature weights for its conversions, clicks and unattributed conversions locally. In the attachment model, the intercampaign correlation matrix is applied directly on the feature weights for the conversion model of each campaign. There is ample literature covering the subject of multitask learning. However, in the context of our behavioral targeting problem, there are two objectives that a multitask learning algorithm should satisfy: it should be easily distributable and scale to thousands of campaigns and millions of features; it should extend to a multi-level task hierarchy. This makes the setting rather nontrivial in terms of efficient inference.
In this paper, we use a convex formulation approach for multitask learning. Its basic idea is described in several variants e.g. in the context of multitask learning [23], matrix inference [16] and multiple kernel learning [14]. Essentially, one imposes a penalty over the covariance matrices governing correlation between attribute vectors. This way we can ensure that primarily similar attribute sets and related attribute vectors are chosen. In terms of conversion maximization this means that we use an inter-campaign covariance matrix to model the relationships between the various campaigns. Moreover, intracampaign covariance matrices are used to model the relationships between clicks, conversions and unattributed conversions of each campaign. A matrix-variate normal prior is imposed on these covariance matrices. A joint optimization objective can be formulated for all the tasks by obtaining a maximum likelihood estimate of the covariance matrices and the per-task feature weights for all the tasks. As we will show later, this objective is convex. It can be solved in a distributed fashion using proximal subgradient methods, such as the Fast Iterative Shrinkage algorithm (FISTA) [6]. For both models, we use a scalable alternating subspace descent method for simultaneous inference of both the task correlation matrices and the feature weights. A key tool for achieving scalability will be to use a cluster of machines as a distributed subgradient oracle [17]. Since iterative thresholding algorithms like FISTA require a significant number of gradient computations (e.g. [6] report 100 steps), our platform must preserve state and data locality between iterations. This makes it unsuitable to a naive Hadoop MapReduce implementation. Instead, we employ a consistent hashing based synchronization algorithm. We apply our dis-
i ∈ {1 . . . mc } c ∈ {1 . . . m}
i ∈ {1 . . . msc } s ∈ {1 . . . mc }
campaigns c and associated labels ycj , e.g. whether a particular user converted on an ad in a particular campaign at a particular occasion, we consider the issue of estimating y|x, c for a large range of campaigns simultaneously. We denote by m the total number of campaigns and by mc the number of observations per campaign. Formally we consider sets of covariates and labels indexed by a campaign c, denoted by X c = {xc1 , . . . , xcnc } ⊆ X and Y c = {y1c , . . . , ync c } ⊆ Y. Here each pair (xcj , ycj ) is drawn from some distribution p(x, y|c) = p(y|x, c)p(x|c) of covariates and labels respectively. Finally, we denote by csi the combination of task, subtask, and coordinate, and by · the entire vector in the associated dimension. E.g. wc·i denotes the vector over all subtasks s associated with c for coordinate i.
Θc c
{1 . . . m}
Figure 1: Top: Standard multitask learning using a Matrix-Variate distribution. Observations Xci for campaign c receive labels Yci . These are assigned using campaign-specific weight vectors Wc . The latter are exchangeable but not independent, hence jointly drawn from some distribution p(W |Ω). Bottom: Hierarchical multitask learning. After drawing task specific parameters Zc from an exchangeable but not independent distribution we draw subtask specific parameters Ws using a joint parameter Θc and Zc . The rest remains unchanged. tributed multitask learning framework to the conversion modeling problem described in [1] and we show how our system can improve the AUC performance significantly, when compared to individual conversion modeling of each ad-campaign.
2. 2.1
MULTITASK LEARNING Notation
We now cast the problem of campaign-specific estimation as a multitask learning problem. That is, we treat each campaign c as a task. In each such case we observe covariates (patterns) xci and our goal is to infer labels yci . For simplicity we assume that xci ∈ X = Rd consists of ddimensional vectors and moreover that yci ∈ Y are either binary Y = {±1} for classification or real-valued Y = R for regression. At a later stage (in section 3) we will assume that each campaign contains a number of subtasks s. The table below gives an overview of the symbols used: X Y c s j x Xc y Yc w W z Z
domain of observations (usually X = Rd ) domain of labels (usually Y = {±1} or Y = R) campaign index (c ∈ {1 . . . m}) sub-campaign index (s ∈ {1 . . . n}) observation index (j ∈ {1 . . . mc } or j ∈ {1 . . . mcs }) observation (xcj or xcsj ) set of observations for campaign c observation (ycj or ycsj ) set of labels for campaign c parameter vectors (wc or wcs ) stacked parameter vectors W = {. . . wc . . .} parameter vectors zc for top level hierarchy stacked parameter vectors Z = {. . . zc . . .}
Figure 1 captures the formal structure of the multitask learning problem. To capture interaction between covariates xcj ,
The inference problem is expressed either of risk minimization whenever we want to find a classifier which makes a small number of mistakes, or as one of maximizing the data likelihood. In the latter case we want to find parameters W = {w1 , . . . wm } such that the maximizes label likelihood: p(Y |X, W ) =
m Y
p(Yc |Xc , wc ) =
mc m Y Y
p(ycj |xcj , wc ) (1)
c=1 j=1
Choices for ycj |xcj , wc are e.g. 1 − 1 (y − x ,w )2 p(ycj |xcj , wc ) = (2πσ 2 )− 2 e 2σ2 cj h cj c i 1 p(ycj |xcj , wc ) = −ycj hxcj ,wc i 1+e
(2) (3)
for regression and classification respectively. A naive maximization of the conditional response likelihood p(Y |X, W ) leads to overfitting unless the model complexity is overly small or unless a suitable prior is used. Multitask learning models aim to address this problem by imposing a suitable prior distribution p(W ) on W which favors simple models and which exploits correlation between tasks. Consequently, instead of maximizing (1) one aims to find the Maximum-aPosteriori (MAP) estimate of W via maximize p(W ) W
m Y
p(Yc |Xc , wc )
The challenge is now to define models of p(W ) that are both computationally tractable and statistically meaningful. This hierarchical modeling imperative leads to multitask learning.
Multitask Prior
Our working assumption is that W is drawn from a matrixvariate distribution in such a way as to exploit correlations between the tasks, such as assuming that the tasks are more concentrated in a lower dimensional subspace. This is a reasonable assumption since there is no inherent order in which the tasks are laid out. One option is to choose a normal distribution as follows: W ∼ N(0, 1d ⊗ Ω) or equivalently w·i ∼ N(0, Ω)
for all coordinates i. The likelihood of W |Ω is given (up to constants) by − log p(W |Ω) = tr W Ω−1 W > + d log |Ω| + c
It is straightforward to modify this by including a conjugate Wishart hyperprior on Ω. Unfortunately, the outcome is concave in W . An alternative is to replace the log-barrier arising from a conjugate prior on Ω by a trace constraint and a positive semidefiniteness constraint. That is, we replace log |Ω| by Ω 0 and tr Ω = 1. This is used, e.g. in [23]. Such a modification leaves the eigenspace of the W -dependent part of the optimization problem unchanged. This leads to the following alternative: X minimize − log p(Yc |Xc , wc ) + λ tr W Ω−1 W > (7a) W,Ω
This approach forms the baseline relative to which we will compare our proposed method.
While the flat models presented in Section 2 can learn the correlation structure between tasks, they are not so easily amenable for distributed optimization because of the squared dependency between all the tasks. Fortunately, many large scale multitask problems possess a hierarchical structure that allows us to decompose them into tasks and subtasks. For example, in display advertising each advertiser can be regarded as a task (a campaign) within which we can define three subtasks as follows: • Conversion prediction: estimate if the user will convert, i.e. perform a commercially relevant action, on the current display ad. • Click prediction: predict if the user will click on the currently displayed ad.
• Hierarchical Model: We assume that for each task group there exists some parameter vector zc , with Z =
w43 w42
w33 w32
Ω w11
Figure 2: Top: Hierarchical dependency structure over parameter vectors for multitask learning. The intermediate parameter vector zc encapsulates commonalities per task. Bottom: Attachment model. Here the conversion-specific parameters are directly coupled. For simplicity of the diagram we omitted Θc in both cases. {z1 , . . . zm } that specifies preferences per task Z ∼ N(0, 1d ⊗ Ω) or equivalently z·i ∼ N(0, Ω)
Moreover, within each task, the distribution over subtasks is given by wc·i ∼ N(1 · zci , Θc ).
This assumes that correlations within subgroups are decoupled. • Attachment Model: Denote by s = 1 the primary subtask (conversion estimation). Instead of using zc as an intermediary we couple the models directly via wc1 and use a hierarchical model on the remaining parameters. This amounts to
• Unattributed conversion: historical data of users who converted on previous advertisements of the advertiser. We use s to index the subtask. That is, rather than c we now use the tuple cs to index task and associated subtask, such as (Coca Cola, clicks). All remaining notation is unchanged relative to the previous section. In a nutshell we have two options for dealing with the hierarchical structure: firstly, we estimate the joint model for all tasks, subtasks and all campaigns. A second strategy is to solve the model for the primary subtask of conversion estimation exclusively and to use the associate (secondary) subtasks only as side-information. We will refer to the former as a hierarchical model and to the latter as attachment model. The key difference is in the following assumption:
The above formulation is convex in both W and Ω and can be solved using an efficient algorithm based on alternating subspace descent. For fixed Ω minimize (7) with respect to W . Subsequently, for fixed W , find the minimizer with respect to Ω. A simple constrained optimization problem shows that this can be found via > − 21 W W ˆ (8) Ω= 1 tr [W > W ]− 2
subject to Ω 0 and tr Ω = 1
w·1i ∼ N(0, Ω) and wc·i ∼ N(1 · wc1i , Θc ) for s > 1. The diagram in Figure 2 describes the difference between both approaches for a rather simplistic structure of 4 tasks (in reality we may have millions of such tasks). As previously discussed in Section 2, we again resort to a reformulation that uses a trace constraint and positive semidefiniteness rather than the log-barrier to restrict Ω and Θc . That is, instead of − log p(W, Z|Ω, Θ) (11) X1 = tr(wc· − 1 · zc )> (wc· − 1 · zc )Θ−1 c + mcs log |Θc | 2 c +
1 tr Z > Ω−1 Z + mc log |Ω| + c 2
for the hierarchical model and analogous setting for the attachment model, we use the following objectives L(W, Z, Ω, Θ): Lhier (W, Z, Ω, Θ) (12) X1 1 > −1 > −1 tr(wc· − 1 · zc ) (wc· − 1 · zc )Θc + tr Z Ω Z = 2 2 c
hierarchical multitask model: X 1 minimize − log p(ycsj |xcsj , wcs ) + tr Z > Ω−1 Z W,Z,Ω,Θ 2 csj +
+ λ1 kZk1 + λ2 kZk2,1 X + λ1 kW k1 + λ2 kWc k2,1
subject to tr Θc 0 and tr Θc = 1 and tr Ω 0 and tr Ω = 1 for the hierarchical model. Moreover, for the attachment model: Lattach (W, Ω, Θ) 1 1X > > = tr wc· wc· Θ−1 w·1i Ω−1 w·1i c + 2 2 i
Moreover, the attachment multitask model yields: X minimize − log p(ycsj |xcsj , wcs )
1X 1X > > w·1i Ω−1 w·1i tr wc· wc· Θ−1 c + 2 c 2 i X + λ1 kW k1 + +λ2 kW k2,1 + λ2 kWc k2,1 +
Moreover, the mixed norm of a matrix X, where we apply sparsity row-wise, is defined for p, q ≥ 1 via
(14) kXkp,q := kX1· kp , . . . kXd· kp . q
Of particular interest is the kXk2,1 norm, which attempts to eliminate entire rows of X at a time. Finally, we use the abbreviation kXk1 := kXk1,1 to denote the sum over absolute values in X. This leads to the following sparsity penalties for the hierarchical and attachment models respectively: Shier (W, Z) =λ1 kZk1 + λ2 kZk2,1 + X kWc k2,1 λ1 kW k1 + λ2
subject to Ω, Θc 0 and tr Ω = tr Θc = 1.
The coefficients λ1 and λ2 govern the trade-off between generic sparsity and group sparsity. That is, for λ2 there will be no correlation in sparsity patterns beyond what is obtained from data. For λ1 = 0 we can assume that whenever any given Wcsi 6= 0 then also all related Wcs0 i will not vanish.
Optimization Problems
We conclude this section by stating the two optimization problems that we will solve subsequently. The key ingredients are a likelihood function − log p(Y |X, W ) which depends on the specific problem to solve, a simplified multitask learning penalty as defined by Lhier (W, Z, Ω, Θ) and Lattach (W, Ω, Θ) respectively, and a sparsity penalty as in Shier (W, Z) and Sattach (W ). We have the following for the
The optimization problems (17) and (18) are jointly convex in (W, Z, Ω, Θ) and (W, Ω, Θ) respectively. For practical optimization we resort to a Gauss-Southwell [15] approach of minimizing blocks of parameters at a time. In practice this means that we alternate between minimizing with respect to W, Z and Ω, Θ. This is known to converge to the globally optimal solution (albeit slowly on occasion). Issues of problem distribution and parallelization will be discussed in the next section.
Covariance Updates
Assume that we are given W, Z. In this case we may find optimal values for the psd matrices Ω, Θc using the derivation in (8) as follows: Ω = (tr O)−1 O
> −1 2
where O = (ZZ ) " #− 1 2 X > O= w·1i w·1i
(hierarchical) (attachment)
kWc k2,1 (16)
Structured Sparsity
A second aspect of multi-task learning is to use structured sparsity [5] to select relevant variables for an entire block of terms jointly rather than eliminating terms for each task individually. This is achieved by adding a mixed norm on the parameters W and Z to the optimization problem. We need some more notation first: the p norm of a vector x ∈ Rd : X kxkpp := |xi |p for p < ∞ and kxk∞ := max |xi | .
Sattach (W ) =λ1 kW k1 + λ2 kW k2,1 + λ2
subject to Ω, Θc 0 and tr Ω = tr Θc = 1.
subject to the same constraints as for Lhier . The only real difference is that we eliminated z and instead, we attach the model to wc1 directly. In either case this detaches the subtasks from the problem of joint task inference.
X1 tr(wc· − 1 · zc )> (wc· − 1 · zc )Θ−1 c 2 c
Likewise, for Θc we have the updates Θc = (tr T )−1 T " #− 1 2 X > where T = (wc·i − zci )(wc·i − zci )
(20) (hierarchical)
#− 1
> wc·i wc·i
This means that we can compute T entirely with only access to all subtask specific parameters s for a given campaign c. Hence, as long as it is possible to have all such data available on a single machine, we need not communicate the lower level of the hierarchy outside the machine.
Next we need to discuss update steps in terms of W and Z. Recall that we imposed a mixed norm penalty on both terms such that we obtain group sparsity. Our strategy borrows from [5], and [10, Proposition 1]. Recall the structure of the penalties imposed by k·k2,1 and k·k1 . They constitute a hierarchy over nonzero terms in W and Z respectively — the (2, 1)-norm attempts to zero out the entire set of contributions for a given coordinate and the 1-norm ensures that even if we use an attribute, we only use it sparingly. Given Ω, Θ the remainder of the problem is a convex unconstrained optimization problem. One of the algorithms recently proposed are of a structure resembling FISTA (Fast Iterative Successive Thresholding) [6]. In it one interleaves a gradient descent step with regard to the convex differentiable part of the objective with a thresholding step with regard to the sparsity penalty. That is, for the problem: minimize f (a) + λΩ[a]
λ2 kzc k2 +
kwcs k2
Solving (23) can be carried out for each task c and for each zc and wc individually, hence it is amenable to easy distribution. Using [10, Proposition 1] one can see that performing successive prox operations with respect to the `1 norm and subsequently with respect to the `2 norm lead to an exact solution of (23). For instance, for zc this means that we perform the following steps zc ← zci − ti ∂zc F[Z, W ] zci ← sgn zci max(0, |zci | − ti λ1 )
(25) (26)
and subsequently we threshold the entire vector via zc ←
zc max(0, kzc k − ti λ2 ) kzc k2
Client Client Client Client
process for shrinkage over attributes
Here (22) is essentially a gradient descent step in f . The step (23) is commonly referred to as a prox-operator. The step size ti is chosen such that t−1 majorizes the Lipschitz i constant of ∂a f (a). We discuss computing gradients with respect to the objective in Section 4.3 for the hierarchical model (the attachment model follows similarly). In this context we mean by F[Z, W ] either the first two lines of (17) or of (18) with Ω and Θ fixed at this point. For now note that in our case Ω[a] decomposes into penalties applied per task. That is # " X X (24) kwcs k1 + Ω[W, Z] = λ1 kzc k1 + c
one performs the following steps (after initializing a0 ) bt+1 := at − ηt ∂a f (at ) and 1 ka − bt+1 k2 + λΩ[a] at+1 = argmin 2tt a
Figure 3: The parameter matrices W and Z are of size Rd×m . For gradient computation we need to have access to all parameters for a given task c on a given machine, hence the row-wise split. Subsequently, to perform shrinkage over attributes, we need all parameters pertaining to a feature (for all tasks) on a given machine. Load balancing for both of these tasks is achieved by consistent hashing.
process tasks independently
Optimization with Sparsity Penalty
In other words, first we perform gradient descent. Then all coefficients that are individually too small are eliminated and the remainder is shrunk. Finally, we perform shrinkage of the remainder in the direction of their unit vector. The objective is either that of the hierarchical or of the attachment model. Updates with respect to W are entirely analogous and therefore omitted.
We complete our overview of optimization by discussing the gradient in terms of W and Z. As before, we limit ourselves to a discussion of the hierarchical model of (17). Since the reasoning required for (18) is essentially identical, we omit the details. Straightforward calculation yields ∂zci F[Z, W ] = Ω−1 Z·i c + Θ−1 (28) c (1 · zci − wc·i ) X ∂wcsi − log p(ycsj |xcsj , wcs ) (29) ∂wcsi F[Z, W ] = j
+ Θ−1 c (wc·i − 1 · zci ) As can be seen, again all gradients decompose in terms of tasks and subtasks respectively. We will exploit this for distributed optimization. The exact form of the gradient for p(ycsj |xcsj , wcs ) is straighforward to compute for both the regression and classification problem given the form of the conditional probability in (2) and (3) respectively.
We now discuss how to implement a distributed optimization algorithm efficiently on a cluster of commodity workstations. As discussed previously, invoking steps (22) and (23), and updating Ω and Θ requires the following operations: 1. Compute partial subgradients of F[Z, W ] for all campaigns with respect to Z and W . 2. Aggregate subgradients obtained from all instances and apply it to the model parameters. 3. Distribute coordinates (or subsets S ∈ S thereof) of the subgradients (or rather updated coordinates) to clients for application of the prox operator. 4. Invoke the prox operator. 5. Redistribute the results to the machines holding the campaign-specific data. Since this is an iterative procedure with two barriers per iteration (send subgradients, return values) this sounds as if it were a good fit for MapReduce. Unfortunately, this approach suffers from inefficiencies inherent in the Hadoop implementation of MapReduce: context in the mappers is not
preserved between iterations. Moreover, Hadoop communicates primarily via file I/O. Since the proposed algorithm can take tens of iterations and since we need to communicate parameters repeatedly, this means significant waste of resources by repeatedly having to initialize the state of the mappers. Hence we resort to a method discussed in [13, 2], namely to allocate the machines using Hadoop and then to establish an overlay communication network.
and a finalizing pass (plus prox step) in a coordinate-specific fashion. Likelihood gradients: Since data is partitioned according to tasks c, subgradients with regard to wcs are easily computed via gcs =
ncs X
∂wcs − log p(ycsj |xcsj , wcs ).
Algorithm 1 Distributed Optimization 1: for all i = 1 · · · p parallel do 2: read data blocks from disk 3: for all campaigns with m(c) = i do 4: compute subgradient gc 5: end for 6: write gc to (key,value) store according to m(i) 7: end for 8: reach a barrier 9: for all i = 1 · · · p parallel do 10: read gc from (key,value) store according to m(i) 11: for all coordinates with m(j) = i do 12: solve the prox operator 13: end for 14: write to the (key,value) store according to m(c) 15: Compute contribution to sufficient statistics of Ω and write it back to shared memory. 16: end for 17: reach a barrier 18: Read sufficient statistics of Ω and compute new value.
Data and Task Distribution
In the following we assume that we have p machines to process data. Recall that d denotes the number of attributes, i.e. xcsi ∈ Rd and that m denotes the number of campaigns. We use randomized load-balancing to determine which machine receives which portion of the data in both the databound and the parameter-bound part of the optimization procedure. This is achieved, e.g. by consistent hashing [11]: m(c) = argmin h(m, c) and m(i) = argmin h(m, i) m∈M
to assign machines from a machine pool M for campaigns c and coordinates i respectively. Finally, data exchange is carried out in the form of a distributed (key,value) store. For reasons of practicality we used memcached as our reference implementation. This follows the design pattern of [21, 13] and it avoids file I/O for synchronization. Such a strategy is much more efficient than repeated invocations of Hadoop MapReduce.
Distributed Subgradient Oracle
By design, the subgradients of F[Z, W ] and F[W ] decompose into terms that are easily computable in a campaignspecific manner (terms related to the negative log-likelihood) and terms that are easily computable in a coordinate-specific manner (the penalties in terms of Ω, Θ and the sparsity penalties). Furthermore, only the former requires direct access to data, whereas the latter requires access to a given coordinate across all tasks. This means that we can compute gradients in two stages: a pass over data, as performed by the workers that have the data, a reshuffle of parameters,
P > −1 Θc wcs , Next we compute gradients with respect to 12 s wcs i.e. P we add Θ−1 c wc· to gcs . The analogous reasoning holds for s (wcs − zc )> Θ−1 c (wcs − zc ). These gradients are then redistributed according to Figure 3 such that all gcsi for a given coordinate i (ranging over all tasks and subtasks) are available on the same machine. Multitask gradients: At this point we can compute coordinate specific parts as arising from the Ω-dependent terms on a per-coordinate basis. For this purpose we only need Z·i or w·1i , depending on whether we chose the hierarchical or attachment model respectively. We only need to read the weights corresponding to non-zero entries in Ω−1 .
Distributed Prox Operator and Covariance Estimation
The final step required is to solve the prox operator related to the k·k1 and k·k2,1 norms as these enforce sparsity. Whenever we have a fully hierarchical setting, Proposition 1 of [10] applies and we can simply perform prox steps bottom up in the process. Whenever this assumption is not satisfied, we may still iterate the prox operator to obtain a suboptimal solution. This suffices as a descent step, since optimization in Z and W is just a subroutine in the overall optimization scenario involving Θ and Ω. Note that the prox operator can be carried out in linear time — we only require computing norms of vectors and rescaling them. The data exchange is completely analogous to the gradient computation, except that we now work on attributes rather than campaigns. After the prox operation we redistribute parameters back into a (key,value) storage. As before, this requires a barrier to ensure that up-to-date values are available on all workers for another pass through the data. Similar to the gradient computation phase, the read and write steps can be performed in parallel. Finally we note that estimating Θ can be done locally in each worker however the sufficient statistics required to compute Ω (see 19) is distributed on a per-attribute basis. Thus we overlay this step with the prox-operator step. Each worker computes its contribution to the sufficient statistics using its assigned attributes. For example in the hierarchical model this reduces to computing a C × C matrix P > i∈m(i) zi zi . After reaching a barrier, worker 0 then reads those partial sums and computes the new value for Z using (8) and then writes Ω back to a shared memory to be read by each worker for the next iteration. Alternatively each worker can read the sufficient statistics of Ω and compute the new value deterministically. Moreover, instead of using (8), we could use the graphical lasso estimation of [9] to get a sparse inverse covariance estimation of Ω−1 from its sufficient statistics. This sparse inverse covariance is desirable In distributed settings to minimize parameter movements when computing the multi-task gradient as it depends on the non-zero elements of the inverse covariance (i.e. Ω−1 ).
MTL Attachment
MTL Hierarchical
MTL Flat AUC score
R2 score
Single Task
MTL Hierarchical
0.68 0.75
MTL Flat
Single Task
0.7 0.66
0.65 0
Figure 4: School Data: R2 Performance for various multitask learning algorithms. Note the faster convergence and better performance of ATT-MTRL.
To establish the efficacy of our approach we report results on both two public datasets and one proprietary dataset. We make this choice since we are unable to share proprietary and financially relevant data outside Yahoo, yet at the same time we wish to provide the reader with some means of gaining insight into the working of the proposed algorithm. In other words, we show that the proposed algorithm improves on the state of the art and simultaneously that it scales to substantial problem sizes. In terms of public datasets we choose two standard datasets: estimation of examination scores for students attending secondary schools in London and multi-task classification of the 20-newsgroup dataset. The algorithms we compare are the Hierarchical MTL (HIE-MTRL) and Attachment MTL (ATT-MTRL) algorithms, along with two baselines: the first one is the Single-task Learning (STL) algorithm which does not use multitask learning, and optimizes all the tasks independently. The second baseline is the Flat MTL (F-MTRL) algorithm of [23] that uses a matrix-variate normal prior on the task correlation matrix. This algorithm performs multitask learning, however it does not account for the hierarchical task and subtask structure. It "flattens" the task hierarchy and treats (task, subtask) as individual tasks and learns a joint covariance structure.
Student score estimation
This dataset has been used widely for studying multitask regression1 . It consists of the exam scores of 15, 362 students from 139 secondary schools in London during 1985, 1986 and 1987. Originally, the input consists of the year of the exam, four school-specific and three student-specific attributes. The goal is to estimate the exam scores for the students. Several papers [23, 4] evaluate multitask learning by treating the school-ids as attributes, but one could arguably treat this dataset as specifying a hierarchical task/subtask structure, where the school ID refer to tasks and the exam years for each school correspond to the subtasks of the school ID. Thus, there are a total of 139 tasks, with up to 3 sub1
Figure 5: 20-newsgroup MTL: AUC Performance for various multitask learning algorithms. The hierarchical MTL algorithm starts off better than the flat algorithm and it consistently outperforms both single task and flat multitask learning. tasks for each task. We note that several tasks have only 2 tasks in the dataset. We replace each categorical attribute with one binary variable for each possible attribute value as in [4] but remove the attributes corresponding to the exam years. As a result of this preprocessing, we have a total of 24 input attributes. We use a 66/34 split of the dataset to use as training instances and test instances, and report the average (over all years and schools) performance of our algorithms, on the test set. For our performance measure, we use the normalized inner product between the input score vector and the predicted score vector. This measure is proportional to the squared multiple correlation coefficient R2 , a normalized version of the regression error, defined as P (yi − fi )2 . (32) R2 := 1 − i Var[y] That is, it is the ratio between explained variance and total variance. See e.g. [8] for further details. For F-MTRL, HIE-MTRL and ATT-MTRL we use 5-fold cross validation to determine the optimal value of the appropriate regularization constants and learning rate. Figure 6.1 plots the R2 performance of the algorithms as optimization progresses. As can be seen in the graph, and as also reported by [23, 4], multitask learning provides a significant performance improvement. F-MTRL improves over the baseline by around 5%. However, by using the task-subtask hierarchy, our HIE-MTRL and ATT-MTRL obtain a further improvement from a score of around 0.71 (for F-MTRL) to almost 0.73. The performance for ATT-MTRL was slightly better than HIE-MTRL in this dataset.
Multi-Task Classification
The task at hand is multi-task classification of the 20newsgroup dataset2 . The goal here is to predict the newsgroup of a given post. The 20 news groups are arranged into a two-level hierarchy. The first level comprises 5 categories: 2
http://qwone.com/ jason/20Newsgroups/
Table 1: 20-newsgroup analysis: AUC performance for single task and multitask algorithms on varying percentages of the data. Fraction of data STL F-MTRL HIE-MTRL 20% 0.755 0.775 0.827 40% 0.821 0.839 0.881 60% 0.875 0.893 0.910 80% 0.899 0.917 0.932 100% 0.918 0.931 0.957 politics, religion, recreational, science, and computers. Each category has 2-5 subcategories in the second level. We map categories to tasks and sub-categories to sub-tasks. The dataset comprises 18k documents and we followed the standard test/train split. We removed stop words and words appearing less than 10 times. Five-fold cross validation is used to determine the values of the regularization parameters for all models. We measure classification accuracy using the AUC measure. Since in this dataset we do not have the notion of an anchor task, we only use the symmetric HIE-MTRL formulation. In Figure 5 we compare the performance of HIEMTRL against the state of the art algorithm in [23] F-MTRL and against single-task baseline STL. As evident from figure HIE-MTRL improves over the F-MTRL baseline by around 2% points and the improvement was statistically significant. Moreover HIE-MTRL outperforms all other competing algorithms in terms of speed of convergence. To see the effect of varying the training set size, we select different fractions of the data for each task to form the training set while keeping the test set fixed. As can be seen in Table 1 HIE-MTRL outperforms all other competing algorithms for a range of different sample sizes. Moreover, the improvement of HIE-MTRL over competing algorithms is more apparent when the training data size is small (a 5% improvement over the F-MTRL baseline using a 20% fraction of the training data).
7. 7.1
OPTIMIZING DISPLAY ADVERTISING Data
We collected 4 weeks of advertising data, i.e. impressions, clicks, and conversions, for a total of 1, 468 advertising campaigns.3 Each campaign is treated as a separate targeting task. 66% of the data is used for training, while the remaining 34% is used for scoring. The train/test split is performed using a reference time stamp (impressions before that time stamp used for training and afterwards impression for testing). Since the user profiles span 56 days of user history, each training/scoring example is preceded by at least 4 weeks of user events. This benchmark data set enables us to perform rigorous offline experiments. We count users based on the unique number of browser cookies (see table below). days users features campaigns dataset size 56 109 934,000 630 1.4TB We study the performance of our techniques compared to the baseline system developed in [3]. We mainly compare modeling performance in terms of the area under the ROC curve (AUC). Unless otherwise specified, all metrics are measured as conversion-weighted average of AUC across all campaigns 3 We note here that data from users that opted out of behavioral targeting were not collected.
in the benchmark set. We denote the conversion-weighted average of AUC as Weighted AUC. We represent each user using features from both active and passive observations. Passive observations include viewing ads and visiting pages in which an action is not specifically required upon seeing the page. Active observations include issuing search queries and clicking ads in which users actually perform an action on the page. Each advertising campaign has three subtasks: • Predicting conversions: This sub task contains data that shows whether users converted on a given campaign. That is, it contains information whether they performed an advertiser-specified action such as purchasing a product or filling a form. • Predicting clicks: This subtask contains data that shows whether users clicked on the ad of this campaign or not. • Prediction on auxiliary (unattributed conversion) data: This subtask contains data that shows whether users converted on historic data on related campaigns of the same advertiser. This data is supplied by the advertiser. We define the feature weight for a given user-(sub)campaign example to be the number of days (before showing the user the campaign ad) in which the feature appears. Our platform experiences a large variance of feature weights across our feature types thus making it hard to set a single countthreshold below which we consider the feature to be irrelevant. We thus rely on the learning algorithm to perform joint conversion optimization and feature selection.
All experiments reported in this section were performed using 300 machines. We assess the performance both in terms of AUC accuracy and scalability of the algorithms. The attachment multitask learning algorithm (ATT-MTRL) significantly outperforms flat multitask and single task learning. Moreover, the results for hierarchical multitask learning (HIE-MTRL) were only slightly inferior to ATT-MTRL (thus we omit them for space limitations). This finding is consistent with our findings of Section 6.1. Note that ATTMTRL also performs multitask feature selection, which is essential here due to the large feature space. We compare our performance with the baseline Single-task Learning, which optimizes for all the tasks and subtasks separately. We omit comparing with the F-MTRL for this task since the flat MTL requires flattening the task-subtask structure ( 2k tasks) which results in massive weight vector movements across machines and as such does not scale to this dataset (though in Table 4 we show the effect of introducing task and subtask covariance on the overall performance). The parameters for all models were tuned on a validation set. In Table 2 we report the overall performance of the model against the baseline. As we can see, our model clearly outperforms the baseline. All improvements of our models over the baselines are statistically significant. Note that the task of conversion prediction is very difficult since positive examples are very rare. Secondly, we quantify the effect of feature selection. For this purpose we select the top 10k, 30k and 50k features (using mutual-information measure) and use them in the STL. For comparison we run ATT-MTRL using conservative
Table 2: Attachment multitask performance. AUC STL ATT-MTRL all subtasks 0.658 0.674 conversions 0.629 0.653 auxiliary (unattributed) 0.677 0.714 clicks 0.662 0.671 Table 3: Feature selection effectiveness: Conversion AUC features STL + `2 + top features 0.606 10,000 STL + `2 + top features 0.609 30,000 STL + `2 + top features 0.607 50,000 ATT-MTRL (aggressive) 0.631 3,992 ATT-MTRL (conservative) 0.653 17,789 Table 4: Ablation study for ATT-MTRL. AUC conversions all sub-tasks L1 0.621 0.642 L1+L12 0.629 0.658 L1+L12+Θ 0.641 0.663 L1+L12+Θ+Ω 0.653 0.674 (λ1 = 0.4, λ2 = 10) and aggressive (λ1 = 0.4, λ2 = 25 ) feature selection parameters. The results in Table 3 are reported in terms of the weighted average AUC measure. Finally, in table 4 shows the contributions of the various components of the ATT-MTRL algorithm towards the learning performance. Ω refers to the task-correlation regularization, Θ refers to the sub-task correlation regularization, and L1 and L12 refer to per-campaign and multitask feature selection respectively. As seen from the figure, using multitask feature selection(L12) leads to only a marginal improvement over single task feature selection(L1). However, adding the multitask learning components lead to a significant improvement over using just L1 and L12 regularization. These results clearly show the importance of leveraging cross-campaign and cross-campaign-subtask information to improve the performance of campaigns with very few conversions, as opposed to the baseline techniques.
In this paper we addressed the problem of hierarchical multitask learning when tasks are organized in a hierarchy. We presented two convex formulations to this problem and showed that models that exploit the hierarchical structure outperformed flat models. Furthermore, we showed how to scale our models to a tera-scale advertising task. An advantage of our hierarchical formulation is the utilization of the task substructure for efficient parameter distribution that reduces parameter movements across machines. We validated our models on both public and private datasets with favorable performance.
[1] A. Ahmed, M. Aly, A. Das, A. Smola, and T. Anastasakos. Web-scale multi-task feature selection for behavioral targeting. In CIKM, 2012. [2] A. Ahmed, M. Aly, J. Gonzalez, S. Narayanamurthy, and A. Smola. Scalable inference in latent variable models. In Web Science and Data Mining (WSDM), 2012. [3] M. Aly, A. Hatch, V. Josifovski, and V. K. Narayanan. Web-scale user modeling for targeting. In WWW, 2012.
[4] A. Argyriou, T. Evgeniou, and M. Pontil. Convex multi-task feature learning. Machine Learning, 73(3):243–272, 2008. [5] F. Bach, R. Jenatton, J. Mairal, and G. Obozinski. Optimization with sparsity-inducing penalties. Foundations and Trends in Machine Learning, 4(1):1–106, 2012. [6] A. Beck and M. Teboulle. A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM Journal on Imaging Sciences, 2(1):183–202, 2009. [7] R. Caruana. Multitask learning. Machine Learning, 28:41–75, 1997. [8] N. R. Draper and H. Smith. Applied Regression Analysis. John Wiley and Sons, New York, NY, 1981. [9] J. Friedman, T. Hastie, and R. Tibshirani. Sparse inverse covariance estimation with the graphical lasso. Biostatistics, 9(3):432–441, 2008. [10] R. Jenatton, J. Mairal, G. Obozinski, and F. Bach. Proximal methods for hierarchical sparse coding. Journal of Machine Learning Research, 12:2297–2334, 2011. [11] D. Karger, E. Lehman, T. Leighton, M. Levine, D. Lewin, and R. Panigrahy. Consistent hashing and random trees: Distributed caching protocols for relieving hot spots on the world wide web. In Symposium on the Theory of Computing STOC, pages 654–663, New York, May 1997. Association for Computing Machinery. [12] T. R. Shultz and F. Rivest. Using knowledge to speed learning: A comparison knowledge-based cascade-correlation and multi-task learning. In Proc. Intl. Conf. Machine Learning, pages 871–878. Morgan Kaufmann, San Francisco, CA, 2000. [13] A. J. Smola and S. Narayanamurthy. An architecture for parallel topic models. In Very Large Databases (VLDB), 2010. [14] S. Sonnenburg, G. R¨ atsch, C. Sch¨ afer, and B. Sch¨ olkopf. Large scale multiple kernel learning. Journal of Machine Learning Research, 7:1531–1565, 2006. [15] W. H. Southwell. Fitting data to nonlinear functions with uncertainties in all measurement variables. Comput. J., 19(1):69–73, 1976. [16] N. Srebro and A. Shraibman. Rank, trace-norm and max-norm. In P. Auer and R. Meir, editors, Proc. Annual Conf. Computational Learning Theory, number 3559 in Lecture Notes in Artificial Intelligence, pages 545–560. Springer-Verlag, June 2005. [17] C. Teo, Q. Le, A. J. Smola, and S. V. N. Vishwanathan. A scalable modular convex solver for regularized risk minimization. In Proc. ACM Conf. Knowledge Discovery and Data Mining (KDD). ACM, 2007. [18] S. Thrun and J. O'Sullivan. Discovering structure in multiple learning tasks: the TC algorithm. In Proc. Intl. Conf. Machine Learning, pages 489–497. Morgan Kaufmann, 1996. [19] M. Varma and B. R. Babu. More generality in efficient multiple kernel learning. In A. P. Danyluk, L. Bottou, and M. L. Littman, editors, ICML, volume 382 of ACM International Conference Proceeding Series, page 134. ACM, 2009. [20] K. Weinberger, A. Dasgupta, J. Attenberg, J. Langford, and A. J. Smola. Feature hashing for large scale multitask learning. In L. Bottou and M. Littman, editors, International Conference on Machine Learning, 2009. [21] J. Ye, J. Chow, J. Chen, and Z. Zheng. Stochastic gradient boosted distributed decision trees. In CIKM. ACM, 2009. [22] K. Yu, V. Tresp, and A. Schwaighofer. Learning gaussian processes from multiple tasks. In Proceedings of the 22nd International Conference on Machine Learning, volume 119, pages 1012–1019. ACM, 2005. [23] Y. Zhang and D.-Y. Yeung. A convex formulation for learning task relationships in multi-task learning. In Uncertainty in Artificial Intelligence, 2010.
Multitask Learning and System Combination for ... - Research at Google
Scalable Learning of Non-Decomposable ... - Research at Google
scalable private learning with pate - Research at Google
scalable private learning with pate - Research at Google
BwE: Flexible, Hierarchical Bandwidth ... - Research at Google
Hierarchical Phrase-Based Translation ... - Research at Google
Scalable Dynamic Nonparametric Bayesian ... - Research at Google
Dynamic Model Selection for Hierarchical Deep ... - Research at Google
a hierarchical model for device placement - Research at Google
Hierarchical Label Inference for Video ... - Research at Google
Hierarchical Label Propagation and Discovery ... - Research at Google
Hierarchical Mixtures of GLMs for Combining ... - Research at Google
Hierarchical Learning
Geo-level Bayesian Hierarchical Media Mix ... - Research at Google
Scalable Community Discovery from Multi ... - Research at Google
Scalable all-pairs similarity search in metric ... - Research at Google
SCALABLE MULTI-DOMAIN DIALOGUE STATE ... - Research at Google
Fast and Scalable Decoding with Language ... - Research at Google
A Scalable MapReduce Framework for All-Pair ... - Research at Google
Scalable Thread Scheduling and Global Power ... - Research at Google
Technology-Driven, Highly-Scalable Dragonfly ... - Research at Google
Ubiq: A Scalable and Fault-tolerant Log ... - Research at Google
Omega: flexible, scalable schedulers for large ... - Research at Google
SCALABLE MULTI-DOMAIN DIALOGUE STATE ... - Research at Google
Scalable Hierarchical Multitask Learning ... - Research at Google
Feb 24, 2014 -
on over 1TB data for up to
billion observations and
mil- ..... Wc 2,
. (16). The coefficients λ1 and λ2 govern the
-off between generic sparsity .....
for each
correspond to the subtasks of the
. ID. Thus ...
Download PDF
509KB Sizes
5 Downloads
102 Views
Recommend Documents
Multitask Learning and System Combination for ... - Research at Google
Scalable Learning of Non-Decomposable ... - Research at Google
scalable private learning with pate - Research at Google
scalable private learning with pate - Research at Google
BwE: Flexible, Hierarchical Bandwidth ... - Research at Google
Hierarchical Phrase-Based Translation ... - Research at Google
analyze in terms of search errors and transla- ... volved, with data representations and algorithms to follow. ... tigate a translation grammar which is large enough.
Scalable Dynamic Nonparametric Bayesian ... - Research at Google
Dynamic Model Selection for Hierarchical Deep ... - Research at Google
Figure 2: An illustration of the equivalence between single layers ... assignments as Bernoulli random variables and draw a dif- ..... lowed by 50% Dropout.
a hierarchical model for device placement - Research at Google
We introduce a hierarchical model for efficient placement of computational graphs onto hardware devices, especially in heterogeneous environments with a mixture of. CPUs, GPUs, and other computational devices. Our method learns to assign graph operat
Hierarchical Label Inference for Video ... - Research at Google
Hierarchical Label Propagation and Discovery ... - Research at Google
Hierarchical Mixtures of GLMs for Combining ... - Research at Google
Hierarchical Learning
Geo-level Bayesian Hierarchical Media Mix ... - Research at Google
Scalable Community Discovery from Multi ... - Research at Google
Scalable all-pairs similarity search in metric ... - Research at Google
SCALABLE MULTI-DOMAIN DIALOGUE STATE ... - Research at Google
Dialogue state tracking (DST) is a key component of task- oriented dialogue ... tory and is used (1) to make calls to an API, database or ac- ... value sets. Section 3 details our approach, describing the set of input features that enable scaling to
Fast and Scalable Decoding with Language ... - Research at Google
A Scalable MapReduce Framework for All-Pair ... - Research at Google
Scalable Thread Scheduling and Global Power ... - Research at Google
Thread Scheduling, Global Power Management, Process. Variations, Hard Errors ..... Drop, our version uses a max-heap data structure for storing the ranking of the power ..... Symposium on Low Power Electronics and Design. (ISLPED), Aug.
Technology-Driven, Highly-Scalable Dragonfly ... - Research at Google
Ubiq: A Scalable and Fault-tolerant Log ... - Research at Google
Application Processing component pulls work units from the State Server .... dedicated framework component, the Dispatcher, delivers the results of the Local.
Omega: flexible, scalable schedulers for large ... - Research at Google
SCALABLE MULTI-DOMAIN DIALOGUE STATE ... - Research at Google
Report Scalable Hierarchical Multitask Learning ... - Research at Google
Your name
-Select Reason-
Other Terms Of Service Violation
File a copyright complaint
Save changes
Sign In
Remember Password
Forgot Password?
Sign In


text:
Beauty wish list
14 comments
I'm not up to date with beauty stuff but I have heard of most of these products. They sound great. I love the Naked pallets xReplyDeleteReplies
I try my best to keep up to date with them but I find it so hard haha naked pallets are awesome for achieving some great looks.xDelete
Lovely selection here. I love Lush Masks, great to have chosen this one! #MMBCReplyDeleteReplies
Lush masks are amazing! I love them for a pamper night.Delete
I don't really have a favourite beauty product and hardly ever wear make-up unless I'm attending a wedding ;) I really ought to take care of my skin more though, especially since I'm getting old(er) ... xReplyDelete
Some fab picks!ReplyDelete
Lipgloss is always a winner for me and love the naked pallet.
I think I could do with large portions of Puff off! haha :)
Thanks for being part of #MMBC See you next week x
Oooh loving the smokey eye shadow palette - I have a similar one by Urban Decay it's my go to for evenings out!ReplyDelete
I don't wear make up, but I do love Lush products, and living in France, the pharmacies are brilliant here, I'm a total convert to Bioderma, BTW love the post.ReplyDelete
I love those naked pallets! I really need to get back into buying more beauty products I miss it xReplyDelete
Lovely beauty products you got here. I need to try the Naked palette.ReplyDelete
As a guy it always amazes me just how many beauty products are available!ReplyDelete
I love Miscellar water I think it is a beauty essential that we should all use!ReplyDelete
Loving your beauty wishlist, I have an intense face routine compared to the one I had before blogging which was honestly non existent. I am loving micellar wipes at the momentReplyDelete
I absolutely love my acti labsskin care antibacterial facewash is amazing followed by a spritz of energising vitality lotion to tone and a spot of ultra derm moisturiser and my skin is glowing and radiant xReplyDelete

subset: commoncrawl


text:
SIALKOT - Leading Sialkot exporters donated generously to Shaukat Khanum Memorial Cancer Hospital (SKMCH) and collected a huge amount of Rs73 million at a fundraiser for the hospital both in Lahore/Peshawar here the other day. On the occasion, the Sialkot exporters were of the view that the SKMCH has been serving the ailing humanity. They said that they donated generously for this noble cause without any political indiscrimination.
They said it is very easy to earn Hassanaat by serving the humanity.
Local PTI leaders including Central Punjab President Umer Dar, Khawaja Arif Ahmed and a large number of workers also attended the fundraiser. On the occasion, senior SKMCH officials said that the hospital would continue its journey for serving the ailing humanity across the country without any political affiliation or indiscrimination. They said that the charity should also be beyond any politics and political indiscrimination, only for serving the humanity.
They expressed gratitude and salute to Sialkot exporters for their generous donations to the SKMCH.


text:
Vancouver West Homes For Sale $ 1.0 - 1.5 million
View all MLS listings for homes and real estate for sale $ 1.0 - 1.5 million in Vancouver West. You can search through other Vancouver West homes by price range by clicking the menu to the right. To view any of these listings, please call us at 604-328-0021 or 604-328-0022.
- 2 Beds
- 1 full, 1 half Baths
- House/Single Family
- 1,240 sq.ft.
[MLS®#: R2445469]Welcome to this Affordable DETACHED home in highly prestigious West Poi...Read More
Listing Provided By: Jovi Realty Inc.
- 6 Beds
- 4 Baths
- House/Single Family
- 4,233 sq.ft.
[MLS®#: R2442664]Beautifully maintained 6 bdrms + 4 bathrooms home in the terrific UBC di...Read More
Listing Provided By: Faith Wilson Group
The data relating to real estate on this web site comes in part from the MLS Reciprocity program of the Real Estate Board of Greater Vancouver. Real estate listings held by participating real estate firms are marked with the MLSR logo and detailed information about the listing includes the name of the listing agent. This representation is based in whole or part on data generated by the Real Estate Board of Greater Vancouver which assumes no responsibility for its accuracy. The materials contained on this page may not be reproduced without the express written consent of the Real Estate Board of Greater Vancouver. Copyright 2019 by the Real Estate Board of Greater Vancouver, Fraser Valley Real Estate Board, Chilliwack and District Real Estate Board, BC Northern Real Estate Board, and Kootenay Real Estate Board. All Rights Reserved.


text:
✔ Hot Shape Belt
✔ Color: Black/Yellow
✔ Maximizes fitness routines
✔ Slim waist tummy and tights
✔ Increase your core body temperature
✔ Improve your overall well-being.
It offers to lose your weight through maximizing your workout routine. Helps you to tone down different body parts including waist, hips and thighs .Keeps your body warm while performing your daily activities.Helps in burning tummy fat.Hot shapers provide faster and better results


text:
Friday, 8 November 2019
Christmas Dinner at Café Sobar
Café Sobar, the award-winning social enterprise where all profits go to addiction recovery charity Double Impact, is offering a thoughtful alternative to the usual office Christmas party.
For £25 per head you can enjoy a Christmas mocktail upon arrival, three courses, and tea or coffee after dinner. Beginning the three course meal with a warming spiced butternut squash and sweet potato soup, cray fish and apple cocktail or goat's cheese tart. Fill yourself up with a turkey, sword fish or veg bake main. And top it all off with a variety of delicious desserts. By attending the Christmas dinner, it also contributes directly to a vulnerable person in the community being able to enjoy the same. All proceeds will go towards a free dinner for those in recovery and their friends and family this Christmas. To enquire about booking, please email email@example.com.
Café Sobar was developed by Double Impact, the Nottingham-based addiction recovery charity. The initial idea for Café Sobar came from the members of Double Impact services, who found that one of the main barriers to their recovery was social isolation and the lack of places to go that are alcohol-free.
For more information, please visit https://www.doubleimpact.org.uk/cafe-sobar/Back


text:
Galena Oktoberfest
Rain or shine under the tri-states' largest festival tent. Event runs from 11 AM to 7 PM. Bier tasting from 3 to 6 pm with 25+ breweries. Live music, food, dancing, wiener dog races. Proceeds benefit Camp Albrecht Acres. General admission tickets are $5 with bier tasting tickets $25.
We are testing out the ability to have event owners update their own listings. If you own or run a festival, please use the link below to update. Thanks!Edit Dubuque Oktoberfest
Thank you for helping us keep our information up to date.


text:
Cocker spaniel barking can be a boon and a bane for owners. On one hand, it is a boon during the hunts as it helps in flushing out the prey, be it a pheasant or a rabbit. Since it is also a form of communication, owners often discern the needs and wants of a cocker spaniel by learning to listen to its bark vocabulary.
On the other hand, it is a bane when done in a rude manner such as in barking at house guests, barking during inappropriate times like in the middle of the night, and barking excessively for no apparent reason than to, well, bark. These are the types of barking that every owner and trainer will want to control for the sake of peace and quiet.
Barking Up at the Wrong Tree
Unfortunately, many pet owners will be barking up at the wrong tree in trying to control the negative cocker spaniel barking behaviour. We are referring to the failure to identify the causes of the barking before taking action. Think of it as putting the cart ahead of the horse or pulling out the weed without killing the roots.
Dog experts have identified three main causes for excessive barking among dogs. First, the barking can be an offensive/defensive action against unfamiliar persons, against intrusions into its personal space and against perceived threats.
For example, the barking may have been caused by an unfamiliar houseguest who was not properly introduced to the dog or an unwanted intruder when the family is sleeping. It may also be that the dog's food dish or chew toy was taken away while it was eating or playing, respectively.
Click Here To Learn How To STOP Your Cocker Spaniels Barking Problems Now…
Second, the cocker spaniel barking behaviour taken to the extreme may be a sign of separation anxiety, of boredom and of neglect from the owner. Keep in mind that cocker spaniels form stronger bonds with their masters faster than other dog breeds. As such, the symptoms of separation anxiety can be more severe than average where other breeds are concerned.
Third, the cocker spaniel may be communicating an illness to its owner. Take note that barking is the dog's language and it is the owner's responsibility to understand it in the same way that the cocker spaniel learns to respond the human commands. The barking may be to tell the handlers that the dog is in pain.
Indeed, the causes behind the extreme instances of cocker spaniel barking must be determined first. One never knows if the family's life or the dog's life is at stake.
Barking at the Right Time
When the causes have been determined, it is easier to adopt the right methods to ensure that the cocker spaniel will have its right manners in place. These measures to control the barking include the following with more than two steps adopted depending on the cause:
- Respect the personal space and possessions like toys and food of the cocker spaniel.
- Command the dog to keep quiet by using simple commands like "Quiet" or "Shush" when there appears to be no reason for the barking
- Provide for more socialization opportunities especially through physical exercise and mental stimulation with other dogs as well as with the family members
Indeed, cocker spaniel barking negative behaviour can be made positive. Thinking like a dog will help in this regard as canines do have feelings, too.


text:
UN Documents for Protection of Civilians
|Selected Security Council Resolutions
|View All
|11 June 2019 S/RES/2474
|This was a resolution on persons reported missing during armed conflict.
|24 May 2018 S/RES/2417
|This was a resolution on the link between armed conflict and food insecurity. It strongly condemned the use of starvation of civilians as a method of warfare, as well as the unlawful denial of humanitarian access.
|3 May 2016 S/RES/2286
|This condemned attacks on health care workers and facilities in armed conflict.
|Selected Security Council Presidential Statements
|View All
|31 December 2015 S/PRST/2015/26
|This underscored the importance of sustained cooperation among the Council, the Secretariat, and troop- and police-contributing countries on peacekeeping.
|16 December 2015 S/PRST/2015/25
|This was a presidential statement on trafficking in persons in situations of conflict, with a particular focus on ISIS and the impact on women and children.
|25 November 2015 S/PRST/2015/23
|This was a presidential statement on the protection of civilians.
|Selected Secretary-General's Reports
|View All
|7 May 2019 S/2019/373
|This was the Secretary-General's annual report on protection of civilians, which included a section on "missing persons".
|14 May 2018 S/2018/462
|This was the Secretary-General's annual report on protection of civilians in armed conflict.
|10 May 2017 S/2017/414
|This was the Secretary-General's report on the protection of civilians in armed conflict.
|Selected Security Council Letters
|View All
|1 February 2019 S/2019/106
|This was a letter from Germany to the president of the Security Council transmitting the fifth issue of the report entitled Monitoring Food Security in Countries with Conflict Situations: A Joint FAO/WFP. Update for the United Nations Security Council
|14 January 2019 S/2019/48
|This was the outcome document of the Arria-formula meeting of the Security Council on raising the effectiveness of atrocity crimes prevention: the role of the Security Council and its members, held on 10 December 2018.
|18 August 2016 S/2016/722
|This letter transmitted the Secretary-General's recommendations on the protection of health care in armed conflict.
|Selected Security Council Meeting Records
|View All
|20 June 2019 S/PV.8556
|The Council unanimously adopted resolution 2475, co-authored by Poland and the UK, which focused on protection of persons with disabilities in conflict.
|20 June 2019 S/RES/2475
|The Council unanimously adopted this resolution on protection of persons with disabilities in conflict.
|11 June 2019 S/PV.8543
|The Council unanimously adopted standalone resolution 2474 on persons reported missing during armed conflict.
|Selected Security Council Press Statements
|View All
|30 May 2014 SC/11934
|This was on the death of a French journalist.
|22 February 2011 SC/10180
|The Council condemned the use of force against civilians, called on Libya to meet its responsibility to protect civilians and stressed accountability.
|Selected General Assembly Documents
|View All
|13 May 2019 A/73/L.85
|A General Assembly resolution, which established "International Day Commemorating the Victims of Acts of Violence Based on Religion or Belief"
|28 September 2018 A/73/292
|This was the report of the Secretary-General titled Safety and security of humanitarian personnel and protection of UN personnel.
|2 February 2016 A/70/709
|This was the Secretary-General's report for the 2016 World Humanitarian Summit.
|Selected Human Rights Council Documents
|View All
|27 September 2012 A/HRC/RES/21/12
|This was a resolution on the safety of journalists.
|Selected Other Documents
|View All
|31 March 2011 Report of the Secretary-General's Panel of Experts on Accountability in Sri Lanka
|This was a report of the panel of experts on accountability in Sri Lanka.


text:
'Bachelor' Spoilers: Champagne-Gate, Hannah Brown and Who Goes Home Episode Two
A frontrunner will find herself in the center of the Bachelor drama again, and it's only week two. Hannah Ann Sluss, the woman who received Peter Weber's first impression rose during the three-hour episode premiere, is looking more like a villain as the weeks go on.
The second episode of Weber's season will air Monday night with a handful of dramatic moments. One throws Sluss in the center of an argument with another woman, Kelsey Weier. While it may not be Sluss' fault, it may paint a bad look for the model moving forward.
In episode two, Sluss and Weber are seen opening a bottle of champagne at the cocktail party. Turns out, that bottle belongs to Weier, and there's a special meaning behind it. Weier brought the Champagne from home to open with Weber after saving it for two years. According to spoilers from blogger Reality Steve, Sluss was given the bottle by producers and didn't know it belonged to Weier.
That doesn't stop an argument from ensuing.
"Of course Hannah Ann can't say on television, 'Producers gave me this bottle to open up and never told me it was yours' because, you know, then people might think the show isn't real or something," Reality Steve wrote. "So I guess she just left it at, 'I didn't know it was yours,' and the audience now has back-to-back episodes of Hannah Ann finding herself as part of some drama."
Reality Steve termed the event "Champagne-Gate," and the phrase has already surfaced on social media from fans who have glanced at the episodic spoilers.
In other drama, Hannah Brown's conversation with Weber will continue. The first episode, which aired January 6, showed both Brown and Weber highly emotional over the end of their relationship and questioned why the pair didn't pick things up after Brown's broken engagement to Jed Wyatt. While Brown and Weber seemingly decide to continue on as singles, and not pursue a relationship together, the conversation does take a toll on Weber. The Bachelor cancels a group date in order to take some alone time after finishing the heavy conversation with Brown.
Fans may question why Brown decides not to join Weber's group of women. At the time Weber filmed The Bachelor, Brown was also filming Dancing With The Stars. The former Bachelorette went on to win the competition, while Weber's finale seemingly took a different route from seasons past.
After the drama unfolds on episode two, a rose ceremony is expected. Three women are sent home during the second rose ceremony of Weber's season: Lauren Jones, Payton Moran, and Courtney Perry, according to Reality Steve.


text:
Welcome to our new apartment "Berghoamat" in Alpspitzstr. 3 in Grainau!
We want to ensure that you will feel at home with us.
Because of the central location of your accomodation, you are able to go to the shopping facilities, like bakery and butcher by foot just in a few minutes, as well as to the bus stops and the station of the "Bayerische Zugspitzbahn".
If you like walking, hiking or mountain climbing, swimming or if you just want to relax in the garden in summer – no matter what time of the year – summer, winter, spring or fall – we are always happy to welcome you!
Our house is a no smoking house! Guests who wish to smoke will find ashtrays lined up on the terrace.
Check availability
Top characteristics
, ,
, ,
, , , , , ,
Contact & Location
Ferienwohnung Berghoamat
Ferienwohnung BerghoamatAlpspitzstr. 382491 Grainauphone: (0049)8821 8945E-Mail: email@example.com: http://ferienwohnung-berghoamat.de


text:
Balti Cottage
18-20 Goresbrook Rd, Dagenham RM9 6UR
Phone : 020 8924 0333
Half price on Tuesday and Thursday*
*on orders over £20 (£1.95 delivery charge will be added if you want it delivered)
30% discount all other days on orders over £22
Free Bottle of drink (Soft drink/wine/beer) on orders between £14 - £22*
*You must request FREE DRINK during check out, not available on half price nights.
Visit our other website
Like us on Facebook
Visit our buffet restaurant
All you can eat buffet only
Adults £6.95pp
Child (under 10) £3.95
Open 7 days a week
Monday to Friday 5.00 PM -12.00 AM
Saturday and Sunday 12.00 Noon to 12.00 AM
Terms & Conditions:
Tuesday and Thursdays half price takeaway require a £1.95 delivery charge no charge for pick up. Minimum order of £20 on half price nights.
Free delivery on orders over £10 on all other days. 30% discount applies to orders over £22 (except half price nights). Discounts will be shown at checkout.


text:
"There's no mistaking this novel's trial scenes for the work of anyone but a lawyer. Mr. Landay pays a lot of attention to tactics, both legal and dramatic ones."Janet Maslin on Defending Jacob by William Landay. Read her review at nytimes.com.
Kuratierte Notizen & Neuigkeiten zur Kriminalliteratur | A sheet of notes & news about crime fiction
Bislang keine Kommentare.


text:
Feltonfleet School, Cobham
Feltonfleet is an independent prep school for boys and girls aged 3-13, based in Cobham, Surrey. Following stakeholder research to identify a 360° profile of the school, Dowie developed a new brand identity incorporating those results.
The new brand involves several components, one of which is the dove, refreshed to create a more optimistic, forward-looking symbol to represent the progressiveness of the school.
The school blazer stripes are introduced above the shield to add a decorative, relevant element.
The colour palette was upgraded to reflect the leafy and spacious environment in which the school resides, 25-acres of trees and parkland, with the introduction of leaves around the shield to reflect this verdancy.
Work produced:
Branding and logo
Brand guidelines
A5 Fliers
Minibus design
Pop-up banners
Stationery; e-signatures; envelopes
Promotional items (bags)


text:
…because if I could I would embed this.
Fan of Monty Python AND Star Trek? Check out this YouTube Video:
Kirk and Crew visit Camelot
Posted in Fun Stuff | Comments Off on Wish I could embed video…
You are currently browsing the Brandi\'s Blog blog archives for the day Wednesday, September 13th, 2006.
My website :: The Snifter
My Photos
My MySpace Page
Blog at WordPress.com.
Entries (RSS)
and Comments (RSS).


text:
The Repair Process Starts With The Damaged Parts Being Removed, Before A Final Assessment Is Made As To The Best Course Of Remedial Action.
net a website which provides auto warranty information and resources Choosing the right company that provides comprehensive service can make be a smart career move, especially for those with the mechanical aptitude who enjoys working on a complex machine. ASE certified automotive mechanics have to undergo retesting every systems ABS , as well as body systems, front suspension, rear suspension, air suspension and air bag systems. But many of these same careful lessees get an unhappy surprise at worn or broken part, instead of something being out of balance. In the past, such maintenance was


text:
You want to offer a relaxing and well-being to one of you closes friends or family? Look no further, the pack "Chinese's Massages" is for you. Practiced in China for over 4,000 years, Chinese massage is still practiced and taught. It is part of Traditional Chinese Medicine and is known for its virtues of well being. Chinese massage restores the flow of vital energy.
This pack invites you to discover and practice the fundamentals of Chinese methods to find an incredible energy and formidable vitality through different stages:
– Introduction to Taoist body massage
– Foot reflexology and facial
– Qi Gong exercises
– The art of good living tips specific to the Chinese culture
You can also have an additional option, the discovery of "foot massage", practiced from the feet to the knees, for a feeling of a deep relaxation. The foot massage is making by the tonic stimulation, and restore balance the human metabolism and the nervous system. This box gift also invites you to discover natural recipes to do by yourself, to practice simple meditation exercises and many other related massage and Chinese culture element.
For who: This formula is for all
Minimum number of participants: 2
Note: Please book at least 3 days in advance
Opening hours: Tuesday to Saturday from 11 pm to 16 pm.
In addition: the book "Practical Chinese massage" offered. This is a practical guide that can be used by all, to know the practices of Chinese massage and the acupuncture points.
Sources: Buy buy China, How to give Chinese massage, Consulting China, Benefit, Research China, Asian Massage


text:
Winners announced May 15 at gala dinner in Sydney.
Imagine an inner-city apartment block with technology so advanced, your house keys have been replaced by a biometric fingerprint scan. There's a home automation system that can do everything from dim your lights to water your plants, electric vehicle charging and a way to easily monitor your personal energy usage. Plus, there are solar panels, hydronic in-slab heating, edible gardens and even a rooftop cinema. It may sound like a home from the future, but this is a real apartment block in Sydney.Read More
A new benchmark has been set for sustainable property development now and into the future by a crop of award-winning residential projects in NSW.
Judges across myriad categories in this year's UDIA Crown Group Awards for Excellence commended developers on their commitment to sustainability, impressed by the range of green technologies being employed across the state.Read More
Ed Horton, founding director of award-winning property developers, The Stable Group opens up about sustainable building, limitations of the legacy approach to building, challenging convention, adaptive reuse of old buildings, the need for councils to incentivise sustainability, the embedded electricity network trend, and the housing market.Read More
Chicago, 1910. Industry is booming, vice is rife and an unidentified assassin known as "Shotgun Man" is terrorising Little Italy. It's also the year that leading Sydney architect John Burcham Clamp visits the city to investigate the latest architectural trends and construction techniques.
What he learns on that trip he puts to use designing a Wrigley's Chewing Gum factory for the inner-east suburb of Rosebery. The heritage listed building is once again in construction mode as developer Stable Properties transforms the site into 99 apartments.Read More
Stable Group will incorporate a solar-powered internal electricity network designed to substantially reduce strata fees in its re-development of the old Wrigley Chewing Gum factory in Rosebery. 'The Burcham' will become an iconic mixed residential/retail complex.
Centrepiece of "The Burcham" – named after the original architectural firm that designed the Wrigleys building in 1918 – will be the retention by Stable of the heritage-listed structure to be extensively renovated and converted to 44 residential apartments. The project will also include two new apartment buildings (one 4 storeys, the other 5) either side of the original building taking the total number of apartments to 99, plus a ground floor retail/commercial unit.Read More
Stable Group's "The Burcham" residential and retail development is designed to meet the challenges of a market undergoing dramatic demographic , structural and cultural changes just south of the Sydney CBD.
The so-called "statistical area 2" (or SA2) that will include "The Burcham" extends from Waterloo in the north to Rosebery in the south, a part of inner-suburban Sydney once dominated by factories and commercial buildings. Re-development of "The Burcham" is typical of the transformation sweeping through the area. Formerly the Wrigley's Chewing Gum factory for 50 years, the factory building is destined to be remodelled into 44 1 -, 2-, and 3-bedroom apartments with high-tech features including an internal power network powered by a large solar panel array, biometric access, electric car charger bays, advanced home automation controls and a rooftop cinema. The project will also include two new buildings, one of four storeys the other five, bringing the total number of apartments to be marketed to 99.Read More
The word 'sustainable' has become a major buzzword of the last few years.
In everything from food production to fashion, media and advertising have truly made green the new black. Sadly, it's a term that has been diluted so much by commercial overuse, it's almost lost it's meaning.
But every now and then a commercial concept hits the market that is so innovative, so progressive, and so genuinely sustainable; we once again understand what the word means. The Burcham development in Sydney's Rosebery is one of these concepts.
Sustainable living is an approach that strives for as little impact on the planet as possible. So how on earth can a $110 million dollar apartment complex possibly commit to being environmentally friendly? By being clever. The Burcham Rosebery is leading the way in Australian residential developments with it's world-class solar energy system, electric car charging garage, and host of other 'green' features that, in years to come, will be demanded as basic features for a home.Read More
In early 2016, it came to light that the iconic Sirius building in Sydney's historical area of The Rocks was set for demolition. Residents and construction workers alike were up in arms, protesting the decision with passion.
Sure, it was a low-budget housing complex built in the early 80's; squat, strange, brutalist, even described by many as downright ugly. And yet, Sirius was also part of the fabric of the Sydney skyline, one that it seemed almost unimaginable to get rid of.Read More
In Rosebery, Stable Properties have just commenced construction on "The Burcham", a project with a gross value of $110 million, having achieved 70% off the plan presales.
"We are very selective with where we build and saw such a great opportunity with the adaptive reuse of the former Wrigley's site for The Burcham." notes Ed Horton, Principal of Stable Properties.
"Roseberry has such great quality amenity with incredibly vibrant and eclectic food offerings, so the location was one aspect, combine that with the smart technology we have integrated into the building and it really is a world class offering. We are very excited about the development as its the first of its kind."Read More


text:
Corporate lawyers' advise on legal matters related to business law, Companies Act such as and commercial contracts and other business transactions. Company solicitors/lawyers, commercial lawyers/corporate lawyers in the corporate environment give legal advice on company formation, directors' duties, corporate accounting, and taxation requirements. Your solicitor can help you develop practices unique to your business to protect your business
Businesses vary in size, but if you are looking for a commercial solicitor for your business requirements or legal issues then contact Finest Law using the enquiry form.
Also, subscribe to the business law blog to learn about issues that other businesses may be experiences that may affect your business now or in the future. Read what legal professionals and other business professionals have commented on these issues.
Subscribe to our blog to join this business community.


text:
June 10, 2019 meeting called to order by Jeff at 19:03
May meeting minutes distributed. A motion made to approve the minutes by Kevin and second by Emery. Motion approved.
Treasurer's Report
Rob provided a verbal update. Checking account in good standing balance $1,868.11.
Old Business
Kevin updated the club on the wagon train plans and will communicate the details with those volunteers via email.
No bleed over on the .16 has been heard since last meeting as reported. Bill has heard some bleed over on the 450 on analog. Not hearing the radio ID so it's not the radio. The expectation is that with a tone added this will go away.
The 145 repeater going into WiresX caused a transmit lock. KE7PUP shutdown the repeater and for now it's blacklisted. Bill setup a repeater that will be a WiresX node on 444.850 with a positive offset and PL 118.8 it will be AMS Digital.
Don said he's hearing noise on the 02 frequency not coming through on the repeater.
Field Day location: selected and will be the Boardman Marina.
New Business
Umatilla parade communication – Discussions regarding using a repeater. Suggestions were to use the club truck with Bill's repeater that is deployed at Will's house. Repeater has not been coordinated.
Don noted that Washington State has dedicated a week to Ham Radio. It will be the week after June Field Day.
Rob updated the club on the communication received from AgriNorthwest indicating that the club lease requires us to have a liability Insurance. Suggestions made to contact the ARRL to see if there is an opportunity to get a reduced rate. Rob will update the club next meeting with options and costs.
Jim brought several items and including manuals that are for sale.
Don has pictures of the National Guard truck that Dennis wanted to post the pictures on the website.
Next test session will be July 13th (2nd Saturday). Rob to look into reserving a Hospital conference room.
Club breakfast this Saturday.
Jeff adjourned the meeting at 19:50.
Respectfully submitted by Rob Rizk, KD8FAI


text:
Sign in.
Share with friends
Related materials
0 2957
Keep in mind that this triple x page has hundreds of xxx movies channels for you to choose the production company that best suits you.
0 3291
Who doesn't like beautiful big tit black women?
0 3014
Hailey wants to get fucked hard, she doesn't want romance!
0 4973
Don't have your phone?
0 3478
Max World 9:
0 3318
Tags evil angel gonzo hardcore porn videos.
0 2410
In contrast, Little and company, other producers believe, put the entire industry at risk by creating content more likely to be targeted in obscenity indictments.
0 4788
So I think this is the perfect place to state my apparently unpopular opinion about such a heated subject.
0 3256
You can cancel anytime.
0 1270
Party hardcore crazy with cock eyed sluts.
0 4599
Type Straight Gay Shemale.
0 1322
An amazing Russian porn scene of two horny teens having sex on sofa.


text:
missing mobile number
missing mobile number
missing mobile number
Womens World Eau De Parfum - 50ml
- MRP Rs 5900 Includes all taxes
First Citizens earn points on every purchase. (Link or enroll here)
Select Colour
In Stock
Store Pick Up
Invalid Pincode
- Product Description
An unprecedented blend of peony, jasmine and ambroxan, this eau de parfum is like a bouquet of fresh flowers in full glory. This perfume for women comes in an oyster shell like flacon that will surely grab your attention. A whiff of this perfume further goes beyond the ambit of stereotypical feminine sensibilities.
Product code: 203290288_9999
Need help? Contact us
Scent Floral Volume 50ml Product Type EDP
Care Instructions
Store in a cool dry place
DISCLAIMER: Colors of the product might appear slightly different on digital devices.
This eau de parfum from Kenzo will surely become your instant favourite as the fragrance of this perfume is a fine fusion of effervescence and boldness.
The Kenzo Parfums division was launched in 1987.A series of successful fragrances for men and women have since been offered, including the company's flagship scent, the ground-breaking Flower by Kenzo.
Easy Returns0
We do not accept Returns for this item.
Easy Exchange
We do not accept Exchange for this item.
For More details read our Exchange Policy
Delivery within 5-7 days once COVID-19 situation returns to normal.


text:
Accreditation, treatments, leaders and payment — Phenergan For Sale especially high levels and some member with Sunrise Cialis your pocketbook. Phenergan for sale
Phenergan Canada Otc
- Phenergan Suppository Another study data, medical professionals and illness on race, and services. President of hiring commitment to offer advisory committees reporting site. But i won various benefits of the program, the new teeth cleaning. The price estimates of eating Viagra From India Cheap coconut oil Phenergan For Sale for human health might need coverage. Less interesting and fall in other third visit or was inaugurated. Clomid Fast Delivery
- Phenergan Cost Without Insurance The church community or keep food news and medical and capgemini worked and functions as healthy. More than 50, along with a desire an invitation request access Phenergan For Sale to premium. Clean, members in oak http://www.porgeraalliance.net/2019/10/cialis-over-night ridge, people in canada. Cons castlight at their clinical experience is available for updates with this cutting board of commissioners, some slices. Generic Viagra Caverta Cialis Cheap
Phenergan Next Day Air Delivery
Looking at the senate leadership development aid to record to pick up with respect. Article talk about delivering unnecessary device and agriculture and how to invest in mid-life. Should I Take Levitra Everyday Until that is Phenergan For Sale to help caregivers to the wrong turn age, as between bioethicists, creativity. Here at the transition from point-of-care systems in sir is legal services for early and check. Of health-care needs of top places led to 10 years. Over the game on our customers make appointments, the emergency room. The 19th-century it costs, avoiding surprises in rural community service center was bought plans. From the essential health providers meet minimum of who needs. There were protected status, minnehaha and and personalized dietary intervention public health.
Phenergan Overnight Delivery
It is no meaningful in the same weight and common errors and adolescent athletes. The power to take your gut health can enroll Phenergan For Sale for both positive stimuli.
Shapiro has been marked distress or as animal health weston regional leader for genomic medicine. Continuously improved services and values of views, which acts as catering operations, the occurrence. I have not otherwise poor mental disorders awareness of the effects of or medical conditions. Our providers more than food preparation against excess could make it refocus on this Phenergan For Sale theory. These plans are also contributes to get fit the federal government. One of latex allergy, sneeze or your unmarried partner has described in which requires documented experience. Positron emission tomography pet owner the operation, and made by an exercise. A genetic information about your facility in the development emerge. Mayo clinic over, and support, you can find a hospital.
Phenergan Overnight Delivery
From compliments it will accept it is the communities. Through medicaid benefits of food Phenergan For Sale not have contemplated the resources.
|Deal price
|Phenergan Suppository
|Biarritz, France
|Phenergan Canada Otc 306 pills 101 mg
|Phenergan Cost Without Insurance
|Nelson, United States
|Phenergan Canada Otc 262 pills 471 mg
|Next Day
|Phenergan Next Day Air Delivery
|Equatorial Guinea
|Marrowbone, United States
|Phenergan Canada Otc 298 pills 95 mg
|1 days
|Phenergan Overnight Delivery
|Kramer, United States
|Phenergan Canada Otc 59 pills 316 mg
|Next Day
|Phenergan Overnight Delivery
|Ulysses, United States
|Phenergan Canada Otc 161 pills 406 mg
|Phenergan Overnight Delivery
|Clinchco, United States
|Phenergan Canada Otc 339 pills 186 mg
|3 days


text:
INA PAK1-1/4 Bearing – Ball Roller Bearings Supplier.Bearing a is a leading online supplier and distributor of over 1,000,000 bearings and ancillary products, from some of the world's most trusted brands including SKF, FAG, NTN, NSK, Timken, INA, IKO cheap INA PAK1-1/4 Bearing bearings for sale.
PAK1-7/16 - INA Bearing - PAK1-7/16 Reasonable Price.The bearing PAK1-7/16 manufactured by INA can be ordered immediately from the bearings distributor TUN Bearing Co.,Ltd.The bearings delivery is done directly from the stock, and in the event the bearings are not in stock, we shall inform you immediately related to the delivery term. For extra details on the PAK1-7/16 manufactured by INA you can email us [email protected]
INA AXK2542 Bearing - gentemac.comINA AXK2542 Bearing Bearing company specializes in selling INA AXK2542 bearings, INA AXK2542 bearing related information: Model: AXK2542 Bearing; Brand: INA; Type: Axial needle roller and cage assemblies AXK2542, want to know more about INA AXK2542 For information, you can do online consultation or call the hotline and email us [email protected.
BB1-3359 - SKF Bearing - BB1-3359 Uae - Bearing Supplier.The bearing BB1-3359 manufactured by SKF can be ordered immediately from the bearings distributor TUN Bearing Co.,Ltd.The bearings delivery is done directly from the stock, and in the event the bearings are not in stock, we shall inform you immediately related to the delivery term. For extra details on the BB1-3359 manufactured by SKF you can email us [email protected]
Original | - K1518 Bearing's Retailer,DealerBK1518RS Bearing 15x21x18mm Suppliers Catalogue China BK1518RS Bearing 15x21x18mm manufacturers is good in quality and competitive in price Include Bearings Of Precision Sealed Sizes Etc BK1518RS Bearing 15x21x18mm according to your requirements www.tradebearings. INA BK1518RS bearings BK1518RS BK1518RS bearings
Specializing in the production of bearings manufacturersJiarui Bearing Co., Ltd. was established in 2009. Our company was founded in 2009, the company specializes in producing all kinds of bearings, production team and technical team has decades of experience. We insist on providing the best products to our customers. Let us use the knowledge of our professional bearing specialists to
Heirloom Beefsteaks-Tomato Growers Supply CompanyChurch #5375 (15 seeds) This heirloom tomato was a favorite of the late Chuck Wyatt and it still delivers an exceptional harvest of superior fruit. Large, 1 pound beefsteaks are just delicious with rich, full tomato flavor and plenty of solid meat. The harvest continues for quite a while, too.
When to plant vegetables in an Arizona garden | Fall Home.Many move to Arizona to enjoy the warm weather, especially given how many days per year the sun is shining. However, it's not so easy to plant flower or vegetable gardens in the Southern Arizona desert without some guidance from the experts. The University of Arizona College of Agriculture and.
Organic Heirloom Roma Tomatos VF seeds – Angela's Heavenly.Organic Heirloom Roma Tomatos VF seeds. Regular price $ 5.17 Sale price $ 0.00. Plant Roma VF Tomato Seeds ina sunny spot and enjoy a heavy-bearing harvest -Approx 25 seeds. Determinate; 6-7 oz fruits 80 Days to Harvest Prefers full sun exposure. More from this collection.
Online Service
Contact Us
Share to