text:
THE SQL Server Blog Spot on the Web
This post is for SQL Server developers who have experienced the special kind of frustration, which only comes from spending hours trying to convince the query optimizer to generate a parallel execution plan. This situation often occurs when making an apparently innocuous change to the text of a moderately complex query; a change which somehow manages to turn a parallel plan that executes in ten seconds, into a five-minute serially-executing monster.
SQL Server provides a number of query and table hints that allow the experienced practitioner to take greater control over the final form of a query plan. These hints are usually seen as a tool of last resort, because they can make code harder to maintain, introduce extra dependencies, and may prevent the optimizer reacting to future changes in indexing or data distribution. One such query hint is the (over) popular OPTION (MAXDOP 1), which prevents the optimizer from considering plans that use parallelism. Sadly, there is currently no corresponding hint to force the optimizer to choose a parallel plan.
The result of all this is a great deal of wasted time trying increasingly obscure query syntax, until eventually the desired parallel plan is obtained, or the developer gives up in despair. Even where success is achieved, the price is often fragile code that risks reverting to serial execution any time indexing or statistics change. In any case, the resulting SQL is usually hard to read, and scary to maintain.
Whenever the query optimizer produces a serial plan instead of the 'obviously better' parallel plan, there is always a reason. Leaving aside the more obvious causes, such as the configuration setting max degree of parallelism being set to one, running under a Resource Governor workload group with MAX_DOP = 1, or having only one logical processor available to SQL Server, the usual causes of a serial plan are parallelism-inhibiting operations, cardinality estimation errors, costing model limitations, and code path issues.
There are many things that prevent parallelism, either because they make no sense in a parallel plan, or because the product just does not support them yet. Some of these force the whole plan to run serially, others require a 'serial zone' – a part of the plan that runs serially, even though other parts may use multiple threads concurrently.
That list changes from version to version, but for example these things make the whole plan serial on SQL Server 2008 R2 SP1:
Inconveniently, the list of intrinsic functions is quite long and does not seem to follow a pattern. ERROR_NUMBER and @@TRANCOUNT also force a serial plan, @@ERROR and @@NESTLEVEL do not. The T-SQL scalar function restriction is also a bit sneaky. Any reference to a table (or view) with a computed column that uses a T-SQL scalar function will result in a serial plan, even if the problematic column is not referenced in the query.
These query features are examples that require a serial zone in the plan:
The information presented above is based on the original list published by Craig Freedman, and updated for 2008 R2.
One way to check that a query does not have any parallelism-inhibiting components is to test the query using a CPU cost multiplier. This should only be done on a private test system where you are able to flush the whole plan cache after testing. The idea is to use an undocumented and unsupported DBCC command to temporarily increase the CPU cost of the query plan operators. It is not a fool-proof test (some rare parallelizable queries will not generate a parallel plan with this technique) but it is quite reliable:
-- Query to test
FROM Production.Product AS p
LEFT JOIN Production.TransactionHistory AS th ON
p.ProductID = th.ProductID
The final commands to reset the CPU weighting factor and flush the plan cache are very important.
If you get an parallel estimated plan for a particular test query, it shows that a parallel plan is at least possible. Varying the value passed to the DBCC command adjusts the multiplier applied to normal CPU costs, so you will likely see different plans for different values. The illustrated factor of a thousand is often enough to produce a parallel estimated plan, but you may need to experiment with higher values. It is not recommended to use estimated plans obtained using this technique directly in USE PLAN hints or plan guides because these are not plans the optimizer would ever produce naturally. To be clear, direct use of the plans would likely render a production system unsupported and the person responsible might be fired, shot, or possibly both.
If there is nothing that absolutely prevents parallelism in the target query, the optimizer may still choose a serial alternative if it has a lower estimated cost. For that reason, there are a couple of things we can do to promote the parallel option here, all based on the very sound notion of giving the optimizer accurate information to base its estimates on. The considerations here go well beyond just ensuring statistics are up-to-date, or building them with the FULLSCAN option. For example, depending on the nature of the query, you may need to provide all or some of the following:
In general, anything you can do to ensure that estimated row counts are close to the runtime values will help the optimizer cost the serial and parallel alternatives more accurately. Many failures to choose a parallel plan are caused by inaccurate row counts.
SQL Server uses a model to estimate the runtime cost of each operator in a query plan. The exact calculations vary between operators, but most are based on a minimum cost, with an additional per-row component. None of these estimates of expected CPU and I/O cost take into account the specific hardware SQL Server finds itself running on. The advantage of this is that plans from one machine can be readily reproduced and compared on another machine running the same version of the software, without having to worry about hardware differences.
Not all operators can be costed reasonably, and things like functions are particularly problematic because the optimizer has no clue how many rows might be produced or what the distribution of values might look like. Even very normal-looking operators can pose problems. Consider the task of estimating the number of rows and distribution of values resulting from a join or a complex GROUP BY clause. Even where reasonable estimates can be made, the derived statistics that propagate up the query tree (from the persistent statistics at the leaves) tend to become quickly less reliable. The optimizer includes many heuristics that aim to prevent these inaccuracies getting out of control, so it might resort to complete guesses after only a few operators as the compounding effect of deriving new statistics takes hold.
There are many other assumptions and limitations of the model that will not fit into a blog post, the interested reader can find more detailed information in Chapter 8 of the indispensable SQL Server 2008 Internals book.
When SQL Server costs a parallel plan, it generally reduces the CPU cost for a parallel iterator by the a factor equal to the expected runtime DOP. For example the previous query can produce the following serial and parallel plans:
Taking the Merge Join operator as an example, the parallel version has its CPU cost reduced by a factor of 4 when the expected runtime DOP is four (serial plan on the left, parallel on the right):
On the other hand, the Index Scans show no reduction in I/O cost, though the CPU cost is again reduced by a factor of four:
As mentioned earlier, different operators cost themselves differently (for example a many-to-many merge join also has an I/O cost component that also happens to be reduced by a factor of four). These details also vary somewhat between releases, so the presentation here is to give you an appreciation of the general approach rather than to dwell too much on the specifics.
Looking again at the serial and parallel plans, it is clear that which of the two plans costs cheaper depends on whether the parallel plan saves enough by reducing CPU and I/O costs in the various operators, to pay for the extra operators in the plan. In this case, the extra operators are three exchange (parallelism) operators – two Repartition Streams to redistribute rows for correct results when joining, and one Gather Streams to merge the threads back to a single final result.
The way the numbers work means that it is often a tight race between the best parallel and serial plan alternatives. In many real-world cases, the difference between the two can be extremely small – making it even more frustrating when the serial version turns out to take fifty times as long as the parallel version to execute. One other point worth mentioning again here is that the DOP estimate is limited to the number of logical processors that SQL Server sees, divided by two. My test machine has eight cores, all available to SQL Server, but the DOP estimate used for costing calculations is limited to four. This has obvious consequences for costing, where CPU and I/O costs are typically divided by the estimated DOP of four, rather than eight.
Plans with Nested Loops joins can be a particular problem, because the inner side almost always runs multiple threads serially. The parallelism icons are still present, but they indicate that there are DOP independent serial threads. The distinction is perhaps a subtle one, but it (a) explains why operators that normally force a serial zone can run 'in parallel' on the inner side of a loops join; and (b) the optimizer does not reduce the CPU costs on the inner side by the estimated runtime DOP. This puts nested loops at an unfair disadvantage when it comes to parallelism costing, compared with Hash and Merge Joins.
This last category concerns the fact that the optimizer may not get as far as evaluating a parallel plan at all. One way this can occur is if a final plan is found during the Trivial Plan stage. If a Trivial Plan is possible, and the resulting cost is less than the configured cost threshold for parallelism, the full optimization stages are skipped and a serial plan is returned immediately.
The following query has an estimated serial plan cost of around 85 units, but with the parallelism threshold set to 100 a Trivial Plan is produced (as shown in the plan property 'Optimization Level' or by checking the changes in sys.dm_exec_query_optimizer_info as shown below:
FROM sys.dm_exec_query_optimizer_info AS deqoi
[counter] IN ('trivial plan', 'search 0', 'search 1', 'search 2')
FROM dbo.bigTransactionHistory AS bth
When the cost threshold is reduced to 84 we get a parallel plan…
A deeper analysis shows that the query still qualified for Trivial Plan (and the stage was run) but the final cost exceeded the parallelism threshold so optimization continued. This query does not qualify for 'search 0' (TP or Transaction Processing) because a minimum of three tables are required there.
So, optimization moves on to 'search 1' (Quick Plan) which runs twice. It runs once considering only serial plans, and comes out with a best cost of 84.6181. Since this exceeds the threshold of 84, Quick Plan is re-run with the parallel plan option enabled. The result is a parallel plan at cost 44.7854. The plan does not meet the entry conditions for 'search 2' (Full Optimization) so the finished plan is copied out.
Returning to code path reasons that prevent a parallel plan, the last category covers queries that enter the Quick Plan stage, but that stage terminates early, either with a Good Enough Plan Found message, or a Time Out. Both of these are heuristics to prevent the optimizer spending more time optimizing than it stands to gain by reducing estimated execution time (cost). Good Enough Plan results when the current lowest cost plan is so cheap that further optimization effort is no longer justified.
Time Out is a related phenomenon: at the start of a stage, the optimizer sets itself a 'budget' of a number of rule applications it estimates it can perform in the time justified by the initial cost of the plan. This means that query trees that start with a higher cost get a correspondingly larger allowance of rule applications (roughly comparable to the number of moves a chess program thinks ahead). If the optimizer explores the allowed number of rules before the natural end of the optimization stage, it returns the best complete plan at that point with a Time Out message. This may well occur during the first run of 'search 1', preventing us reaching the second run that adds parallelism.
One interesting consequence of the rule concerning Trivial Plan and the cost threshold for parallelism is that a system configured with a threshold of zero can never produce a Trivial Plan. Bearing this in mind, we can generate a surprising Time Out with this query:
SELECT * FROM Production.Product AS p
As you would expect, this query is normally optimized using Trivial Plan (there are no cost-based plan choices here):
…but when the cost threshold is set to zero, we get Full Optimization with a Time Out…the optimizer timed out working out how to do SELECT * from a single table!
In this particular case, the optimizer 'timed out' after 15 tasks (it normally runs through many thousands). A Time Out result can sometimes also be an indicator that the input query is over-complex, but the interpretation is not at all that straightforward.
We need a robust query plan hint, analogous to MAXDOP, that we can specify as a last resort when all other techniques still result in a serial plan, and where the parallel alternative is much to be preferred of course. I really want to emphasise that very many cases of unwanted serial plans are due to designers and developers not giving the optimizer good quality information. I see very few systems with things like proper multi-column statistics, filtered indexes/statistics, and adequate constraints. Even less frequently, do I see (perhaps non-persisted) computed columns created on query filter expressions to aid cardinality estimation. On the other hand, non-relational database designs with poor indexing, and decidedly non-relational queries are extremely common. (As are database developers complaining about the poor decisions the optimizer makes sometimes!)
In the meantime, there is a workaround. It's not perfect (and most certainly a choice of very last resort) but there is an undocumented (and unsupported) trace flag that effectively lowers the cost threshold to zero for a particular query. It actually goes a little further than that; for example, the following query will not generate a parallel plan even with a zero cost threshold:
FROM Production.Product AS p
JOIN Production.TransactionHistory AS th ON
th.ProductID = p.ProductID
This is a completely trivial query plan of course – the first row from the scan is joined to a single row from the seek. The total estimated cost of the serial plan is 0.0065893. Returning the cost threshold for parallelism to the default of 5 just for completeness, we can obtain a parallel plan (purely for demonstration purposes) using the trace flag:
The parallel alternative is returned, despite the fact it costs much higher at 0.0349929 (5.3 times the cost of the serial plan). In my testing, this trace flag has proved invaluable in certain particularly tricky cases where a parallel plan is essential, but there is no reasonable way to get it from the standard optimizer.
Even experts with decades of SQL Server experience and detailed internal knowledge will want to be careful with this trace flag. I cannot recommend you use it directly in production unless advised by Microsoft, but you might like to use it on a test system as an extreme last resort, perhaps to generate a plan guide or USE PLAN hint for use in production (after careful review).
This is an arguably lower risk strategy, but bear in mind that the parallel plans produced under this trace flag are not guaranteed to be ones the optimizer would normally consider. If you can improve the quality of information provided to the optimizer instead to get a parallel plan, go that way :)
If you would prefer to see a fully supported T-SQL OPTION (MINDOP) or OPTION (PARALLEL_PLAN) hint, please vote here:
Update: For SQL Server 2016 SP1 CU2 and later, there is a new undocumented hint that does not require administrator privileges:
See this post by Erik Darling and this one by Dmitry Pilugin.
© 2011 Paul White
Twitter: @SQL_Kiwi
Email: firstname.lastname@example.com
If you would like to receive an email when updates are made to this post, please register here
Subscribe to this post's comments using
Great information as usual Paul! One nugget I wasn't aware of was this "The T-SQL scalar function restriction is also a bit sneaky. Any reference to a table with a computed column that uses such a function will result in a serial plan, even if the problematic column is not referenced in the query.". Adds to my "Death by UDF" beliefs! :-)
I also like your possible use of this undocumented/unsupported TF in a non-production environment as a way to get a parallel plan on which you can base a plan guide!
Great post, as always.
The timing is also perfect, and I included it in my letter to Santa (I already voted, one up). I hope we can get this gift, but as you clearly stated, we developers have to be good with the optimizer.
Hi Kevin,
Thanks! And you're welcome for the nugget :)
Hey Alejandro,
I just *knew* Santa was real all along :)
Paul, this is another well presented article and I agree there should be better hints for forcing parallelism in the relatively few cases when it's needed.
However, your post will read to many inexperienced readers as if parallelism is always good, propagating the SQL myth that it should always be left switched on (as, unfortunately, it is by default).
Many SQL OLTP systems have suffered severe performance degradation from uncontrolled query parallelism & it is unfortunate that Microsoft leaves this feature switched on be default. Note that Oracle does the opposite - you have to configure Oracle specifically to parallelise queries, which is generally done for Data Warehouse systems, which are the vast minority of installed systems both on Oracle & SQL. Oracle has this the right way around imo as more systems benefit from the default configuration & those that genuinely need parallelism can switch it on as needed.
Your post is insightful but many will read into it that parallelism is universally good & should be used widely when the opposite is in fact that case. Most SQL systems are OLTP, not Data Warehouse & OLTP systems are generally configured with multi CPU cores to support concurrent queries, not parallelism within queries.
Your post also leaves out the crucial context of impact from concurrent workload. As with other recent posts on query parallelism, your performance measurements are taken in a clinical single query scenario. If you ran multiple instances of these queries & compared the serial vs parallel performance your results would highly likely be very different & reflect better what SQL Server generally does in real world workloads - processing many queries concurrently. Even Data Warehouses often need to support concurrent query processing so hard-wiring parallelism into query hints is questionable when the developer doesn't know for sure up front whether the query will always have all CPUs available to it..
Hi Greg,
Thanks for taking the time to post such a detailed commentary, and I appreciate the kind words regarding presentation and insight!
This is a very tightly focussed post aimed at any advanced developer that has ever struggled to convince the query optimizer that a parallel plan really is the best way to go. Implicit in that is the idea that parallelism, in the specific case considered, is not only desirable but pretty much essential.
OLTP-type workloads are not the target of this post, as these are already well-served with configuration options (workload groups, server MAXDOP) as well as the query-level MAXDOP(1) hint as noted at the top of this entry.
Certainly, true high-concurrency OLTP-type workloads do not often require parallelism to keep schedulers busy doing useful work. For those systems, very selectively overriding server MAXDOP using a hint (or using a different workload group for Resource Governor) can be the way to go, in case there are off-peak queries that would benefit and where sufficient processor resource is likely to be available.
Nevertheless, regardless of the system, there will always be times where the occasional query fails to achieve desirable parallelism due to the causes noted in this entry and in my Connect item. In those limited cases, I would hope the details presented here still add value.
My posts are somewhat aimed at advanced practitioners, so I do rather assume the reader knows not to over-drive parallelism, as well as being aware of concurrency implications.
Thanks again for the comment.
I follow your intent & agree that it might be good to have more options to force parallelism in specific Data Warehousing scenarios but I can also see how this would very likely be a mis-used feature in OLTP workloads
My concern is that parallelism is already a widespread problem with OLTP workloads & articles such as this are easily mis-interpreted to mean that it should be used even more widely. You might be targetting an expert audience but a wide variety of readers will interpret this information as meaning it applies to their systems & given that most SQL users are OLTP users, it's important to point out the difference between applicability to OLTP vs DW systems.
Very interesting post and helps explain why I often get that "why isn't it using a parallel plan" moment.
However as one who has been pushing parallelism for many years I still have to take up this OLTP vs DSS "thing" I generally support operational systems, they are not data warehouses so I guess they could be OLTP but I've never believed in this black and white view of database applications. I've struggled over the years with decisions to disable parallelism on systems which have clearly needed parallelism to scale and in fact with falling clock speeds putting your app on 80 cores with parallelism turned off is never going to achieve very much ( in my view ). However in my "own world" I've been tuning queries by often having to force parallelism for many years - in fact I think I've commented on this subject on your blog before.
As ever I'm grateful for your work which is awesome and enhances my knowledge of SQL Server constantly - Have a good christmas.
Hi Colin,
Very high quality comment as always. Have a great Christmas too!
Hi Grumpy, if query parallelism is good in operational / OLTP DBs why does Microsoft switch it off during officially audited TPC benchmarks?
TO check this yourself:
Pull down the Supporting files, extract files, open ./Clause2/Utility/Database_Options_2.sql
You'll see they leave MAXDOP set to 0 during the data load phase prior to the benchmark setup but then explicity set it to 1 during runtime for the benchmark test.
They have been doing this for years with all OLTP Benchmark Kits & they don't certify benchmarks if the hardware vendor who runs the test alters the configuration.
Leaving query parallelism in the default configuration of zero is simply wrong for OLTP workloads. It only makes sense even vaguely in Data Warehouse workloads.
Is it possible that Colin's mixed-workload OLTP systems don't conform to your idea of OLTP? Seems quite likely; based on your comments, it's become clear your idea of a data warehouse workload doesn't conform with mine.
Apparently -- surprise, surprise -- there are a huge number of different types of workloads out there, and it's tough to make generic recommendations that work for all of them. Hopefully our readers are smart enough to figure that out on their own, so I think your concerns are rather unfounded.
I'm referring to the Transaction Processing Council's definition of OLTP, which includes Microsoft's agreement with Oracle & IBM on how OLTP should be defined for performance measurement purposes: www.TPC.org.
How could it be any clearer that my definition agrees with Microsoft's?
I tend to agree with the views of Colin and Adam, though there may well be workloads out there that are pure OLTP (whatever that means these days), where parallelism is rarely required. I say rarely, because in my personal experience most mixed-workload systems have queries that absolutely do benefit from parallelism.
To me, 'pure OLTP' implies a system with a very high volume of concurrent navigational queries that affect very few rows, performing many joins and typically using nested loops to do so. These are exactly the sorts of queries that SQL Server will generally not invoke parallelism for anyway: this is exactly what the trivial plan and transaction processing stages of optimization are for.
As discussed in the main text, neither of these optimizer stages are capable of producing a parallel plan, so any queries that get as far as the second run of Quick Plan optimization must at least appear to the optimizer as benefiting from parallel execution. In that case, something must be wrong with the query, database design, or indexing - things that are probably not best 'solved' simply by enforcing MAXDOP 1 globally.
It is also perfectly possible for a parallel query to use less CPU while executing faster than the equivalent serial query. This is a point I explored in a previous post:
I don't personally have much of a problem with the default configuration being MAXDOP 0 with a cost threshold of 5, but it seems to me that any case for change would be better sited on Connect than in the comments section of a blog post like this one.
Paul thanks for another excellent insight into the QO.
These flags are certainly not to be used as startup options ;-) but would help troubleshoot certain 'heavy' queries that insist on running serially despite the lack of parallelism-inhibiting components.
This computed column/scalar function combo is something worth a look. I've got one on a critical table where parallelism is a distant dream.
Good work!
Merry Christmas, Paul!
As a geek, I did enjoy reading this, and thank you for writing this up.
However, putting on my business hat, I have very mixed feelings.
First, when we are developing solutions, the following business requirements usually influence the choices we make when we implement them:
1. Is this a one time task, or is this going to run all the time from multiple connections, so we need the best, the fastest possible algorithm.
2. What performance is good enough, like "it should return under 30 seconds, and use up as little resources as possible", or "executive priority, return ASAP, let everyone else get out of its way and wait".
3. Which resources (memory, CPU) should be used in moderation.
If we had a clear and direct way to communicate these requirements to the optimizer, SQL Server would be a more useful tool. Maybe I am missing something, but good developers just do not implement solutions until they get these requirements straight, until they know what the customers want. Maybe, just maybe, the optimizer should let its customers communicate what they want more directly. Please correct me if I am wrong.
Second, when query tuning get this complicated, other alternatives get more profitable. It may be cheaper to add hardware, and/or develop more in other languages, and let SQL Server only do simpler tasks.
For example, we recently had a problem when the optimizer was not giving up a parallel plan. After verifying that all needed statistics were present and up to date, we just removed several conditions from WHERE clause, moving some filtering to the client (about 5% rows were filtered on the client). As a result, we ended up with a simpler query that was getting a parallel plan for all combinations of parameters, delivering good and predictable performance.
Moving some filtering to the client was a very simple solution, it did not need a highly skilled expert and/or time-consuming research. Also I think it resulted in more predictable performance.
Because SQL Server optimizer is a closed source thing, we cannot just debug through the process of choosing the plan and see what is happening. As a result, in some cases we are getting the feeling that optimizing a complex query cannot be done done - we are likely to get back to it later on, losing time and money every time we have to re-optimize, and eroding customers' confidence.
From my perspective, this seems to be yet another situation when complexity sucks, and a simpler optimizer might be a better fit for my customers' requirements.
What do you think?
Hi Alex,
I agree with your views to a very great extent. Complexity does suck, by and large, and there are many variables to be juggled in the real world.
On the other hand, a good part of your argument seems to go in favour of extra hints like OPTION (PARALLEL_PLAN): this gives us a direct way to communicate what we want to the optimizer, and is likely to result in predictable performance without e.g. moving filtering to the client.
I would very much like to be able to construct query plans directly, much in the way SSIS allows us to build data flows, but I doubt that level of control will ever be afforded to us.
It's a very interesting area overall. Merry Christmas to you also, and I'm glad your inner geek enjoyed this post :)
As I generally work in an isolated environment or if you like I don't work with anyone with an in depth sql knowledge so blogs are an important source for me, Paul and Linchi especially put out stuff which is of great interest to me ( I also read many others ). I'd probably say that in general I support systems which have challenges with performance, over the years these have been in banking, pharmaceuticals, travel and "other" financial systems - many have been international. Performance Tuning is vital and I don't really know what you'd define these databases as - they're not a Datawarehouse ( I have been there too and used lots of parallel stuff there ) Sadly I admit to reading the full disclosure of many benchamrking tests but do I think these have much relevance to what In do? Probably not! Do I find the many blogs relevent ? Absolutely and every so often there's something which shines a light on something I'm working with. Sitting outside the OLTP vs DSS thing it just makes sense to realise that parallel processing must be the way to go. I've enjoyed the discussion on this one Paul, Alex, Greg, Adam.
I definitely do want to be control, as this seems to be more profitable - we are losing time and money when we spend too much time trying to figure out how a closed source black box optimizer works.
I am not sure "more hints" is the right word, because most hints may be silently ignored. A stronger word is needed, so that when I execute a query and require, not hint, "and BTW your budget is 1 CPU and 4GB", then the query either uses no more than 4 Gb, or throws an exception, just like when I misspell a column name. Does it make sense?
I frequently write my own execution plans in C#, it is one of my best/easiest options whenever the optimizer is struggling. Because much of my data is temporal, RDBMS is not a perfect tool for it anyway, so I would not blame the optimizer for struggling with time series and such. Anyway, in the best case a self join developed in C# outperformed an "already optimized" set based query by twenty thousand times, and with little effort on both SQL and C# sides. Because T-SQL is quite verbose and C# may be very succinct, the C# code was shorter and more more readable than the T-SQL one.
In addition, it is so very much easier to unit test C# code than to unit test T-SQL, and NUnit tests using mocks run much faster.
So, I am writing less and less complex T-SQL. This is especially true since I saw the pricing on Enterprise Edition. I am currently running 2008 R2 Enterprise Edition, but I will definitely not upgrade to 2012 Enterprise Edition. This is yet another incentive to move complex processing off SQL Server,especially for frequently running queries. What do you think?
Most hints (e.g. INDEX, LOOP JOIN) are already directives, though some (e.g. ROWLOCK) are not, that is true. I was thinking more of directives, to be clear about it. As far as CPU and memory budgets are concerned, isn't that what Resource Governor gives you? Admittedly, this does require Enterprise Edition.
I agree that SQL Server is in desperate need of native temporal support, though CLR functions and aggregates are a good workaround in many cases while we are waiting for that to arrive. In general, yes, I do agree that simple SQL is best - for all sorts of reasons, not just optimizer-related issues.
Thank you for the good article, Paul.
Great article, thanks
I was under the impression that when you limit MAXDOP through Resource Governor that it doesn't act like the instance wide maxdop setting, or even maxdop option within a query, but rather effects the number of cores the query will be run on rather than the number of cores for which the plan will be generated.
For example, if you limit maxdop to 1 with Resource Governor on an instance that has been configured for a higher maxdop, it will generate a parallel plan but this parallel plan will be run serially, providing different behaviour to that which occurs when the instance has been set with global maxdop 1.
Hi Nick,
There are a number of differences between RG-enforced DOP and the max degree of parallelism option, for example the latter can be overridden per query with an OPTION (MAXDOP n) hint, but the RG limit is absolute.
It may be possible to generate and cache a parallel plan that is then restricted to running serially via RG, but this isn't a scenario I have ever tested for myself. I did write before about how SQL Server can execute a cached parallel plan serially by removing exchanges at run time: http://sqlblog.com/blogs/paul_white/archive/2010/11/04/myth-sql-server-caches-a-serial-plan-with-every-parallel-plan.aspx
Thanks for the question!
Thanks for the article.
By the way, talking about not using Trivial Plan. Maybe it would be interesting to you that there is a special trace flag that allows you to skip trivial plan phase - it's 8757, you may try it.
Also I've noticed that you are strongly interesed in optimizer, so it may interest you. I have discovered a set of undocumented or ever not mentioned anywhere flags that allows to:
Thanks but I 'discovered' those a long time ago :)
I've been presenting on this topic at user groups and SQL Saturdays over the last month or so. You can download the slides and demos at e.g. http://www.sqlsaturday.com/viewsession.aspx?sat=139&sessionid=8412
Well, ok then, thanks!
Great article!
I've published intresting test's result to connect.microsoft.com.
May be you will find it intresting too and explain to the community what happens?
Hi SomeName (aka M2M telematics),
Thank you. The tipping point for parallelism for me on that query (also on 2012) is 2257. The estimated cost for the parallel 2257 plan is 11.2158; for the serial 2258 plan it is 11.2162. It is often possible to find the 'edge' where the costing component produces very close costs for serial and parallel options.
In this case, the cost savings from performing a parallel scan are offset by the increased estimated cost of the conversion in the Compute Scalar. The cost of that conversion is related to the size of the data type, so it makes sense that varying that parameter affects the balance between serial and parallel options.
Hello again )
It's good and reasonable explanation, but it concerns only 2) and 3).
Key point of this test is query 1) vs 2) and 3).
Query 1) doesn't contain conversion at all and field Text contains much less characters than 695 at any row but plan doesn't contain parallelism.
Have I missed something?
For query 1, the serial plan estimated cost is 10.6047; getting a parallel plan with TF 8649, the estimated cost is 12.0579. So, the optimizer chooses the serial plan alternative - the lower cost of the parallel scan is not enough to pay for the Gather Streams operator.
If your cost threshold for parallelism is set to 11 or higher, the plan will be a TRIVIAL one, and parallelism will not even be considered.
This post helped me a lot, except for one thing: You have to be a sysadmin to use QUERYTRACEON. I have reports in SSRS called by a website using a SQL user for authentication and I'd like that user to be able to force parallelism since the SQL engine doesn't want to (which is baffling). But it seems like a big security risk to make that user a sysadmin.
Yes, that is a big restriction, and another) good reason not to use this trace flag in production. We really need a query hint - so I hope you voted and commented on the Connect item :)
Okay now I'm really confused. I have a login without sysadmin rights that is able to execute a stored proc that contains QUERYTRACEON. When that login executes DBCC TRACEON it gets the message below.
Msg 2571, Level 14, State 3, Line 1
User 'Website' does not have permission to run DBCC TRACEON.
How does this make any sense?
Ownership chaining allows OPTION (QUERYTRACEON 8649) in a procedure where separate DBCC TRACEON (8649) and DBCC TRACEOFF (8649 ) statements would fail permission checks. This may or may not be fully by design behaviour, but it is the way it works.
See the links below for information on ownership chaining (if necessary):
Thanks! I figured it had to be something like that but I didn't realize ownership chaining went so far as to effectively allow sysadmin rights (based on the contents on the sproc). I will see The Hobbit this weekend in your honor.
This trace flag just saved my bacon Paul!! Had a very complex/ugly nested loop, outer apply, spatial mess of a query that just wouldn't parallelize despite 4-figure query cost. I thought the trace flag for SQL 2012 TOP-slow-query would do it for me (there was a top in the query) but it didn't. 8649 got me a parallel plan and cut run-times by an order of magnitude+. And for something I had to iterate almost 400 times (and schedule for once a day going forward) that REALLY made me happy!! :-)
Going a step further...it would be nice to able to direct a parallel plan for a specific operation say a union or join. Same with mindop/maxdop.
I have a case where each side of a "union all" takes .5 and .3 and when run with maxdop(1) it shows an expected .8. When I encourage a parallel plan using the hint you suggest the parallelization occurs not at the union but rather at multiple levels. When using maxdop(2) the optimizer did not place the parallel operation at the union point which for my case at least would like result in about a 40% improvement (assuming each side of the union does not interact which is the case).
Thanks for the article!
Hi crokusek,
Not sure I follow all that completely (it's hard to describe plans in words sometimes!) but I do agree it would be nicer to have finer control over parallelism sometimes. Having a documented way to generate a good parallel plan would be a good start...
The query optimizer does not always choose an optimal strategy when joining partitioned tables. This
" Even experts with decades of SQL Server experience and detailed internal knowledge will want to be
Hi Paul, thanks again for this article.
Actually, it is not the first time I read it, but this time I just wanted to share with you one of my observations that helped me force (or I should probably say just "get") a parallel plan in one of my particular scenarios.
The main thing is this - it seems, SQL Optimizer makes correct estimation of the number of rows in a table variable in case this table variable 1) is queried inside dynamic SQL, and 2) is being passed there as parameter (which essentially means it is of a user-defined type).
It is probably a well-known fact, but at least I was not aware of it before.
The root cause for being stuck with a serial plan in my particular case (or, at least, what seemed to be a root cause) was incorrect cardinality estimation. Which, in its turn, was caused by the fact that I was using table variables inside one of my functions.
For myself, I found the following explanation to this fact: since Optimizer under "normal" conditions estimates the output of a table variable as one single row, it greatly under-estimated efforts needed to execute my query, which led to a compile-time decision that a serial plan would be good enough.
When I wrapped a call to my function in a dymanic sql statement (I did it for considerations which are irrelevant to my main point here), I suddenly got a perfect parallel execution plan.
It is when I started to dig further into this that I discovered the main thing that I am sharing here.
P.S. I have a script that proves my point in case you are interested - not sure how to attach it here though..
P.P.S. there is a small chance you may actually remember me - I am the guy how approached you in the intermission of your first session at PASS 2012; my question was related to the main subject here - forcing parallel execution plan using dynamic SQL..
Hi Eugene,
Yes I do remember your question at the Summit - it's good to hear the details. It is true that the cardinality of a table variable is most often estimated at a single row, but the optimizer can see the true cardinality if a statement-level recompile occurs. The easiest way to demonstrate that is to add OPTION (RECOMPILE) to the statement (though there are other causes of statement-level recompilation of course).
DECLARE @T AS TABLE (col1 int NULL);
The post-execution (actual) execution plan shows an estimate of 1 row for the first SELECT, and 3 rows for the second SELECT. Table variables don't support statistics though (not even those normally associated with indexes) so the optimizer still doesn't have all the information it could have, but this technique does increase your chances of a parallel plan as you have seen.
I realize you weren't using OPTION (RECOMPILE) in your example, but what you saw was the result of a statement-level recompilation.
Thank you for the information, Paul
very nice article ...
I just ran across this -- amazing work as usual Paul! I always learn so much from your articles.
I have a question and a comment:
Can you point me to any articles that explain why mTVFs never produce a parallel plan? I have seen this discussed but have never found it documented.
Turning mTVFs into iTVFs has become one of my favorite low-hanging-fruit performance optimizations" I have been using lately with great success is to whenever possible. My experience has been that many developers create mTVFs because they don't know any better. I see lots of this:
CREATE FUNCTION f(p)
RETURNS t @TABLE (c1 int, c2 int)
INSERT INTO @x SELECT blah FROM blahblah
and have had huge success changing them to
SELECT blah FROM blahblah
Hi Alan,
Multi-statement table-valued functions force a serial branch in the calling plan, but parallelism can exist in other branches of the same plan.
It is also possible to see parallel execution within the TVF, though naturally not on the statement that modifies the table variable.
Replacing TVFs with in-line functions ("parameterized views") is very often a very good idea, I agree.
I'm afraid I don't know why the serial-branch restriction on TVFs exists. It may be because the function might contain its own parallelism, and it would be difficult to ensure the DOP concurrent thread limit is respected in case a parallel plan calls a parallel TVF, but that's just a guess.
Thanks for the kind words about the blog, I appreciate it :)
I found this post to be well written and informative. I would note however that as with most hints, a hint to force parallelism might be rather brittle. I would recommend instead a Cardinality hint, such as the one implemented in Oracle. Instead of directing the planner about what to do, it allows the developer to give it a better clue about the cardinality of intermediate result sets. In my personal battles with failing to parallelize, the problem has always stemmed from mis-estimating the cardinality involved.
Hi Paul,
Thanks for this wonderful article, it helped me with my current issue.
Interesting tidbit on the pseudo-parallelism on the inner side of a nested loops join. But what is the point of having multiple threads running serially compared to having a dedicated worker processing the inner side? Wouldn't the performance be about the same, but also fewer context switches in the latter case?
Thank you!
Mark, there are indeed DOP dedicated threads on the inner side that pick up the next correlation value on each rebind. The effect is to have DOP threads running serially, so I don't think there's a difference. Please feel free to restate the point if I misunderstood you.
Sorry, what I meant to ask was, what advantage does a parallelized 'loops join' offer over a non-parallelized one in terms of speed, given that the inner side runs serially in both cases?
A parallel nested loops join at DOP 4 has 4 serial threads running the inner side independently (with different correlated parameters) *simultaneously*.
There is nothing complicated about the inner side, there are no extra threads or exchanges (in general). At e.g. DOP 4, the four threads running the inner side simultaneously are the same four threads running the outer side.
See also my article on SQLperformance.com (though it isn't specifically about nested loops, the concepts of threads and branches are very relevant):
Got it. Thanks.
I was wondering, is the list of "Parallelism-Inhibiting Components" valid for SQL Server 2012 and 2014 aswell?
Hi Blaž,
Off the top of my head, the only one that might be out of date is the common spool one, I'll check it out and update the post for 2012 and 2014 when I get time.
There were trace flags enforcing parallel plans even on single CPU boxes. We needed it when shipping the first release of SQL Server with parallel plans (7.0) since there were only a few machines with multi-CPUs in our labs (no multi-cores then) and that was the only way for testers to see the parallel plans on thier single CPU boxes. I'm not sure if the trace flags are still working; they were never made public so they don't need to be supported. Note: I'm not working in Microsoft any more so I don't have access to the code; I was the PM for QO and QP from 7.0 till 2005 shipped.
Thanks for the insight, Lubor! Very interesting!
I'm wondering if accessing an in-memory (aka memory-optimized, aka Hekaton) table should go in one of the lists of causes of serial plans.
In the Books Online page about memory-optimized table variables, it contains one statement,
"Like memory-optimized tables, memory-optimized table variables,
Do not support parallel plans."
I haven't done much testing around this yet, though, and was wondering if you happen to know more.
Hi Kendra,
The list was never meant to be exhaustive, but perhaps it would be useful to make it so. I'll give it some thought, and possibly update the post again to include all the scenarios I can think of.
"I say rarely, because in my personal experience most mixed-workload systems have queries that absolutely do benefit from parallelism."
This is so important for most real world applications. I'm still looking for a server that's not a sharepoint box that would operate more efficiently at 1 then at 8.
Now, to be brutally honest I can't tune to the degree Paul can but I honestly haven't seen an oltp box that would benefit from setting it to 1.
I think the shared tenant model exacerbates this even further. How many real world boxes have one really well performing database on it?
One last thought, I'd like to thank Paul and Adam for corresponding with me on this topic. I used to think I knew what parallelism was but didn't and both Paul and Adam helped me out via articles and through personal communication.
Paul, above you stated:
"Inconveniently, the list of intrinsic functions is quite long and does not seem to follow a pattern. ERROR_NUMBER and @@TRANCOUNT also force a serial plan, @@ERROR and @@NESTLEVEL do not."
Does that mean if we use ERROR_NUMBER() OR @@TRANCOUNT in a catch that it will cause our plan to become serial? That is perturbing as my standard error handler for 2008 and prior is below. I could remove the @@trancount as it technically should be there to rollback, but removing the call to error_number would break the logic of my catch. With 2012 and higher I just call throw;. Can you confirm if my pattern is invalid or do I need to change it?
/* server 2008 rethrow :( */
@ErrorMessage nvarchar(4000) = ERROR_MESSAGE(),
@ErrorNumber int = ERROR_NUMBER(),
@ErrorSeverity int = ERROR_SEVERITY(),
@ErrorState int = ERROR_STATE(),
@ErrorLine int = ERROR_LINE(),
@ErrorProcedure nvarchar(200) = ISNULL(ERROR_PROCEDURE(), '-');
SELECT @ErrorMessage = N'Error %d, Level %d, State %d, Procedure %s, Line %d, Message: ' + @ErrorMessage;
RAISERROR (@ErrorMessage, @ErrorSeverity, 1, @ErrorNumber, @ErrorSeverity, @ErrorState, @ErrorProcedure, @ErrorLine)
Hi Tim, no those intrinsics only affect the *statement* in which they appear. Other statements in the batch may use parallelism.
HI Paul,
Thanks for the great post. I work in a DW application in Finance domain. We have massive workload batches and achieving desirable performance is always challenge.
The queries involve joins of sometimes 10s of tables. Our data model is such that almost all of the tables are huge 10+ Million rows and there is no clear small dimension tables that I can use as "Driver" table in a parallel apply pattern, etc.
Hence almost all of the joins are with multiple similar sized huge tables (10+ Mill rows each). Also, almost all the tables have Primary Key on something called a ApplicationRecordId. This is system generated & ever increasing key generated seperately and is used across all tables in star schema. all of our star schema tables are joined by this common ApplicationRecordId field.
some of the complicated queries takes for ever to run. Checking dm_os_waiting_tasks.resource_description revealed the wait bottlenecs are a handful of nodes and all of them are Repartition streams operator.
Checking the query plan revealed:
1. No matter parallel Hash / Merge join is used
2. No matter all the tables are Primary Keyed and joined on the common ApplicationRecordId column,
Each table is acted by RePartitionStreams operator before join(Hash or Merge Join) which massively slows down the query. Infact serial version of this query is running faster.
Can you please help me understand:
1. In a parallel operation, if RepartitionStreams operator is necessary to happen before a join between two equally sized BIG tables (10+ mill rows)
2. Any tips on how to attack & resolve this performance issue.
Hey Paul,
As always, I bow, scrape, etc. This is really useful.
What's not clear to Microsoft, I think, is just how useful this is in the case of Columnstore indexes. You only get Batch Mode processing when the plan goes parallel. BUT, tiny little plans with estimates that you wouldn't think would benefit in any way from parallel execution still have huge benefits when a Columnstore index goes to batch mode. I suspect when word gets out on this, there's going to be a ton of use of this thing.
Hey Grant, sorry I've only just noticed you commented. My apologies. Yes, I agree with your concerns, even given the 2016 announcements about serial batch mode and so on. We still need a way to ensure we get parallelism when we know, more likely than not, that good performance will require it.
Thanks, great article!
Do CAST(), ABS(), or CHECKSUM() inhibit parallelism?
I do wonder, if using persisted computed columns will help to assuage the performance impact caused by a computed column, even if it still forces serialization. Any thoughts?
At long last, I have finally become a footnote in one of your posts.
Thanks Paul!


text:
Meet Holly and Luke, the First Dates guinea pigs who are on their own quest for love. No humans or guinea pigs were harmed in the making of this show.
Share page
Titles, characters and all other elements are the trade marks and copyright of TVNZ or its licensors. All rights reserved. Further details available here.

subset: commoncrawl


text:
Carlos Dengler, bassist with New York indie oufit Interpol is set to
be a movie star after landing a role as a drug dealer in My Friends Told Me About You, a Chicago based thriller that Dengler will also be producing . Other than making great music, Interpol also have to be the snappiest dressers in the music biz at the moment, which is why I'm featuring them here.
I would have given that accolade to Rufus Wainwright, but lately he's started to dress as if his wardrobe was inspired by a trip to DFS. And don't get me started on Doherty. Two hats? Why? Just take a look at the picture above to see if you don't just agree on how sharp these guys look.


text:
Joe The Barbarian is where I first remember paying attention to Sean Murphy as an artist. After Hellblazer: City of Demons and American Vampire: Survival Of the Fittest, Murphy joined my "must buy" as an artist – a category he shares with only two other artists, Sam Keith and Scottie Young. So when Punk Rock Jesus was announced as a limited series from Vertigo I made the note right away to pre-order the trade as soon as possible. It wasn't until after I re-read the solicitations that I realized that Sean wasn't only attached to this as the artist, but also the writer my curiosity was peaked and I became more and more impatient for the book as time passed. When I finally received the trade I drank it in going so far as to read it while my wife was fretting in the waiting room while my son was having tubes surgically implanted into his ears. In brief, as an artist on my must buy list, Punk Rock Jesus did not disappoint. As story, Sean Murphy has a while to go before he makes that list as a writer.
The art in Punk Rock Jesus is amazing, I've stated before my growing respect for black and white comics, and all I can say in that regard is that color would have muddled the book. Murphy is able to create pages and pages of art with so much going on that visually needs to be taken in that color would have been a waste of time and resources. From a visual perspective, nothing would be added by the inclusion of blues, reds, etc while the tone and feel of the book would have been lost had they been used. Murphy reminds me of Ted McKeever and his ability to use the whole page to tell the story. No white space is wasted and every line serves a purpose. In short, the book is beautiful.
The writing, on the other hand, leaves a lot to be desired. Punk Rock Jesus tells the story of a young man who was cloned from DNA taken from The Shroud Of Turin for the purposes of creating a reality show around Jesus version 2's (J2) life. Basically, The Truman Show meets Christianity with a bit of self serving capitalism tossed in. Theoretically, this is the perfect set-up for a great social commentary on the nature of faith, television, technology, and the free market. In reality though, Sean Murphy was not the person to write it.
In no way am I trying to say that the story is bad, it's just not as interesting or as deep as it wants to be. There are barely two likable characters in the whole comic, and one of them is rarely seen and the other abandons his promises to his Lord as soon as it becomes hard to maintain his vow. Neither of the likable character are the Jesus clone, Chris, who turns into a huge douche bag by the time he is old enough to have an original thought, and goes on to act like a high-school student who just discovered atheism rather than a main character whose voice the reader is supposed to listen. The basic tenant of Chirs' message is "religion bad" and Murphy did not include any kind of redeemable religious character who could challenge this assessment. Instead what we get are protagonists we are supposed to (but don't) like who are anti-religion and religious antagonists who contribute nothing positive to society and rely on violence and anger to move their message forward. In the world of Punk Rock Jesus , there is no grey area when it comes to religion and faith, and this hurts the legitimacy of the point Murphy is trying to make.
On the whole, the art in Punk Rock Jesus outshines the story to the 10th degree, and you should read the book just for it alone. Keep in mind, though, as you read the book that the comic that, while the story is interesting and entertaining, the themes and message of the book are, at best, simplistic and unbalanced, and, at worst, juvenile and pedantic.
Off topic, if you are looking for a great Sean Murphy book where he is the artist and writer, I recommend Off Road, easily the funniest comic I've read since Barry Ween.


text:
Good luck with us getting any bike infrastructure in a hurry given the current amounts of car related infrastructure currently underway.
Widening of the Wardell Street/Samford Road intersection.
Widening of Beckett Road which will ultimately mean an upgrade of the Rode Road roundabout due to the increased traffic flows from Beckett Road.
And now a push for a bypass through Flockton Street joining Old Northern Road and Stafford Road by Murray Watt (probably because he knows that the upgrade of Beckett Road is going to mean even greater pressure on the Old Northern Road/Stafford Road intersection).
If you are like me and think that this continued motor vehicle infrastructure is a poor solution to the traffic problem then I suggest that you write to both Murray Watt and Tim Mander to voice your displeasure.
I did this just this week:
I have just sent the below email to Murray Watt but felt that I should also send this same correspondence to your office for a response.
This "rant†has arisen from the article that I have read today in the Northern Times along with the articles on Murray's website.
I want my 3 boys to be able to live in a city that they and their children can be proud to live in. Not some tunnelled, overpassed LA style mess.
I have copied my email to Murray word for word.
I am sorry Murray but the proposed bypass and all of the other traffic infrastructure "solves†nothing.
It is time to make a hard decision and actually do something that will be in the interests of our suburbs rather than take the easy route and continue to band aid the real problem which is that we have far too many people using our roads and far too few workable alternative options.
The tunnels and bypasses that we are seeing spring up all over Brisbane will give us a temporary fix but it has been proven in the past that these "fixes†ultimately end up just as crowded as the previous problem.
Yes it is the popular choice by your voters who refuse to entertain alternatives but I do not believe it should be your job to take the popular option (although I do recognise that if you don't stay popular, you don't stay).
When will it be time to say enough is enough with roads?
How many suburbs do we need to destroy?
Kedron is almost there, along with large sections along Lutwyche Road.
Wooloowin is suffering too.
Enoggera and now Everton Park?
Out further we have major widening of Becketts Road that ultimately will funnel into the Rode Roundabout.
I can see the next project being a rebuild of this roundabout as it will not cope with the extra traffic flows that will come.
Not so much your area but the upgrade of Gap Creek Road has led to substantial increases to traffic via that route and through The Gap.
Hamilton Road from Old Northern Road through to Chermside has seen likewise.
I cannot think of a single time when additional vehicle infrastructure has seen an ongoing improvement to traffic flow.
The problem is not roads.
It is traffic.
It is the people that use these roads.
And it is these people that need to be re-educated about their actions and "actively encouraged†to use the alternatives.
Providing free and improved access to the CBD does nothing to this end.
Not sure if you have ever seen this video and information, but perhaps it is time for our governments (local, State and Federal) to start thinking about this.
It worked in San Francisco, Portland Oregon and in Seoul Korea so there is absolutely no reason it cannot work here.
Jeffery Kenworthy, from our Curtin University and who was involved in the Korean Cheonggyecheon project effectively says in the attached video, that traffic behaves like a gas and will expand across all infrastructure until it is clogged regardless of how much infrastructure is provided. So ultimately you are trying to "solve†a puzzle that cannot be solved by adding more roadways.
http://daily.sightline.org/2011/04/01/g ... -in-seoul/
http://www.infrastructurist.com/2009/07 ... ve-a-city/
http://www.youtube.com/watch?v=KtV1M0Cv ... r_embedded
More public transport at the expense of car lanes is what is needed along with improved cycling infrastructure (up from basically none in our Northern suburbs despite the much talked about bikeway program).
I hope that you actually spend the time to read and watch the links provided and then spend some time thinking about what is being said and how similar our current situation is to these cases.
Then take the time to work out how you (and others in power) can actually FIX the problem rather than taking the popular option.
Don't be one of our politicians who are remembered for creating more ugly LA style fly overs.
I look forward to your response


text:
The 71st General Assembly of the World Health Organization ( WHO ) was declared opened in Geneva, Switzerland with the Director-General of the organization, Dr Tedros Adhanom Gbebreyesus calling for a shared sense of purpose in tackling the world s health challenges. In his keynote address, Gbereyesus said there is no commodity in the world moreRead More The post WHO General Assembly opens with call for urgent actions appeared first on The Nation Nigeria. The post WHO General Assembly opens with call for ur
Read on the original site
Sunday 21 October 2018
WHO General Assembly opens with call for urgent actions
Sharta declares for dtha,vows to rescue Udu- vanguardngr
UMUAHIA: Jubilation as Ishi Gate Market returns- vanguardngr
Curtain falls on Asaba 2018- vanguardngr
Nigeria to bid for 2025 IAAF World championship- vanguardngr
Confusion in the Senate over party in majority- vanguardngr
8 Marks of a true church by Billy Graham- vanguardngr
APCON reform, if fully implemented, will boost industry growth — Odigbo, AAAN President- vanguardngr
Eternal Life through Christ the only Saviour- vanguardngr
Nigeria Air: A Threat To Domestic Airline Operators?- leadership
Many Guber Aspirants In Ogun Lack Vision – Paseda- leadership
The Face of Change?- nigeria70
2019: Now Or Never For Atiku, Lamido- leadership
PDP And The Return Of Saraki, Tambuwal, Kwankwaso- leadership
Nigeria's Crude Export Rises To Four Month High- leadership
France To Boost Agribusiness Ties In Nigeria- leadership
OAU loses accreditation for Law, Dentistry, Botany- nigerianeye
Taking A Cue From The Prosperity Of Isaac (Part 1)- leadership
Making Maximum Impact In Life (5)- leadership
OAU loses accreditation for Law, Dentistry, Botany- nigerianeye
Economic Growth: Kwara Ranks Second In Africa- leadership
Democratic Republic of the Congo: New Ebola outbreak in the DRC: UNICEF mobilizes staff and supplies to help with the response- reliefweb
CPC Cautions Consumers On Valsartan Intake- leadership
GTBank To Launch Services On WhatsApp- leadership
Inlaks Reiterates Commitment to CBN's Vision 2020- leadership
Nigeria, Niger And The Quest For Petroleum Products- leadership
FG Confiscates N3m Perishable Exports Rejected By UK- leadership
INEC Assures Women Of Level Playing Ground- leadership
Globacom's memorable leveraging of the 2018 World Cup- vanguardngr
Senators take on PMB lawmakers: Why Saraki won't resign!- vanguardngr
The Abracadabra Of Zimbabwe's Elections- vanguardngr
There is hope for Nigeria even with defections — Ambode- vanguardngr
The Auditor-General's Report- vanguardngr
Ighalo just can't stop scroing, nets 12th goal in China- vanguardngr
God's message to young people- vanguardngr
Understanding APC Change- vanguardngr
Understanding APC and the Crisis of Defections- nigeria70
2018 Eko Theatre Festival rounds off today- nigeria70
Gov Emmanuel Grants Amnesty To Militants- leadership
Oba Akiolu Endorses PMB, Ambode- leadership
Polling Can Reduce Election Violence – Says Expert- leadership
Tanker Crushes 4 To Death In Rivers- leadership
Stop Wooing Our Members, PDP Tells Presidency- leadership
Nigeria's Gas Production To Grow By 35%- leadership
Patients, Their Rights And Responsibilities- leadership
The Election of President Muhammadu Buhari as ECOWAS Chairman and Regional Issues and Questions- nigeria70
2019: Dankwambo Declares For President- leadership
Democratic Republic of the Congo: Ebola virus disease – Democratic Republic of the Congo: Disease outbreak news, 4 August 2018- reliefweb
PDP writes INEC, DSS, Police over Kashamu's expulsion- nigerianeye
Iheanacho targets goals against Manchester United- nigerianeye
God sees everything you do- vanguardngr
Earthquake: Akpabio set to dump PDP for APC- vanguardngr
Our lawmakers displaying political immaturity —SAGAY- vanguardngr
The wisdom of the wise (2)- vanguardngr
MVZE ready to drop new single 'By Your Side'- vanguardngr
Expect a new beginning- vanguardngr
Plateau lawmaker, NAF personnel in auto crash- nigeria70
Iheanacho targets goals against Manchester United- nigerianeye
PDP writes INEC, DSS, Police over Kashamu's expulsion- nigerianeye
Healing wings (2)- nigeria70
Serena Williams withdraws from Rogers Cup- politicsngr
Okonkwo, others storm Abundant Life for Superfest- vanguardngr
Healing wings (2)- vanguardngr
Skales Blasts ACP Yomi Shogunle for End Sars Mockery- realchannel65
Serena withdraws from Rogers Cup- PM News
Gateways to contacting the spirit of faith!- thenationonlineng
Defection goes viral as dustbin women sweeps everything- thenationonlineng
Expulsion: PDP writes INEC, DSS, Police over Kashamu, others- thenationonlineng
Okonkwo, others for Superfest- thenationonlineng
Christians and men of God, learn of Jesus, the truth- thenationonlineng
'Why fake Prophets thrive in Nigeria'- thenationonlineng
'Churches must promote probity, accountability'- thenationonlineng
Ranches and deaths and government-inspired tradeoffs- thenationonlineng
Baba Lekki refines his theory: Defection is shifting cultivation- thenationonlineng
MFM defamation: Court slams N3.5 million against lawyer- thenationonlineng
JOYCE 08140948440- sunnewsonline
What does postpartum depression look like?- nigeria70
Democratic Republic of the Congo: Ebola virus disease – Democratic Republic of the Congo, 4 August 2018- reliefweb
Upcoming Actress, Mimisola Daniel reveals she can't marry a man who is not sexually fit in bed- informationng
BBNaija Product, Gifty shares bedroom underwear photos- informationng
Libya: Pushing the boundaries: insights into the EU's response to mixed migration on the Central Mediterranean Route- reliefweb
Senator Godswill Akpabio dumps PDP for APC- nigerianeye
Revive, acquire Brass, OKLNG projects, FG tells NLNG- vanguardngr
Senator Godswill Akpabio dumps PDP for APC- nigerianeye
MOSOP Berate HYPREP Over Failure to Commence Ogoni Cleanup- nigerianvoice
Asaba 2018: Nigeria Records 5 Gold Medals- nigerianvoice
North Korea hasn't stopped nuclear, missile programs- UN report- pmnewsnigeria
Athletes laud Asaba 2018 spectators- pmnewsnigeria
Neymar returns as PSG thrash Monaco in China- nigeria70
Lauretta Onochie lists many sins of Dr Saraki against APC, Buhari's govt – Vanguard News- nigerianbulletin
Youth Unemployment: Obaseki proffers technical education as panacea at UI Alumni Lecture- vanguardngr
Zenit edge Arsenal as Moscow derby ends in stalemate- vanguardngr
Neymar returns as PSG thrash Monaco in China- vanguardngr
Liverpool thrash Napoli as Alisson makes debut- vanguardngr
Three killed in fresh Plateau attack- nigerianeye
I swear I cannot support Atiku for president – Obasanjo- nigerianeye
If you want to have children, stay away from large fish – Prof. Ashiru – Vanguard News- nigerianbulletin
Community Shield team news: Chelsea injury blow, Sarri without six, Mahrez Man City debut – – The Sun News- nigerianbulletin
Akpabio defecting to APC, Buhari's aide confirms- premiumtimesng
Gombe governor declares interest for 2019 presidency- pmnewsnigeria
Three killed in fresh Plateau attack- nigerianeye
Willian returns to Chelsea training – – The Sun News- nigerianbulletin
Ex-Chief Justice Idris Kutigi dies- pmnewsnigeria
Atiku vs Buhari: CAN tells Christians what to do- dailypost
As Buhari's credibility gathers momentum- pmnewsnigeria
Nollywood Producer, Ekejimbe is dead- realchannel65
Davido and Chioma's relationship still very strong – Friends- informationng
At last, Shehu Sani bids APC farewell- sunnewsonline
Insecurity: NAF trains 530 personnel- pmnewsnigeria
Liverpool Coach On Nigerian Messi Form: He Has Been A Threat In Every Game- allnigeriasoccer
How medicine store operator caused death of abortion seeker – Police- premiumtimesng
Davido Chioma are truly in love with each other and nothing can shake them – Friends- informationng
Nollywood Stars Cart Away Awards At AMAA 2018 In Rwanda(Photos)- informationng
Army, police, politicians and politics of checkpoints- sunnewsonline
Is every marriage ordained by God?- vanguardngr
Most Popular (6 hours)
Insecurity: NAF trains 530 personnel- pmnewsnigeria
How medicine store operator caused death of abortion seeker – Police- premiumtimesng
Army, police, politicians and politics of checkpoints- sunnewsonline
Lawyer to late Anita Akapson s family asks Police IG to reveal identities of officers who killed her- pulse
Gov Okowa Mourns Former CJN, Justice Kutigi- leadership
Beyond Fayose: The Future Of Ekiti By Reuben Abati- saharareporters
South Africa:Questions Over Free State Development Corporation s R104 Million VBS Investment- allafrica
Former CJN Idris Kutigi dies in London- channelstv
Ex-Chief Justice of Nigeria, Idris Kutigi Dies At 78- nigerianmonitor
Tonto Dikeh's Sister, Tatiana Dikeh Attempts Suicide- realchannel65
Okowa as new face of democracy- sunnewsonline
Boko Haram attacks two Borno communities, kills many, burns houses- premiumtimesng
Starving, sick Delta IDPs cry out for assistance- vanguardngr
We have so much respect for Ojukwu, BIM tells APGA- vanguardngr
Nigerians urged to reconcile with God for national healing- pmnewsnigeria
Atiku vs Buhari: "You are too corrupt, greedy to lead Nigeria' – Presidency mocks PDP presidential candidate- dailypost
Ooni Of Ife's Wife Stepped On "Osùn", Not Blood- timeofgist
Full list of AMAA 2018 winners- timeofgist
We 'll resist any attempt to impeach Governor Ambode----Lagos Peoples Assembly -(LPA)- nigerianvoice
Rivers 2019: Why we must replace Wike – Amaechi- dailypost
Iraq: UN's Special Representative Kubiš Urges Political Parties to Support PM-Designate in Government Formation within Constitutional Timelines, Calls for Inclusion of Women and Minorities [EN/AR/KU]- reliefweb
Sudan: Sudan: Population Operational Update: South Sudanese Refugee Response (1-30 September 2018)- reliefweb
Udinese s Troost-Ekong Returns To Training Less Than 24 Hours After Horror Show Vs Napoli- allnigeriasoccer
Most Popular (24 hours)
Speak well of Nigeria, Aisha Buhari urges Nigerians- premiumtimesng
Beyond Owning An Oil Well License- leadership
OBI KELELENKE – Latest Nigerian Movies- timeofgist
Tor Tiv Seeks Construction Of New Market For Gboko- leadership
2018 QMA cultural pageant: rebranding culture- vanguardngr
"Dear son, your daddies are proud of you" – Bisi Alimi declares support for Nigerian guy who recently came out as gay- informationng
Music: DJ Kaywise, DJ Maphorisa ft Mr Eazi – Alert- realchannel65
Cristiano Ronaldo Becomes First Player To Score 400 Goals In Europe's Top 5 Leagues With This Goal(Video)- informationng
The Big 5: Nigerian Army warms up for another Python D APC reacts to Shehu Sani's exit from ruling party | More stories- ynaija
Ondo North: PDP candidate lauds party leadership- vanguardngr
Photos Of Slay Queen Holding Her Bridal Shower Inside Mosquitoes Net Goes Viral (See Them)- informationng
Open Heaven 21 October 2018 – You Can Be Perfect- Flatimes.com
Where to Listen to Nnamdi Kanu's Message to Biafrans on Sunday- realchannel65
Tiwa Savage Wears Ordinary Bra As She Poses With Dbanj- informationng
Ini Edo Reveals Real Reason Why Men Enter into Relationship- realchannel65
The Late 5: After long-drawn battle with El-rufai, Shehu Sani quits APC; Lawyer confirms Nnamdi Kanu is alive, to address world press conference Sunday | More stories- ynaija
BBNaija's BamBam Stuns as a Bridesmaid in Friend's Wedding- informationng
Christians and Muslims clash : Fifty-five dead after market row in Kaduna : presidency- vanguardngr
Resist money politics, Buhari urges electorate- pmnewsnigeria
Why Saraki was not impeached –Bwacha – Newtelegraph- nigerianbulletin
Polio victim trends online after completing compulsory NYSC program, Nigerians react- informationng
Actress, Shola Fapson reacts after she was offered a huge sum of money to act in a porn movie- informationng
How Reuters, Economist, others 'baffled' me with question on Atiku, Buhari's corruption war — Lai Mohammed – Premium Times Nigeria- nigerianbulletin
Timi Dakolo reveals the secret of his successful marriage- informationng
Northern political leaders divided over Buhari, Atiku- sunnewsonline
Oshiomhole, APC used and dumped Shehu Sani – Omokri- nigerianeye
BamBam slays as a Bridesmaid in Friend's Wedding (Photos)- informationng
2019: Fresh trouble for Atiku in Southeast- thenationonlineng
Nollywood producer, Chris Ekejimbe, is dead- premiumtimesng
Ex-Chief Justice Idris Kutigi dies- thenationonlineng
A Russian love song brought Me Fame In Dubai—PTERPAN- vanguardngr
Nigeria:Cyber Thieves Raid Banks, Inflict Losses- allafrica
Most Popular (a week)
Uganda: Rights in exile policy paper: Uganda s refugee po the history, the politics, the way forward - October 2018- reliefweb
Family kicks as police gun down ex-finance minister s daughter- nigerianvoice
[LEAKED AUDIO] "I'm Tired Of Writing Statements" – Fayose Blows Hot From EFCC's Custody- nigerianmonitor
The Big 5: Security chiefs raise alarm of stockpiled arms by politicians ahead of Nigeria's population now 195.9 million – UNFPA | More stories- ynaija
Open Heaven 15 October 2018 – Learn To Trust God- Flatimes.com
LIFESTYLE: How to live long and beat chronic diseases — Professor- premiumtimesng
Listen To Fela's Music For Solutions To Nation's Problems, Falana Advises Presidential Candidates- leadership
See Photos From The Wedding Ceremony Of The Ooni Of Ife And His New Bride Evangelist Naomi – Nigerian Entertainment Today- nigerianbulletin
Defaulters of local content law to lose 5% on contract sum- sunnewsonline
Afenifere reacts to Buhari's travel ban on some Nigerians, reveals what president is turning country into- dailypost
Healthy lifestyle critical to productivity – LG- thenationonlineng
Libya vs Nigeria: Check Out How Super Eagles Coach, Gernot Rohr, Reacted To Throwing Away A Two Goal Lead- informationng
Entertainment Roundup: Roseanne Barr reacts after she is knocked out from 'The Conners'; Idris Elba in talks to feature in 'Cats' | More stories- ynaija
Listen to Fela's music for Nigeria's solutions, Falana advises presidential candidates- nigerianeye
Lawyers caution FG against travel ban- vanguardngr
OKON LAGOS – Nigerian Nollywood Comedy Movies- timeofgist
Ooni of Ife's new bride, Prophetess Naomi Oluwaseyi's website crashes due to traffic and her Instagram followers doubles- informationng
Kanayo Kanayo completes law studies, Agbani welcomes first child, other top entertainment stories last week- premiumtimesng
Tiwa, Simi, Teni And Two Ladies Who Should Join 2Baba At "Buckwyld n Breathless: Power Of 1" – Nigerian Entertainment Today- nigerianbulletin
Entertainment Roundup: Drake opens up about his feuds with Pusha-T and K Pete Davidson and Ariana Grande 'split' | More stories- ynaija
Ex-Kogi SSG denies writing petition against FMC Lokoja- politicsngr
Actor IK Ogbonna's Wife Sonia Returns To Instagram And Removes 'Mrs Ogbonna' From Her Handle – Stella Dimoko Korkus- nigerianbulletin
Passion led me into writing books — Umerie- nigeria70
Music: LAX ft Yemi Alade – Bad- realchannel65
Buhari s travel ban will hurt economy - Atiku- nigerianeye
Music: Reminisce – Burushaga- realchannel65
Travel ban: I have been acquitted, says Bafarawa- thenationonlineng
SIDE CHIC GANG – New 2018 Latest Nigerian Movies- timeofgist
3 Movies Screened At 2018 Korean Film Festival- leadership
Lady Gaga Opens Up About Sexual Assault and Mental Health in Vulnerable ELLE Women in Hollywood Acceptance – Ella News- nigerianbulletin
News24.com | Media, advertisers and marketers must collectively combat fake news - Sanef, MMA- news24
A Better Husband – Latest Nollywood movies- timeofgist
Decor Nigeria Michel Akiki Architectural Design host an Interior Design Fabric Open House- nigeria70


text:
Prosperity Classes
9 a.m. Begins Jan. 8. The Center for Spiritual Living, 109 East 52nd St. The Center for Spiritual Living, Savannah is pleased to begin its academic schedule. Our new classes are based on the book: "Spiritual Economics" by Eric Butterworth. Space is limited to 10 students per class to maximise the learning experience for all. The book is available online at Amazon.com and we will also have copies available to purchase on the first night of class if you email a request for a copy. An evening class will also be offered beginning Jan. 9 at 7 p.m. The cost is $160 for eight sessions. This class is suitable for ages 18 and up. For more information, call 912-335-1905.
Genealogy Class Forming
10-11:30 a.m. Jan.17-March 7. Southwest Chatham Library. Join us for this free, comprehensive 8-week program which will give you the skills to create a step by step work plan to research your family history and enable you to leave a lasting legacy for your descendants. Thursdays, Jan. 17 through March 7 from 10-11:30 a.m. You must register by calling 912-925-8305.
Habitat Savannah announces dates for Homeowner Application process
Habitat for Humanity Savannah is opening its 2013 Habitat Homeownership process to pre-qualification applications on Jan.14, 2013. Eligibility is limited to Chatham County residents who have completed and submitted their applications prior to the 5 p.m., Jan. 21, 2013 deadline. Applications submitted following that deadline will not be considered.
Applications are available electronically at www.habitatsavannah.org/Apply.html or by visiting the organization's 1106 E. 70th Street office. A receptacle containing applications is placed on the organization's front porch during the pre-application acceptance period - Jan.14 through Jan. 21.
Book Sale
9 a.m.-12 p.m. Jan. 5. Humane Society parking lot, 7215 Sallie Mood Drive. With the Holiday Season behind us, the Humane Society for Greater Savannah is clearing their shelves and having a book sale. The "pick of the litter" thrift shop will also offer discounts on many items during the book sale. All books are 25 cents.
Cooler in Pooler Resolution 5K and 15K
8:30 a.m. Jan. 5. The Second Annual Cooler in Pooler Resolution 5K and 15K is Jan. 5. Registration is now underway at www.active.com. Individual cost is just $27 for either the 5K or 15K through Jan. 3; $30 after Jan. 3 and 25 percent of the proceeds go to Ronald McDonald House Charities® of the Coastal Empire. For more information contact email@example.com or 912-748-0110.
Master gardener class forming
A new master gardening class is forming and set to begin Jan. 22, and continue on Tuesdays and Thursdays until April 4. The class time will be from 9 a.m.-12:30 p.m. and most classes will be in Lake Mayer's community room. Registration is now being accepted by Dave Linvill, Chatham County extension agent. Contact him at 912-652-7981 or firstname.lastname@example.com.
Yates Astro Resolution Race
7:15 a.m. Registration, 8 a.m. Start. Jan. 5. Bethesda Academy, 9520 Ferguson Ave. The Women's Board of Bethesda will present the Yates Astro Resolution Race, a 5K trail run and walk through scenic environments at Bethesda Academy. The course includes 3.1 miles of roads, grass and gravel paths following the marsh, tree lines, cattle pastures and an organic farm. All proceeds benefit The Women's Board of Bethesda for the purpose of enriching and improving the lives of the boys at Bethesda Academy. Register online at www.active.com or at Fleet Feet Sports Savannah, 3405 Waters Ave. Race packet pickup is Jan. 4 at Fleet Feet Sports.
Grief 101 support group
Begins Jan. 8. Seven-week morning or evening adult educational/support group offering tools to learn with loss meets Tuesday mornings from 10-11 a.m. or Thursday evenings from 6-7 p.m. Next groups begin Jan. 8, 2013 at Full Circle, 450 Mall Blvd, Suite H. Support is offered free of charge as a service of not for profit Hospice Savannah. Help is just a phone call away, 912-303-9442.
Children's' Grief Group
Begins Jan. 8. Seven-week structured educational support group for children ages 6-17 offering support and coping tools and utilizing play and activity to learn to live with loss. Next group begins Tuesday, Jan. 8 from 6- 7 p.m. at Full Circle, 450 Mall Blvd., Suite H. Support is offered free of charge as a service of not for profit Hospice Savannah, Inc. Pre-registration is required: 912-303-9442.
Bethel Missionary Baptist Church Anniversary
7 p.m. nighty Jan. 2-Jan. 4; 5 p.m. Jan. 5; 4 p.m. Jan. 6. Bethel Missionary Baptist Church, 5863 Ogeechee Road. Pastor Jerome Baker and the Bethel Church family are celebrating the 210th year anniversary of the church Jan. 2-6. The services are at 7 p.m. Jan. 2-Jan. 4, a anniversary musical will be presented Jan. 5 at 5 p.m. and the celebration will culminate on Jan. 6 at 4 p.m. All are welcome, to RSVP call, 912-596-1532.
Stay At Home Tea Gospel Musical
5 p.m. Jan. 5. St. Peter's AME Church, 626 Staley Ave. Happy New Year- It's Another New Year and it's time for another Stay At Home Tea Gospel Musical Sponsored by The West Savannah Area Women's Missionary Society. The event will be held Jan. 5 at St. Peter's AME Church. Featuring the Uvalda Community Choir, Uvalda Georgia Pleasant Grove AME Church Sons Of Allen.
Call for artists
Lutheran Services of Georgia invites you to bring out your artistic creations for the Charity Art Auction scheduled for Feb. 28, 2013. We welcome 5 x 7 submissions - whether they be photographs, paintings, watercolors, oils, ceramics - for judging. Other sizes to be auctioned are also welcome.
Items need not be framed. Proceeds of the Art Auction will benefit Savannah programs. Submissions may be dropped off at 6555 Abercorn St, Suite 200, by Feb.7, or call Katherine McKenzie at 912-353-8875, ext. 112.
January featured artists at Gallery 209
January. Gallery 209, 209 East River St. The Gallery 209 featured artist for the month of January 2013 is Melissa Schneider. Melissa is one of our newest members and is a local photograph encaustic artist. She applies layers of encaustic wax over photographs giving them a great depth and beauty. Her subject matter is mostly local historic scenes of Savannah and Beaufort, we hope you will drop by and see her beautiful work. Open daily from 10:30 a.m.-9 p.m. For more information, call 912-236-4583 or go to, www.gallery209savannah.com
Submit your events to firstname.lastname@example.com


text:
Twenty-three "First Day Hikes" are planned for New Year's Day at Oregon State Parks, including three in Southern Oregon.
First-day hikes have become a nationwide tradition, with 657 state parks in all 50 states sponsoring a New Year's Day hike this year.
Valley of the Rogue State Park will host its first such hike, along with Harris Beach State Park in Curry County and Sunset Bay State Park near Coos Bay.
The ranger-led hikes are free, and so is parking that day.
The Valley of the Rogue hike will run from noon to 3 p.m. and cover about three miles of the Rogue Valley Greenway path along the banks of the Rogue River.
"It's open to everyone," says Pete Reinhardt, the park's manager. "With the pavement, it will be appropriate for kids. A lot of families use strollers on it."
Hikers will meet at the program area of the park, which is off Interstate 5 at exit 45B.
The hike, led by Ranger Lyle Scott, will focus on riparian vegetation and local birds, Reinhardt says.
Dogs are welcome on the hike but must be on a leash, and owners must carry doggie bags to collect waste. Hikers are encouraged to bring a water bottle, snacks and binoculars.
Because this is Valley of the Rogue's first hike under the program, park rangers have no idea how many people will take part, Reinhardt says. Rangers would like participants to RSVP so they will have an inkling of what the attendance will be, he says.
Hikers can RSVP by calling 541-761-1673 or emailing email@example.com.
At Harris Beach State Park outside of Brookings, an easy, 1.5-mile hike will travel between Cape Ferrelo and House Rock. It will cover what park officials call excellent vantage points to view gray whales migrating from the north Pacific to Mexico for winter calving.
The hike is scheduled from 10 a.m. to 11:30 a.m., with parks officials providing hot cocoa and light snacks at the end of the hike. Children and dogs are welcome, but dogs must be kept on a leash no longer than 6 feet, and owners must carry doggie bags to collect waste.
It is the Oregon Coast in January, so expect rain. Call ahead at 541-469-0224 to check for any cancellations.
At Sunset Bay State Park outside of Charleston, an easy, four-mile hike will traverse the Oregon Coast Trail from the gazebo at Sunset Bay to the gardens at Shore Acres State Park, then back to the Sunset Bay beach area.
The hike is scheduled to begin at 8 a.m., with kids and dogs welcome under the same restrictions as at Harris Beach. Meet at the Sunset Bay gazebo, and bring rain gear, waterproof shoes and water.
For information, call 541-888-3778, ext. 221.
For information about all the hikes planned in Oregon, www.americasstateparks.org/first-day-hikes.
Reach reporter Mark Freeman at 541-776-4470 or email@example.com.
'First-day hikes' get year off on the right foot
Twenty-three "First Day Hikes" are planned for New Year's Day at Oregon State Parks, including three in Southern Oregon.


text:
Stanley Cup Final: Series even, 1-1
Bruins draw even with win in another OT thrillerBy PAUL LATOUR
The Sports Xchange
June 16. 2013 12:14AM
CHICAGO - Daniel Paille beat Chicago goalie Corey Crawford with a wrist shot in overtime and the Boston Bruins evened the Stanley Cup Final at one game each with a 2-1 overtime victory over the Blackhawks on Saturday night.
Paille, who assisted on Boston's first goal, scored 13:48 into overtime, three days after the teams battled through a three-overtime opener.
Game 3 is Monday in Boston.
The Blackhawks had their six-game home winning streak snapped and the Bruins improved to 6-3 on the road.
The first eight minutes of overtime offered more action than the entire third period and most of the second as both teams tried to avoid another lengthy night.
It was the third straight OT game for the Blackhawks, who beat Los Angeles 4-3 in Game 5 of the Western Conference finals before a draining 4-3, three-overtime victory over Boston in Game 1 on Wednesday night.
Jaromir Jagr nearly ended it 90 seconds into overtime, but his wrist shot that beat Crawford to the glove side ricocheted off the post and out of play. The 41-year-old Jagr had a team-high four shots on goal during regulation.Crawford again stymied the Bruins four minutes later with a point-blank stop on Chris Kelly and then covering the puck in the crease on a deflection.
The Bruins kept the pressure up as a shot by Nathan Horton trickled wide of the left post, and Milan Lucic backhanded a shot wide from the top of the crease.
Chicago missed its best chance to end it about seven minutes into the extra session when a blast from Patrick Sharp sailed just high of the net over Boston goalie Tuukka Rask's glove.
This marked the second consecutive year the first two games of the Final went to overtime. Before that, it hadn't happened in 61 years.
A scoreless third period provided few chances on either side. Crawford, who had 18 saves through regulation, stopped a charging Brad Marchand early in the period.
Crawford also turned aside a low shot from David Krejci a short time later as the Bruins continued to build after being outshot 19-4 in the first period. That was Krejci's first shot, which didn't come until nearly 45 minutes into the game.
Rask made 27 saves in regulation but only eight after the first period.
Neither team was able to take advantage of power plays. The Bruins killed off all three Chicago chances, remaining perfect on their past 22 consecutive penalties.
Chicago, which had allowed a power-play goal in its past two games, killed off all three Boston man-advantages.
The Blackhawks did everything right in the first period and got on the scoreboard first.
Sharp scored from inside the right faceoff circle after a scramble in front of the Bruins goal. With Rask down in the crease, Sharp lifted a high shot into the net at 11:22 for his ninth goal of the playoffs.
About a minute later, Chicago thought it had added to the lead on a wraparound attempt by Jonathan Toews. But it was ruled no goal on the ice and the call stood upon video review.
The Bruins were outshot 19-4 in the first period but almost scored about nine minutes in. However, a Jaromir Jagr shot from the slot was gloved by Crawford.
Rask stopped 18 shots in the first period, after making a career-high 59 saves in the opener on Wednesday night. His most impressive save came six minutes in when he robbed Nick Leddy with a quick glove.Despite their limited scoring chances, the Bruins were able to tie it 1-1 in the second period on a Kelly goal at 14:58. A turnover behind the Chicago net was grabbed by Paille, who skated to the front and fed Kelly for his first goal of the playoffs.Marchand nearly gave the Bruins the lead late in the second period on a short-handed chance. He beat Brent Seabrook to a loose puck in the Blackhawks' zone, but his shot hit the post to Crawford's left.Notes: Horton was back in the Bruins' lineup despite leaving Game 1 on Wednesday night in the first overtime and not returning with a reported shoulder injury. ... Sharp (6) and Marian Hossa (5) had more first-period shots than the Bruins (4). ... Teams that have taken a 2-0 lead in the Stanley Cup Final are 32-3 all time, but two of those losses came in the past four seasons. The Bruins lost the first two games to Vancouver before rallying to win the 2011 Stanley Cup. ... There hasn't been a split of the first two games in the Final since 2004.


text:
pelechati avait besoin d'un nouveau design de Autre et a lancé un concours sur 99designs.
Un gagnant a été sélectionné de 12 designs proposés par 7 designers freelance.
I'm currently the community manager for apture.com. Previously I did project management at Yardbarker.com, merchandising for eBay.com and Apple Computer.
I love the web and user experience. This is my first independent project and I'm really excited about it.
Real Estate Agents and potential buyers who come to an Open House looking to check it out. We want to make this app so simple that agents cant live without it and users are surprised and delighted as they enter there personal information into it.
Goal: To create an application for the iPad that allows a real estate agent to hand the iPad to a potential customer and collect their personal information before being shown the house/apartment.
-Be elegant in the design. Think large buttons to make it simple, but have the backgrounds be very warm so people feel excited to enter their info.
After answering a few questions an email will be automatically sent to the buyer thanking them for stopping by and the agent will receive an email with the info that just got entered.
Be creative, I'm attaching a powerpoint of what I think the flow should be but I'm open to suggestions. The key here is to make it super simple to use, but extremly efficient and powerful for the agent.


text:
TheHammer24 wrote:Additionally, the $1,160 of the $1,200 that starts and stays here is spent on those things as well. No matter how you shake it, the availability of cheap labor benefits us immensely. So we innovate. We create new value. And most of that value stays here and is spent on other things, most of which benefits Americans who sell to those that made the money.
I never said cheap labor is a bad thing. If we suddenly stopped importing cheaply made foreign goods, it would be disastrous for the economy. I'm well aware of that. Cheap labor is never going away and, in fact, that was the point of my first post.
What I'm saying is that the US is filled with low skilled workers. Those workers used to be able to get jobs in factories that paid a "family sustaining" wage. Those people are no longer going to be able to find jobs in the manufacturing sector. Some of them can move into the service industry or other areas where workers without any specific skills can get employment, but those jobs rarely pay as well as factory work. Some of them can "go back to school", but with thousands of college grads unemployed/underemployed, will that really solve anything? I'm not advocating we revert back to digging holes with spoons so please lay off the pointless sarcasm.
What I've been saying is that we're going to be in trouble economically until we can find a way to employ those millions of workers again. It's not going to be in manufacturing though, so we better "innovate" a way to quickly find jobs for unskilled laborers. I don't know where those jobs are coming from and don't pretend to have an answer. I'm just stating what I feel is one of the biggest issues looming in our future.


text:
My guestimate was that it would take about 48 hours for this story to come out. And here it is, right on schedule:
During the years that Senator Joseph R. Biden Jr. was helping the credit card industry win passage of a law making it harder for consumers to file for bankruptcy protection, his son had a consulting agreement that lasted five years with one of the largest companies pushing for the changes, aides to Senator Barack Obama's presidential campaign acknowledged Sunday.
In other words, when Biden gets up there and talks poignantly about the typical American family sitting around the kitchen table late at night worrying how to pay its bills, and then twists the dagger on John McCain's seven kitchen tables, it's a great line politically. but....
It's a total crock, and Biden ought to be embarassed. The sad truth is that Biden not only supported but aggressively pushed for a conservative, anti-working class bill that made life a lot harder for that family that he talked about. Here's a little of the background:
The financial services industry began seeking relief from Congress in the mid-1990s from an increase in bankruptcies that was cutting into its profits. Its initial support came from Republican lawmakers, who repeatedly introduced bills to make it more difficult for consumers to erase their debts. During that time, executives at MBNA, which was bought in 2006 by Bank of America, began donating heavily to both major political parties and many national politicians, including Mr. Biden.
Consumer advocates say that Senator Biden was one of the first Democratic leaders to support the bankruptcy bill, and he voted for it four times — in 1998, 2000, 2001 and in March 2005, when its final version passed the Senate by a vote of 74 to 25.
Travis Plunkett, legislative director of the Consumer Federation of America, a consumer group that opposed the bill, said that Senator Biden had provided a "veneer of bipartisanship" that eventually helped the credit card companies win over other Democrats. "He provided cover to other Democrats to do what the credit industry was urging them to do," Mr. Plunkett said.
Making this even worse, as the story emphasizes, is that Biden's son Hunter was signed up by the company in a lucrative lobbying deal. On the surface, that really stinks -- but watching these things unfold over the years, I predict that Hunter Biden is probably just another candidate for the Sleazy Relatives Hall of Fame, a long list that includes neer-do-wells in both parties like Roger Clinton, Neil Bush, Billy Carter and Donald Nixon, and these usually only cause minor collatoral damage.
Of course, Delaware is also the credit-card capital of the world, and so you can make the case that this is another example of another fine political tradition -- supporting the big-money interests (and, yes, employers) in your home state over the broader interest of people in all 50 states. That was arguably sort-of OK, maybe, when Biden was just a Delaware guy, but now he's got to explain his bankruptcy vote to families around the kitchen table in places like the hard-hit Mon Valley or California's Central Valley. Good luck with that.
So why would Obama go ahead and pick someone with this kind of baggage? The answer is simple: The Republicans can't attack Biden on this issue, because their record is even worse. John McCain voted for this same sad sop to corporate America (Obama opposed it) and as the Times article notes, executives at MBNA (which has since been bought by Bank of America) were even more enthusiastic about supporting George W. Bush than backing Biden.
Can you imagine a negative ad with satanic music, grainy slow motion shots of Biden, and a narrator saying: "Joe Biden: Sleazy Tool of the Same Corporate Interests That We Support 100 Percent of the Time"? Of course not -- they can't attack Biden for backing something that McCain and Bush also supported. It just means that now Obama, who was on the people's side on this one, now won't be able to hit this issue as hard as he could.
No wonder the greatest enthusiam for Biden is not from party activists but from the usual chattering classes in D.C. As Glenn Greenwald wrote:
Ever since it became clear that Obama would be the likely nominee, the political establishment has been demanding of him more and more proof that his "change" rhetoric is just that -- rhetoric, and not anything meant as a genuine threat to the prevailing order of things. Obama, arguably out of political necessity, has repeatedly obliged, eagerly trying to offer proof that he is no threat to them, and the Biden selection is but the latest step in that campaign of reassurance. In sum, Biden is a reliable supporter of virtually every prevailing bit of conventional wisdom within the American elite political consensus, which is why his selection has been widely praised by the establishment, whose principal concern is that their fiefdom not be disrupted and that their consensus not be challenged.
The bottom line is that Biden's shameless kowtowing to the credit card companies won't hurt him or Barack Obama. The only ones getting hurt are you and me, and that poor struggling family at the kitchen table.