











Comments     Threshold


RE: A little late...
By nichow on 9/28/2009 12:50:59 PM , Rating: 3
Actually the methodology used in the article you reference isn't very good. Some of the more obvious problems are:

1- The author never specifies how the machines are staged (at least that I could find). He only indicates that they were dual and quad core machines. To be a valid comparison each OS tested would need to be installed on the same or identical machines to compare apples to apples. Even matching cpu's in two different machines is not sufficient because the rest of the machine config can change perf test results dramatically.

2- He only mentions tracking the clock time for test completion and then draws conclusions without supporting evidence. He would need to at least capture cpu, disk, and ram metrics to determine why the test results were different. This would allow a more convulsive determination as to what the bottleneck is. It may be ram or storage that is the bottleneck and not cpu core scaling. Making a conclusion without appropriate data, isn't very accurate.

3- The selected workload is suspect. Using the Visual Studio developer version of SQL is a bad workload. This is designed only for development and testing purposes, it is not a high performance SQL server that will utilize multiple cores effectively. A real SQL server should be used. As far as the message store or media center workloads go, they would have to be analyzed to determine if their behavior would test cpu scaling, but without collecting any of the metrics i mention above, this determination would be difficult.

I don't know how valid the article's conclusions are, but without more detailed metrics, not even the author can know.

