Examining Intel's Woodcrest performance claims on TPC-C, Floating point, Integer, Java, Web, HPC and application
Today, former Enron execs were found guilty on charges of fraud, false statements and conspiracy. Let's revisit Intel's Woodcrest performance claims. I pointed out that
Intel's changing of the Opteron TPC-C benchmark description from 32 bit x86 to 64 bit x64 was a fraud.
Some of the readers said that Intel simply picked up the highest reported TPC-C results for two way servers, Woodcrest and Opteron, regardless of the operating system used. Let's test this assumption on other benchmarks. Let's look at
floating point performance.
For SpecFP_rate_2000, the
highest reported score for 2P 2.6GHZ Opteron 285 was 85 under Solaris 10. Guess what? Intel ignored this result, instead, it uses a
lower Opteron result for Linux with a score of 72.9.
The 3GHZ Woodcrest scored 83 under Linux. The 3GHZ Woodcrest (Linux) was 3% slower than 2.6GHZ Opteron (Solaris). Also, notice that Intel chose the SPECfp_rate_base2000 scores for comparison. The SPECfp_rate_base2000 is for conservative optmization of the benchmark, so it's always lower than the SPECFp_rate_2000 score. For some strange reason, the DELL 2950 Woodcrest server's optimized SPECfp_rate_2000 score was the same as the conservative SPECfp_rate_base2000 score, which may indicate that there were some issues with how the benchmark was done. Any way, Intel was shopping for the lowest Opteron scores. This clearly shows that Intel knew different configurations lead to different results. Had Intel chosen the highest score regardless of OS, the 2.6GHZ Opteron would outperform 3GHZ Woodcrest in SpecFP_rate_2000.
According to
this report, the 3GHZ Woodcrest (Xeon 5160) will be the topmost chip, the next will be the 2.66GHZ Xeon 5150. Intel's topmost desktop chip the Conroe XE will be at 2.93GHZ. This indicates that a 3GHZ Woodcrest will be a cherry-picked chip. According to
this page, the 2.8GHZ Opteron x90 has been in the wild for quite a while now.
For SpecInt_rate_2000, Intel again
changed the OS description of the systems.
The Woodcrest benchmark was done in 64 bit. The
Opteron benchmark was done in 32 bit. This time, Intel changed the Dell PowerEdge 2950's benchmark description from "Microsoft Windows Server 2003 Enterprise x64 Edition" to just "Microsoft Windows Server 2003", making it look like the same as the Opteron test. This is just like
Intel's Woodcrest TPC-C performance claim fraud.
Let's look at another example:
Intel's page on Java performance. Intel used an unpublished Woodcrest test result on a Fujitsu Siemens PRIMERGY server running Windows Server x64 with BEA JRockit 5.0 P26.4.0 JVM. But for Opteron, Intel decided to use the score from a Tyan S2895 server with two 2.6GHZ Opteron and a SATA drive, the score was only 54490. However, from
www.spec.org, we can find a
Fujitsu Siemens PRIMERGY server with two 2.4GHZ Opteron 280 (running Linux, JRockit 5.0 P26.0.0) scoring 61155. Again, Intel was shopping for the lowest Opteron scores.
Let's look at yet another example:
Intel's page on web performance. An IBM 3GHZ Woodcrest server got a SpecWeb2005 score of 9182. Mysteriously, there is no Opteron scores on this Intel page. However, going to
www.spec.org, we quickly found this
2.4GHZ Opteron 280 server achieving a score of 8394. The 3GHZ Woodcrest has a 25% clockspeed advantage but only 9% performance lead over the 2.4GHZ Opteron.
Let's look at one more example:
Intel's page on application performance. For the SunGard ACR test, Intel sent two servers to a company called
Principled Technologies. One was an Intel built Opteron server and one was a Woodcrest server. Not surprisingly, the Woodcrest won the benchmark. The
details of the benchmark is in this PDF file. The motherboard Intel chose for the Opteron was an UNIWIDE SS232_128_03 model using Nvidia NF4 chipset. One has to ask why Intel built the Opteron server themselves instead of using a proven server such as SUN's X4200 or HP DL385. We know server performance does vary from system to system. Not only Intel built and configured the Opteron server, it also provided the Intel compiled test application "SunGard ACR Intel Demo 2.5". It is unclear how Intel optimized this test application, but in a previous report (later removed), it was reported SunGard ACR is significantly faster for Xeon when compiled with Intel C++ compiler.
The more we examine Intel's presentations, the more problems we find. Looking at
Intel's HPC performance page, pay attention to the fluid dynamics results (Fluent). Intel used a Woodcrest 3GHZ (2530.44) against an IBM 2.2GHZ LS20 Opteron blade (2014.34) , with the Woodcrest having 36.4% clockspeed advantage and 26% performance lead. However, if you go to
the Fluent full results page, you can see there are quite a few Opteron results better than the 2.2GHZ IBM LS20 Opteron blade. In fact, there is a 2.6GHZ IBM LS20 Opteron blade scoring 2404.72. Using this result for 2.6GHZ Opteron, the 3GHZ Woodcrest would have only 5% performance advantage, despite 15% clockspeed advantage. Actually, both results show that Woodcrest being 10% slower than Opteron clock for clock, in agreement with
our previous analysis. One can imagine Intel tabulated the Fluent benchmark results, and decided to use AMD's entry level 2.2GHZ Opteron 275 for comparison against the topmost Woodcrest 3GHZ (Xeon 5160). On the same HPC performance page, for "Finite Element Analysis for Crash Simulation", Intel also picked a low score for Opteron, despite existence of better Opteron results (see user comments).
So, why did Intel change the Opteron TPC-C description from x86 (32 bit) to x64 (b4 bit)? Why did Intel consistently choose the lower Opteron scores for comparison?
The answer is obvious, to create a false impression that the Intel CPU is much better.
Fraud:
Any act, expression, omission, or concealment calculated to deceive another to his or her disadvantage. (Merriam-Webster’s Dictionary of Law, 1996).Intel's behaviour satisfies the above legal defintion 100%.