Intel Woodcrest performance claim a fraud
Update: Please check this page on closer examinations of Intel's claims on FP, Web, Java, HPC and Application performance.
Intel setup a web page showing TPC-C result of Woodcrest comparing to that of an Opteron. The 3GHZ Woodcrest server with 64GB FB-DIMM memory was running Microsoft Windows Server 2003 Enterprise x64 Edition, Microsoft SQL Server 2005 Enterprise x64 Edition. It achieved a score of 169,360 tpmC, $2.93/tpmC. The Woodcrest machine will be available by November 22, 2006.
In comparison, Intel gave the following details of an AMD system:
"Dual-Core AMD Opteron* Processor Model 285 based platform details: HP Proliant DL385 G1* server platform with two Dual-Core AMD Opteron* processor 2.60GHz, 32GB memory, Microsoft Windows Server* 2003 Enterprise x64 Edition, Microsoft SQL Server* 2005 Enterprise x64 Edition. Referenced as published at 113,628 tpmC; $2.99/tpmC; Availability Date as listed in the submitted report is May 5, 2006. Results at http://www.tpc.org/tpcc/results/tpcc_result_detail.asp?id=106032001"
Intel's message to potential customers is clear: Woodcrest is 50% faster than Opteron in database performance. Remember, database is about reading from/writing to disks.
If you go to the executive summary of the HP DL385 at the TPC-C site, you can see that the Opteron machine was running Windows Server 2003 Enterprise x86 Edition SP1 and Microsoft SQL Server 2005 Enterprise (x86) Edition SP1. These are 32 bit versions of Windows Server 2003 and SQL server 2005. If you have any doubt on the OS and SQL server used, read the full disclosure of the DL385 TPC-C results. If it was a 64 bit OS, you would see information on Windows\SysWoW64\, but on the HP DL385 page, you only find Windows\System32\. So, the HP DL385 was running in very inefficient 32 bit PAE mode.
Why did Intel change x86 to x64 for the HP DL385 benchmark description? Let's first rule out the possibility that Intel did this accidentally. 86 and 64 are too distinctive from each other to be confused with one another.
If Intel told you that the Opteron result was actually from 32 bit, you would ask immediately: wait a minute, how about running Opteron in 64 bit and also with 64GB ram (instead of 32 GB PC2700 RAM)?
We know that Opteron performs 20-40% better in 64 bit mode -- which will immediately wipe out Intel's claimed performance lead. So, cunny Intel changed 86 to 64.
By changing x86 to x64 for the Opteron tests, Intel was trying to create an impression that it has a big lead over AMD under similar conditions. This is similar to Intel's previous acts of putting a faster harddrive in its own server and claiming its CPU is faster.
In fact, not only the Woodcrest was running 64 bit OS, it costs $158,000 or 47% more than the Opteron system. The Woodcrest system has 64GB RAM and 48 MSA 60 StorageWorks with 576 SAS drives at 15K RPM. The Opteron system has 32GB ram and 18 MSA 30 StorageWorks wtih 380 SCISI drives. There is a huge difference in storage performance between the two. TPC-C is a database benchmark. A database reads data from disk to memory and write data from memory to disk.
Independent results show Intel's Woodcrest 10% slower than Opteron at the same clock under 64 bit mode.
Fraud: Any act, expression, omission, or concealment calculated to deceive another to his or her disadvantage. (Merriam-Webster’s Dictionary of Law, 1996).
Intel satisifies the above legal definition 100%. More on this (SpecFP_rate, SpecWeb2005 ...).
posted by Sharikou, Ph. D. @ 5/24/2006 09:47:00 AM
44 Comments:
Even at 32 bits, only 2% better performance with a 15% faster clock.
Intel seems really desperate to do something like this.
When their customers realise promised Woodcrest performance is nothing but hot air compared to Opteron shit is gonna really hit the fan.
Even at 32 bits, only 2% better performance with a 15% faster clock
You are right. TPC-C is a database benchmark, it is highly dependent on the amount of memory and the number of disk spindles.
Thanks for the informative blogs as I do enjoy reading them on a regular basis. :)
What level has Intel stooped to? They come up with wacky benchmarks for a FUTURE product comparing it to a lesser available from AMD available NOW? And skew the benchmarks? Just... WOW.
Are IT managers really that stupid where they will believe those benchmarks? What's that going to do to their customers they sell it to when they figured out they paid more for a lesser machine? hmm...
Intel lied about a benchmark or misprinted AMD's results?!?! SAY IT AIN'T SO MAN, SAY IT AIN'T SO! Seriously, is anyone surprised by their actions of this nature anymore?
All I got to say is it's more fuel to the fire, and that bonfire roasting is a GUILTY ruling against Intel for anti-trust violations in the AMD vs. Intel anti-trust case. So keep it up Intel, do what you do best, keep digging your own grave. Payback will be 100 fold upon you, this I predict.
"Even at 32 bits, only 2% better performance with a 15% faster clock"
show me...
""Even at 32 bits, only 2% better performance with a 15% faster clock"
show me..."
Intel Woodcrest - $2.93/tpmC.
AMD Opteron - $2.99/tpmC
.06/2.99 = 2%
Good blog! Illustrative of what Intel really is. Everyone in this computing industry should come to read this.
Hey, Sharikou,
I really think you did an excellent job to dig those things out. People says : the devil is in the details. That is exactly what Intel does, and you dig them out for me..
I appreciate you set up this blog. Keep up the work.
"
Intel Woodcrest - $2.93/tpmC.
AMD Opteron - $2.99/tpmC
.06/2.99 = 2%"
why did u include price for plain performance comparison???
why did u include price for plain performance comparison???
I think the other reader has a point. While price/performance is not the measure of performance per se, it should be taken into account. TPC-C is a databae benchmark, it is heavily dependent on storage performance. If you look at the total cost, the Intel Woodcrest system costs half $1 million. The Woodcrest CPUs only costs a few hundred dollars, a tiny fraction of the total. A fair comparison should be done with the same amount of money spent on storage, memory and with the same software (OS and database).
Even going to 64-bit won't greatly increase the DL385's score.
http://tpc.org/tpcc/results/tpcc_result_detail.asp?id=105042002
http://tpc.org/tpcc/results/tpcc_result_detail.asp?id=105110401
Comparing these two DL585s, the second system goes from 2.2DC to 2.4DC, 64GB memory to 128GB, SQL Server 2000/Windows 2003 32-bit to SQL Server 2005/Windows 2003 64-bit and barely sees a 10% gain.
If you look at this:
http://tpc.org/tpcc/results/tpcc_result_detail.asp?id=105110401
The total Opteron system cost was $472K, with a score of 206,181 tpmC. The Woodcrest system cost $496K with a score of 169,360 tpmC. So the quesion is what if one throws more money at the Opteron?
Let me bring more facts in (The person above SHARIKOU is an Intel Fanboy from Tomshardware.com forums, and lately, their entire forum is Intel lovers, so ignore him w/ his false 64-bit post).
The Opteron system is 32-bit servicing 4 clients and 8 processors that happen to have HyperThreading, so it is servicing 16 threads with 4 processors.
The Woodcrest system is 64-Bit servicing 8 clients but only 2 processors w/ HyperThreading so 4 threads.
The Opteron system is servicing 4x the amount of threads, has DDR1, half the RAM, and under 32-bit.
Those tests are false and don't even use the same set of information than the Woodcrests.
"We know that Opteron performs 20-40% better in 64 bit mode..."
Do you have specific links to back this claim up?
Microsoft's tech doc on AMD64 advantage.
this is pretty serious. did you email intel about this "mistake"?
"The Opteron system is 32-bit servicing 4 clients and 8 processors that happen to have HyperThreading, so it is servicing 16 threads with 4 processors."
um.. there is no hyperthreading in opterons. and FYI, i would consider sharikou to be an AMD fanboy rather than intel fanboy
"um.. there is no hyperthreading in opterons. and FYI, i would consider sharikou to be an AMD fanboy rather than intel fanboy"
Obviously people don't understand what I mean, let me re-iterate:
There is a Server and a Client, the Servers are the Opterons and Woodcrests and both use Xeon's w/ HyperThreading as their Servicing Clients, with me so far?
The Opteron system is 2.6GHz w/ 32-Bit Windows Server 2003 and 32-Bit SQL Server 2005. Got it?
The Opteron system also has only 32GB RAM and is running on PAE mode, which slows performance down. Got it?
The Opteron system's clients were 4 Clients and a total of 8 Physical processors, Single-Core each but they have HyperThreading, so the Opteron 64 system with 2 Dual-Core 2.6GHz Processors is servicing 8 Processors and a total of 16 threads. Understand that?
The Woodcrest system is running 64-Bit Windows Server and 64-Bit SQL Server 2005. Got it?
The Woodcrest system was running with double the RAM @ 64GB as well. Got it?
The Woodcrest system's clients were 8 Clients, but only 2 Processors w/ HyperThreading, so 2 Dual-Core 3.0GHz Woodcrest systems were servicing 2 Processors and a total of 4 threads. Understand?
That is why the test is unfair.
"there is no hyperthreading in opterons"
He's talking about client threads, not server threads. There are total 8 client Xeon processors, with hyperthreading its 16 virtual processors (threads).
Let's try do a comparison of the setup: Woodcrest DL370 TPC-C summary,
Opteron DL385 summary.
You can see the Woodcrest setup uses 50 HP MSA-60 Storage, 576 SAS drives, 64GB RAM. The Opteron setup uses 2 HP MSA-30 SB Storage, 378 SCSI drives, 32 GB RAM. The Woodcrest system costs $496K, the Opteron system costs $338K. There is a big difference in strorage performance for sure. Coupled with the fact the Opteron is running in 32 bit PAE mode. The Opteron was basically fighting with both hands tied to the back. Remember, this is a database benchmark.
I think once we look at the real information, we can find the truth. Head over to my blog to find all the details comparing the 2 configured systems together.
Instead of blaming Intel blame all your favorite AMD server vendors. Intel did nothing but take the highest 2S Opteron result that is currently publically available. The 86/64 mixup is unfortunate and should obviously be corrected but to call this fraud is going to far.
If 64 bit would add 20/40 percent (which it obviously doesn't in this case, given how important storage is) then go tell HP, IBM, Sun or whoever to publish that. Because you really can't expect Intel to downgrade their own systems just to make AMD look better.
The tpcm/$ ratio was disclosed so any serious IT manager would have immediatly seen the difference in RAM/storage.
Why Intel pushed Dempsy up just about four weeks (as rumored) ahead of super chip (as Intel says) Woodcrest start?
May be in reality something wrong is going with Woodcrest?
May be in reality something wrong is going with Woodcrest?
At least some of the units Intel setup for review were having BSODs.
The 86/64 mixup is unfortunate and should obviously be corrected but to call this fraud is going to far.
I proved this was a fraud. Intel's intention was to make two completely different setup (heavily benefiting Woodcrest) look the same to potential customers, so the difference could be attributed to the CPU.
Besides this http://www.realworldtech.com/page.cfm?ArticleID=RWT052306090721&p=1
There are now new Woodcrest preview benchmarks at
http://www.tecchannel.de/specials/english/439566/
http://www.realworldtech.com/page.cfm?ArticleID=RWT052306090721&p=1
http://www.2cpu.com/review.php?id=112
Using different apps (includes server industry SPECmarks, 64-bit multi-threads, etc...) All results are the same, ol' Optys got "fragged"...
Actually, if you look at Microsoft's website, both Microsoft SQL Server 2005 Enterprise Edition and Microsoft Windows Server 2003, Enterprise x86 Edition support x64.
http://www.microsoft.com/sql/prodinfo/overview/whats-new-in-sqlserver2005.mspx
i believe woodcrest is still superior than opteron, but probably not 30% better. intel's "fraud" was just to show woodcrest has 30% performance gain over opterons.
If Woodcrest was really better than opteron, then why is Intel cheating? If they knew they could win a fair race, then they wouldn't have to cheat. The fact than Intel ran a very unfair benchmark just proves in itself that Intel knows it's CPU is inferior.
"The fact than Intel ran a very unfair benchmark just proves in itself that Intel knows it's CPU is inferior."
Not really. You and I know that Intel's cheating, but the public majority don't. Intel is just banking on the fact that the majority people are stupid (which unfortunately has always been true).
Lets suppose the fastest Woodcrest is 10% in average faster than today's fastest Opteron. If Intel honestly showed that, nobody will fall for it because (1) Woodcrest is not out yet, (2) its peformance/price will be worse than Opteron. So to sway the general public opinion, Intel had to pump Woodcrest's performance to 30% better (by cheating or whatever).
Intel's playing a classic "marketing tomorrow's products today" trick that it used to defeat many of its former competitors.
http://www.tecchannel.de/specials/english/439566/
used 2.4 ghz opterons, but there is a 2.8 ghz flavour i think. So why don't they tried that??.
Anyway very interesting site sharikou. Keep up the work.
I can see Intel not wanting to compare to the 4P Opteron result that is faster and cheaper:
TPC-C Throughput: 187,296
Price / Performance: 2.04 US $
Availability Date: 05/31/05
http://www.tpc.org/tpcc/results/tpcc_result_detail.asp?id=105042002
No wonder AMD already has 50% of the 4 Processor Server market!
Why bother with a 2P system that costs more than a 4P system?
DUH!
Before you go off half-cocked and accuse them of fraud, you may want to check the facts.
If you look at the TPC benchmarks, you will notice that there is a DL585 with 64GB of ram and 4 2.8GHz opterons scoring 138k. The DL385 has 32GB of ram and 2 DC 2.6GHz opterons. So the system is quite a bit slower in reality and as such scores only 113k. The difference? The 585 is running x64. If you really think that 40% performance difference is there, you should do the math. 138k - 113k = 25k. 25k/113k * 100 = 22.1%. So, if you add 40% to the original 113k, you get the answer. It was most likely running x64, or there was no benefit to be had. My money is on it being a mistake on the TPC site . I could be wrong. :)
Something else:
Have you checked the performance on SAP? Intel's 'not even released' latest and greatest is barely faster than AMD's platform that came of the drawingboard almost 5 years ago.
___________________________________
2 processors / 4 cores / 4 threads,
Intel XEON Dual-Core, 3.0 GHz, 32 KB L1 cache per core, 4 MB L2 cache per processor, 32 GB main memory:
6,000 SAPS
(See: http://www.sap.com/solutions/benchmark/pdf/cert2706.pdf)
____________________________________
2 processors / 4 cores / 4 threads,
Dual-core AMD Opteron processor Model 275 2.2 GHz, 128 KB L1 cache, 1 MB L2 cache, 16 GB main memory:
4,370 SAPS
(See: http://www.sap.com/solutions/benchmark/pdf/cert0606.pdf)
____________________________________
Intel's Woodcrest is clocked 36.4%faster.
The score is 37.3% higher.
Add in HP's tuning skills (that's actually what these "industry wide recognized" benchmarks have become. Who 'tunes' the best) and an equally clocked 'antique' Opteron scores about the same or better!
And there is more.
What about intel's SPECfp_rate2000 numbers for it's 3GHz woodcrest?
Opteron 285: SPECfp_rate2000 = 85
Xeon 5160: SPECfp_rate2000 = 83
(See: http://www.spec.org/osg/cpu2000/results/res2006q1/cpu2000-20060306-05756.html and http://www.spec.org/cpu2000/results/res2006q2/cpu2000-20060501-05939.html)
Not impressive at all.
Intel, the biggest chipmaker, with a staff that includes the some of the top engineers in the world, could not set up a more fair comparison?
Are we saying these pros did not identify the comparison system before they built the Woodcrest test setup? Did they have to run with a far superior storage system? With twice the RAM? With a fraction of the client processors?
BS. The marketing department told the test engineers what results they needed vs. what competitor. In order to provide those results, the engineers had to skew the test THAT far.
It's simple. Intel is about to lose 50% of the mid-range server market. And everyone knows it.
With superior architecture AMD had to wait 3 years for a wide aceptance in the market. INTEL want to do the same thing, after p4 fiasko in 2 months ???. And what about the eco system that amd was building all this years. Coprocessors in opteron socket is just a drop in the sea. I for one have no doubts in AMD strategy and AMD presence, no matter what INTEL does.
This benchmarking run was set up for one group of people to read: Share Holders. AMD stock is rising and Intel is sliding. So Intel had to put out a result that showed the could compete. And they did not have to worry about the details because most investors seem to just read the headlines. Obviously this benchmark could not stand up to IT professionals. But the real goal of comparing this future product to one you can buy now was to raise the interests of Intel share holds and the stock market.
There sure are some paranoid people here. This clearly was a typo by someone at Intel. The description has been corrected and now says x86. Regardless of that, the original and current descriptions both had the URL to the Opteron system configuration on tpc's website. Why on earth would Intel give everyone a link to the actual system specs and then intentionally post them incorrectly in their description as if they thought no one would notice the difference?
If your tinfoil hats are getting a bit worn out, I've some I can part with at a good price. Just let me know.
Hi. I am Chicagraf0 / Todospara1
A bit of an apology about my accusation that you were using the strong word "fraud" improperly.
I am glad to see you used the very same dictionary quotation I found!
Keep up the good work
This clearly was a typo by someone at Intel. The description has been corrected and now says x86.
Ha ha. I guess AMD lawyers weren't sleeping . But, have Intel addressed the additional issues we pointed out?
If the benchmark is heavily dependant on storage performane, Id say theres hardly a difference between processor choice?
Such a nice one.Thanks for sharing.Its really a good one.
penegra Online
Post a Comment
<< Home