Benchmarkmärchen mit Fortsetzung...

Zum Aufbau dieser Thread-Seite

AMD and Intel benchmarketing exposed

Part 1 of 3: The changing nature of the game
Von miesen Spielregeln und gezinkten Karten
By Jack Russell: Dienstag 27 Mai 2003, 12:43
THERE'S BEEN A LOT of brouhaha lately over benchmarking what with the uproar between NVIDIA and FutureMark and the inevitable mud-slinging that's been going around. But almost lost within that confusion was a grumble of discontent that threaded its way through various reviews when AMD launched the AthlonXP 3200+ earlier this month.

Geschichte, als der P4 kam
AMD and Intel have gone round and round over the fairness of various benchmarks since Sunnyvale first began offering a product that could compete with Santa Clara, but things are only getting worse on the P4 vs. AthlonXP front. When the Pentium 4 was introduced the Athlon pounded it all over the ring, winning almost every benchmark available and making a mockery of Intel's "best available performance" whimpers.

P4 erstmals vorne
This situation didn't change much for the entire first year of the Pentium 4's availability. The P4 on its Willamette core didn't ramp very quickly and market demand for i850 was quite low, with Intel's SDRAM-based version of i845 doing nothing to impress the performance/enthusiast-oriented market segments. The P4 did briefly grab the performance crown when it launched at 2 GHz in late-August (albeit at a much higher price) but by October AMD's 1800+ had yanked the title away again.

Schwächeanfall der XP’s
The next eight months saw the AthlonXP falling farther and farther behind as the P4's speed rocketed upwards, and its only been recently that AMD was able to first approach and then strongly compete with Intel's highest-end P4's. Today's AthlonXP 3200+ is competitive with the P4 3 GHz (though not to the same degree that the 1900+ was over the P4 2 GHz) but the nature of the competition is very different. Then there were few benchmarks (save for the ubiquitous Quake 3) that the P4 did win, now both CPU's seem to be entrenched in their own benchmark camps, each possessing its hoard of guaranteed wins, with an overlap in the middle consisting mainly of games.

Der Teufel steckt im Detail
It's not that either CPU lacks for benchmark wins or that one CPU only wins in office performance while another wins in gaming, but that each CPU has tests in every category that it can take.
The AthlonXP performs much more competitively against the P4 when encoding DVD's with Xmpeg, but tends to lose dramatically in Flask, while the P4 renders POV-Ray's "Glasschess" much faster than the AthlonXP, but loses by a wide margin when rendering "Chess2". <O< case.

Doch gleichwertig?
Although it'd be easy to conclude this is due to Intel's consistent improvement of the Pentium 4, an empirical examination of the evidence proves this is not the case. While Intel has, indeed, improved the P4 dramatically since its launch, AMD has also dramatically improved the Athlon. In fact, the two CPUs are currently running at nearly the same ratios they held when AMD first launched the AthlonXP. AMD has operated at 50% of the P4's bus for most of its life, so the current 400 MHz / 800 MHz gap, despite looking more significant than the 266 / 533 MHz gap, isn't. Both CPUs now sport 512K of cache and both are comparable thermally, so where's the Intel gain coming from?

Die Rolle der Mainboards
It's not explained entirely by platform advancement either. While Intel's 875 Springdale chipset is leagues above the original 845, nForce2 is quite a leap from KT266A (which was only beginning to become widely available when P4 launched) itself. Although P4 enhancements and platform improvements do account for some of the widening performance differential between AthlonXP and P4, the change hasn't been nearly as dramatic as the overall shift in benchmark performance.

Die Rolle der Software
The real reason for the widening performance delta between AMD and Intel lies not within the hardware changes introduced by each company but in the changes to the benchmark software used to measure relative performance between the two products . Fire up Sysmark 2000 and the AthlonXP will destroy the P4 in both Office Performance and Content Creation, switch to Sysmark 2001 and watch the AthlonXP maintain a healthy lead in OP while yielding CC to the P4, switch to 2002 and the P4's lead in Content Creation will widen dramatically and it'll nearly catch the AthlonXP in Office Performance. (It should be noted at this point that AMD officially does not use Sysmark 2002 due to what they feel are dramatic inconsistencies in that benchmark's scoring methodology. Rather than simply complaining about the situation, AMD joined BAPCo over it, so it'll be interesting to see what effect (if any) this has on Sysmark 2003.
This trend isn't limited to BAPCo products. In Content Creation 2001 the AthlonXP walks all over the P4, scores nearly identically to it in Content Creation 2002, and is creamed by Santa Clara in CC2003. Similarly, Business Winstone 2001 will hand an easy win to AthlonXP; Business Winstone 2002 will tie the AthlonXP 3000+ against the P4 3 GHz.
(Please note that all of the above assumes comparable CPUs and platforms).

Charakter der Massenware Spiele
Games, it should be noted, are exempt from the above trend and are one area where the P4 and the AthlonXP maintain near-identical performance parity with only a few exceptions. Gaming, interestingly enough, was once an area where Intel technology most clearly demonstrated its superiority over 3rd-party rivals—now its one area where the two companies are neck-and-neck.

Wahre Stärke ?
So, why do modern benchmarks increasingly favor the Pentium 4? The cynical answer would be that Intel pays/influences companies to build benchmarks that favor the Pentium 4 over the AthlonXP , and to some degree this is undoubtedly true; Intel having shown no tendency in the past to stay up late worrying about the moral implications of playing dirty. Biased benchmarking has a part to play in this puzzle, but unless you believe every competent reviewer is "on the take" and every benchmark author is an Intel-loving slimeball, it's not going to put all the pieces together.
…
Over the last eighteen months, SSE/2 has evolved from an SIMD instruction set whose benefits were strictly theoretical into a widely-used invention that's boosting Pentium 4 performance above that of its AMD counterpart. Back in the days of the P3 when the two CPU's were far more similar, this was much less of an issue, but the fact that the P4's architecture is so different than the AthlonXP's is precisely what's fueling Intel's rising success.
Its not that AMD's CPU is optimized for "old" software nearly so much as it is that Intel, as the 800 lb gorilla on the block, knew that the software industry would inevitably shift in whatever direction it chose to go. By building a processor that emphasized different features than its competitor and used an SIMD set the other didn't have, Intel assured long-term that its flagship CPU would out-pace the other as the very nature of software changed around it. This is good news for Intel, but it doesn't put AMD in a very good position at all. The playing field, however, isn't entirely tilted towards Chipzilla—in Part II of this article we'll be examining how a few rules do give Sunnyvale a fighting chance in the CPU market. µ
This ends Part the First, in which we've discussed the reasons why relative performance between the AthlonXP and Pentium 4 has shifted so much. Part II will examine how performance between the two processors is likely to shift once Athlon 64 and Nocona are introduced.

Zum Vergrößern anklicken....

Ist der P4 unangreifbar?
By Jack Russell: Mittwoch 28 Mai 2003, 08:51
In Part I of this article we discussed the changing nature of benchmarking between AMD and Intel and why the P4's competitive advantage continued to grow.
Der erste Teil behandelte die wechselnde Natur der Benches zwischen AMD gegen Intel, und auch die potentiell wachsenden Vorteile des P4.

In Part II we'll examine other rules of the game that allow AMD to remain competitive - to a point
Der zweite hier vorliegende Teil untersucht die anderen Teile des Benchmarkkrieges, die es AMD erlauben könnte wettbewerbsfähig zu bleiben.

Die Intel- Pest; Standards setzen!
GIVEN OUR CONCLUSIONS in Part I, Intel might seem to ultimately be in an unassailable position. Because the Pentium 4 is still the de facto standard for which programs are written and optimized (and because Intel has shown great willingness to lean on programmers to ensure their products are preferred),
one might expect to see the AthlonXP's comparative performance continue to erode until AMD is pushed back into the budget market.

AMD`s Finanzschwäche & Folgen
Although Intel would have no chance of pushing AMD out of the budget sector, (the AthlonXP demolishes the Celeron too thoroughly for that),
AMD's shaky financial structure couldn't withstand the damage that being forced into strictly budget systems would inflict. Sunnyvale would likely be purchased and their CPU line might even survive, but AMD as an independent company would cease to exist.

Die Intel-Cholera SSE2
There are, however, mitigating factors which make this an unlikely outcome. The largest single reason why Pentium 4 performance has improved over the last two years (other than potential cheating) is the steady increase of SSE2 optimized programs on the open market (and hence included in benchmark programs) .
From the beginning it was obvious that SSE2 was the Pentium 4's key to high FPU performance, but a lack of optimized software left the CPU gasping in FPU-intensive code. With the continuing rise of SSE2 software we see the P4 gaining ground and in some cases even surpassing the AthlonXP.

AMD’s Grundkondition
The fact remains, however, that SSE2 is only able to deliver a performance increase to the P4 in situations where its SIMD abilities can be fully utilized. There is a significant number of programming areas where SSE2 cannot boost performance very much, and it's in these types of applications, including many games, where the AthlonXP will continue to hold ground .
While the two CPUs are differently designed, they still have far more in common then, say, the PowerPC G4e and the P4, and AMD CPUs have often gained performance when Intel introduced new P4 optimizations - though not when those optimizations were SSE2-only.

Die Bleikugel Software
Another temporary factor that prevents newer benchmarks from sweeping away AMD's performance lead is the software industry's current situation.
Although Intel is on record as stating it feels only the newest versions of benchmarks should be used, many reviewers strike a balance between the latest tests and the products people actually own/use.
Adobe might love for everyone to be running Photoshop 7, but if you're a business with a staff extensively trained on Photoshop 5, have a very limited IT budget thanks to the ongoing recession, and don't absolutely require version 7 you probably aren't going to use it.

Hardware ist so haltbar wie frischer Fisch, Software ist lagerfähig wie Dauerwurst
Upgrading hardware tends to be cheaper than upgrading software once the cost of employee retraining is factored, and its more likely in a recession that businesses will upgrade hardware to boost performance in software they already own then to upgrade software on existing and slowing hardware.
Slower software adoption means that benchmarks become obsolete more slowly, regardless of how many years have passed since they were released, and hence are retained for a longer period of time. Because many of the tests AMD prefers most are older tests, Sunnyvale is helped by the current situation.

Der Underdog- Bonus
These are the two largest technological or economic reasons why AMD isn't in as much trouble, but there are other factors still to be considered.
One significant but hard to measure factor is that many buyers prefer having a choice when it comes to buying a product to not having one.
intelfanboys
Only the most die-hard Intel fans (read Paul R. Engel) would argue that AMD has brought nothing to the market, and many enthusiasts who are Intel-only buyers nevertheless acknowledge that AMD's resurgence in the past five years has lowered CPU prices while accelerating the rate of new product introduction as the two companies struggle to surpass each other.
AMD-Fanboys
As long as AMD can offer good performance there will always be a class of buyers who'll be attracted to the company either because of price, price/performance ratio, or a simple desire to support the underdog.

Die FTC & Monopole und Intel’s Interessen
In certain respects it's even in Intel's best interests to ensure that AMD stays alive. In the mid-1990s the FTC initiated an investigation of Intel over alleged monopolistic behavior, only to drop the investigation when it became clear that AMD was offering serious competition and a valid alternative option to Intel's technology.
Even if Intel felt it had a good chance of being cleared in such an investigation or winning a court case if the FTC brought suit, the last thing Santa Clara wants to deal with is a long, drawn-out, and extremely expensive trial ala Microsoft and the DOJ.

Wasche mich aber mache mich nicht nass
Of course, the entire situation is a paradox. Intel wants AMD to continue to exist in order to save itself a tremendous amount of trouble, but simultaneously wants to control the entire CPU market, including that irritating chunk AMD continues to hang on to.
The situation actually parallels Intel's relationship with its 3rd party chipset manufacturers. It wants them to exist because it makes money from their licence fees, just as they make money from AMD's licensing, but at the same time covets the business those third party chipsets generate. It's a classic case of wanting to eat your cake and have it too.

Das wahre Motiv zum Erteilen der SSE- Lizenzen
Of course, if SSE2 is the linchpin on which the Pentium 4's performance hangs, one might wonder why Intel licensed the technology to AMD for inclusion in Athlon 64, or, for that matter, why it licenses the right to make x86 processors at all.
The reason why Santa Clara engages in this type of activity when refusing such licenses would doubtlessly be the fastest way to kill the competition has to do with the requirements of US monopoly law.
Simply put, it's illegal for a company to use its market position to deny another potential competitor the ability to enter the market for the purpose of maintaining a monopoly.
Thus, the Baby Bells, which own much of the phone wire across America, are required to lease it to their competition, and Intel, which owns the right to the x86 processor, is required to license it to those who would manufacture x86-compatible processors.
This fact leaves Intel in a bit of a catch-22. Santa Clara is free to invent new instruction sets to give its processors a competitive advantage (hence, SSE2), yet once such standards are so prevalent that they are becoming a key factor in competitive performance , Intel will face growing pressure to license the design, and could conceivably even be sued for monopolistic behavior should it refuse to do so.
The more successful Intel's unique performance-enhancing technology is, the more likely it is that they'll be required to share it.

Softwarerüstungsspirale
Seen in this light, Intel's willingness to license SSE2, and the company's plans to jump to SSE3 with the launch of Nocona, make a great deal more sense.
Just as we saw the Athlon's performance jump in key benchmarks once that CPU acquired SSE support, Athlon 64 could benefit enormously from SSE2. We could, in fact, be left watching the same scenario repeat itself, with Athlon 64 dramatically out-performing Nocona in SSE2 optimized benchmarks, only to watch SSE3 slowly dominate the market—until Intel chooses to license it rather than avoid the potential for legal action, at which case we'll undoubtedly see SSE4 emerge, and so on and so forth.

Alte Märchen
One of Intel's favourite claims is that AMD is nothing more than a copycat and a thief of Intel technology. This argument held weight when the K6 and K6-2 were billed as a poor man's Pentium/Pentium II, but when the Athlon arrived, that argument's power more or less evaporated as the K7 proved itself capable of beating the P3 and initial versions of the P4 at their own game.
Eigene Anmerkung: Seit dem K5 bewies AMD, dass sie mehr als ein intel-Clone waren. Der K5 war nicht nur ein eigenständiges Design, sondern hatte eine vollständige neue Design-Idee. Wie der PentiumPro war der K5 intern nach den Gesetzmässigkeiten eine RISC-Prozessors gemacht worden. Seit dem K5 war ein CISC-Hardwaredesign bei AMD tot.

Die Geheimwaffe Software
Having lost the edge in CPU design Intel has since turned to software optimization to maintain its lead, and unfortunately it looks as though it'll be very difficult for AMD to crack their hold.
The two CPU's are ultimately on different performance curves, with the very latest and newest tests - the forward-looking ones, so to speak--always favoring Intel more. Some would argue this is due to optimization and some would say its due to strong-arming, but the trend is undeniable no matter its source.
AMD, even when its performance jumps dramatically, always jumps most impressively in the benchmarks of "today", while Intel holds the benchmarks of "tomorrow."
While it's the benchmarks of today that are arguably more important, being able to drive the market towards the strengths of their processor allows Intel to maintain an illusion of being a market leader while AMD merely follows in its shadow.
This, of course, is the whole idea. Given the above, it's very difficult for AMD to position itself to steal a march on Intel and lead the market in technology adoption, but it's not impossible.

In Part III we'll be examining AMD's x86-64 initiative, and why it just might have a chance. Stay tuned. µ

Zum Vergrößern anklicken....

Benchmarking: Can AMD move the market?

Part III: There's always an exception to the rule
By Mike Magee: Donnerstag 29 Mai 2003, 11:00
Kann AMD das Benchmarkschlachtfeld wieder aufmischen?
Und es gibt doch eine Ausnahme von der Regel!
In …
Der Stellenwert von AMD’s Instruktionssatz AMD64
WE'VE DISCUSSED HOW … Now we turn our attention to AMD's x86-64 technology and whether or not Sunnyvale has a shot at breaking a trend that decidedly favours its rival.

Wie das Rambusdesaster?
At first glance, Intel might seem to be holding all the cards, but things aren't always what they seem. In order to gain a bit of historical perspective on the current situation we turn our attention back to 1999 and the beginning of the great Rambus debacle.

Historischer Rückblick in das Jahr 1997/98
Odd as it may sound today, there was a time when RDRAM technology had a cautiously positive reputation in the overall tech market. Back in 1997 and 1998 Intel aggressively backed it as a next-generation memory technology to be introduced as a high-end RAM type with SDRAM holding the mid-range and budget sectors until RDRAM could be ramped into those markets as well. There were rumblings of discontent, particularly from Dramurai who weren't thrilled with the idea of being turned from memory designers into glorified chip foundries, but Mighty Intel had spoken and that was supposed to be that. Even AMD, though it never used it, acquired a licence to produce systems built using RDRAM technology.

Ausbeute, i820, Lernphase
As time passed, however, Rambus' problems didn't vanish and the rumblings of discontent didn't fade into the distance. Intel's i820 chipset was delayed several times and missed its "set in stone" third quarter launch date, while evidence mounted that there were a variety of problems with RDRAM. Yields on the RDRAM chips were very poor, prices were stratospheric, and the design changes necessary to build a functional RDRAM motherboard were making life extremely difficult for Intel's engineers. All of these combined to form an increasingly negative market-view of RDRAM technology, but there were a couple of events that were particularly significant.

Caminogate
Intel's late launch of Camino (i820) with only two RIMM slots rather than the promised three left a sour taste in the mouths of many users, especially given the high cost of RDRAM and the perception that Intel had either chucked the chipset out the door in frustration, rather than finishing it, or that the RDRAM design was so obtuse that a fully-functional chipset design wasn't possible. The true history of events is somewhat harder to puzzle out, as Intel blames Rambus and Rambus blames Intel. Rambus, of course, blames everyone else except itself for its miserable failure, but that's neither here nor there.

SDRAM MT-Hub
Faced with lacklustre i820 sales and high RDRAM prices, Intel attempted to regain a measure of market success by creating and launching what might be its most disastrous product to date: The i820 MTH (Memory Translator Hub). The idea behind the MTH was for users to be able to use SDRAM in an RDRAM motherboard, thereby cutting overall system cost, but as it turned out, simultaneously slashing performance.

Genickbruch für den SDRAM i820
The i820's MTH design, however, had a fatal flaw that caused systems using it to periodically hard lock and crash. This forced a mammoth recall and fed public perception that the i820 motherboard was over-priced, badly designed and should be avoided at all costs.
…

Die öffentliche Meinung
Looking back over the entire Rambus disaster, we see public opinion shifting from cautious optimism to rampant dislike and even hatred as Rambus became one of the most vilified companies in existence. Now let's factor AMD into the equation and see what happens.

Fenster der Gelegenheit
Although AMD held a Rambus licence, the company never exercised it, nor cut a stock deal with Rambus. While Intel was mired in the muck of i820 (Caminogate) and the incredible expense of RDRAM, AMD made a great deal of market headway by not only offering a less-expensive processor, but pairing it with less-expensive RAM at the same time. The only Intel platform that officially supported a 133 MHz bus, and could compete, was the i820 Camino which came in two flavours - horribly expensive or totally unusable. A hole opened in Intel's product line and AMD took full advantage of it.

Argumente für AMD’s vorhandene Marktmacht
As market opinion began to shift against Rambus, AMD took the opportunity to throw its full weight behind DDR memory. Now, of course, DDR is king and RDRAM is rotting on the dung heap of discarded PC memory standards, but it wasn't supposed to happen this way.

Offene Augen und Ohren
According to our trend line, Intel should've won this round and we all should be using RDRAM. Instead, the AMD-backed standard won. Why? Because AMD identified the desire of the market more accurately than Intel did and gave people what they wanted.

Verbrauchermacht
By itself, AMD could never have forced DDR into the marketplace the way Intel attempted to force RDRAM, but without AMD there would've been nothing to stop Intel from ultimately jamming Rambus down the throats of consumers. What made the Rambus situation unique was the degree to which users disliked the RAM standard and wanted to avoid it. Buyers had essentially three choices: Buy a P4 based on RDRAM, buy a P4 based on SDRAM (with 386-like performance) or buy an Athlon-based DDR platform. The only thing that seriously hampered Athlon DDR adoption was that none of the chipsets at the time were all that impressive - had they been, the CPU might've yanked more market space away from the P4.

Der Gorilla bewegt sich
As it was, Intel saw both the desire of consumers and the fact that buyers had a valid alternative through which to exercise their desire and, rather than face losing market share to AMD, threw its own weight behind the DDR standard. AMD didn’t force Intel to change to DDR, the market did - but without AMD that market would never have found its voice and hence never had any real ability to exert pressure on Intel at all.

Juckpulver für den Gorilla
So we have to amend our conclusion a bit. AMD can't muscle the computer industry in a new direction the way Intel can, but Sunnyvale is capable of functioning as a release valve and a solution provider to the point where Intel sits up and pays attention rather than risk losing its own market share.

Die große letzte Chance das Spiel zu diktieren?
The ultimate question is whether or not AMD's x86-64 technology (AMD64/EM64T) has enough market demand behind it to force Intel to de-rail the Itanium processor. While Itanium isn't disliked in the same way Rambus is, the CPU is seen by many to be an overly-expensive proprietary solution that'll take a decade to fully adopt and cause no end of adoption hassles in the process. Whether this translates into a successful market for AMD's x86-64 technology remains to be seen, but it's not an exaggeration to say that it may be AMD's last, best opportunity to break the cycle of catch-up we've detailed here and seize control of the innovation market, at least for a while.

Gesichtsverlust und Investitionskosten
If x86-64 technology does catch, AMD will have stolen a march on Intel and Santa Clara will find itself at last briefly in the position of having to dance to Sunnyvale's tune. Priority #1 at Intel would be to rectify this situation but the cost in terms of market prestige, bragging rights, and failed investment in the Itanic would be immense.

Ist der Hammer das, was der Markt wirklich braucht?
AMD needs Athlon 64 to re-define the playing field and give it a home court advantage similar to the one they wielded in the first 18 months of the Athlon's life.
The gamble is that it's managed to keep its fingers more tightly on the market's pulse and have a product that delivers what buyers are actually looking for.
Given that Intel has sold less than 5000 Itanium CPU's all told it may have a real shot - or Athlon 64 could go down as AMD's ultimately failed "good try."
One thing that's certain is that AMD will never truly excel so long as Intel is calling the shots. Compete, maybe, but surpass? No.

For Sunnyvale, it's time to resign itself to playing second fiddle, or find a way to change the rules one more time. µ

Zum Vergrößern anklicken....

• Code:

  difference (%)	“Cache” 3000+ vs. 2700+	“FSB” 3200+ vs. 3000+
  3DMark		+2.4			+6.5
  RtCW		+2.8			+10.5
  SS2		+6.8			+7.4
  3ds max		+1.5			+2.9
  ICC		+19.3			+6.7
  OP		+9.8			+7.5
  drv-08		-0.5			+5.0
  dx-07		+0.7			+2.8
  light-05	+0.8			+6.1
  MP3		-0.6			+3.8
  MPEG4		+3.9			+9.0
  WinAce		+6.3			+14.7
  WinRAR		+4.4			+19.8
  Read		-0.1			+14.2
  Write		+1.0			+19.4
  Copy		-0.3			+15.0
  CPU RM		+0.5			+2.8

Tyan puts its house in order
Balance is the new watchword
By Mario Rodrigues: Dienstag 03 Juni 2003, 09:03
LAST MONTH the INQUIRER ran a story that was titled: Is Tyan hamstrung by Intel? In that piece, I highlighted how the language used by Tyan to present its Xeon-based motherboards was far more favorable than the way it presented its Opteron product.
Somebody at Tyan has listened up.

Since that story, Tyan has published two more press releases. The first announced a special bundle program for its Opteron based motherboard and Turbolinux 8 for AMD64, and the second was about its latest Xeon-based motherboard.

What's significant about both releases is that the language used to present both products is measured and appropriate for them both.

Tyan's description of its Opteron motherboard changes from "ideal" and "unique" to "leading edge performance" and "for today's most rigorous server applications." Tyan tones down the language used to describe its latest Xeon motherboard to "unique" and "compelling" from the "unprecedented" and "exceptional" used before.

Danny Hsu, Senior Director of Marketing at Tyan, even gets quoted in the Xeon system board press release, so it's fair to say that the top marketing man at Tyan is taking a personal interest. If this change in attitude is permanent, then he needs to be thanked for putting Tyan's house in order. µ
http://www.theinquirer.net/?article=9812

Zum Vergrößern anklicken....