Benchmarking the SPARC T4 - Is the Intel Xeon Business Case Dead?


It's been 6 months now since the release of the SPARC T4, and The Liberator has been patiently waiting for clarifying benchmark results out of Oracle for the system. To quote IBM's Elisabeth Stahl "All I Want for Christmas is a Good T4 Benchmark"
But, Shock! Horror! Oracle has not produced one. Now I've got a customer who is seriously thinking about a T4. So let's revisit some old ground.

"Record Breaking SPECjEnterprise2010!"

That was exciting, for a moment, until one arrives at the actual page. SPEC is THE source for benchmarking data, and new records there should be treated with respect. But help yourself, have a look at the results

First, respect where it is due. The T4's result is bigger than anyone else's. It is, indeed, record-breaking. 6 months later and it's still 50% ahead of the runner-up.

But let's put that in context. SPECjEnterprise2010 is a tiny backwater of a benchmark with very little competition - only 25 entrants since inception two years ago. Not like SPEC CPU2006 with 3,236 results at time of writing and climbing every day. Oracle also used a cluster of six 4-socket machines to run the benchmark, for a total of 192 cores. The next best result used half that many machines. So let's cut to the chase - what does this record-breaking result mean to someone with a real pocketbook?

Per Core Performance? Not So Record Breaking.

A year ago, MYRA began exploring the use of the Intel Xeon X5600-series CPUs as a means of reducing the operating expense of running an Oracle database at enterprise scale. What we found was that the tooth-and-nail fight between Intel and AMD had produced x86 processor performance that had leaped out of the old desktop paradigm and well into the heavyweight UNIX class. No match for IBM's POWER7 architecture, still, and it couldn't quite scale to the top of Sun's M9000 big iron, but a real contender.

Then we reviewed the Oracle costings model - which for the Oracle Enterprise database is based on core count. The more cores in your system, the more more money you pay Oracle. This is true both for initial licensing and for ongoing software support.

So for a given workload (say you've got 1000 users, for example), the fewer cores you can run the load on, the less money you pay. So whose cores win?

When we started looking at it, POWER7 was the winner on per-core performance, but Intel Xeon X5600-series CPUs were nipping at the heels. All this is writ large and in clear type in the SPEC CPU2006 results. Poor SPARC64 (e.g. Sun M5000, M4000) was getting it's butt handed to it.

But Lo! we've got a record-breaking result now from the new SPARC T4! Too bad Oracle still hasn't published CPU results. But we can do some quick arithmetic on the SPECjEnterprise2010 results to get a per-core result. How are those standings now?

Much as they were on the day they were released - the SPARC T4 is playing Second Fiddle to x86 gear. Well, Fifth Fiddle today (today's top 4 all being x86), while it was merely runner-up on its release date. To add insult to injury, 2 of those 4 results were handed out by machines connected to good old-fashioned spinning disk - in contrast with the SPARC T4 setup with massive SSD arrays.

So Wherefore the Intel Business Case?

Well, in absence of a decent CPU-only benchmark, we can only speak broadly about the SPARC T4's per-core performance. SPECjEnterprise2010 definitely suggests the T4 is still getting outperformed by Xeon. Oracle has increased their per-core licensing factor for the T4 compared to the T3, with Xeon and T4 both at 0.5. Poor POWER7, heavyweight champion, is dinged by Oracle at double the licensing cost per-core - else it might be the winner.

But with superior performance (and hence lower licensing/support fees), lower purchase price, lower power consumption, lower hardware support fees and an immense array of market choice in Tier-1 manufacturers, Xeon still has an obvious business case.

And that's good news as far as The Liberator is concerned. You can save money by using open-market equipment and you can save your wrists from what could be manacles at the same time.