EMC's refusal to play in the game in benchmarketing in light of all the criticism is admirable as it leads them open to all kinds of snide comments. Do they not play as they feel that their arrays simply won't get the performance figures or is it some higher motive? I'm not sure it is either but their intransigence has been pretty solid. And BTW EMC, it's not just the other vendors who make snide comments, there's alot of people in user-land who use it to generate internal FUD.
But we are beginning to see some chinks slowly but surely and there is an acknowledgement that there needs to be a community effort to come up with a benchmark which more accurately reflects real-world applications and the mix of applications which users run. Barry Burke and EMC seem to be more than open to this; perhaps the other vendors who read my blog should call their bluff and engage in the effort. There are plenty of end-users who will help talk through typical workloads and what we would like to see from a benchmark.
Can I also ask that we start to get some end-user performance modelling tools please? Some what-if modelling tools like sym-merge but tools that I can get my hands on; I know there are concerns about your competitors getting their hands on them but for example, if IBM want to model workloads on a DMX, I'm sure that Global Services have a few that they can get their hands on. Lets be honest, once an array is out in the wild; it's secrets are known.
So lets try to make everyone's lives a bit easier; full disclosure of performance figures would be start. I mean even getting figures out of some vendors about disk-spindle performance has been tough at times, I've had Technical Pre-Sales passing me these documents with dire warnings not to disclose where I got them from only to discover that they are the same as the industry standard metrics. Actually getting figures for a standard 70/30 OLTP type workload; well, it is almost as if I'd asked for the secret of the universe.
