Storage

Randy Bias has written an interesting piece here on the impact of complexity on reliability and availability; as you build more complex systems, it becomes harder and harder to engineer in multiple 9’s availability. I read the piece with a smile on my face and especially the references to storage; sitting with an array flat on it’s arse and already thinking about the DAS vs SAN argument for availability.

How many people design highly-available systems with no single points of failure until it hits the storage array? Multiple servers with fail-over capability, multiple network paths and multiple SAN connections; that’s pretty much standard but multiple arrays to support availability? It rarely happens. And to be honest, arrays don’t fall over that often, so people don’t tend to even consider it until it happens to them.

An array outage is a massive headache though; when an array goes bad, it is normally something fairly catastrophic and you are looking at a prolonged outage but often not so prolonged that anyone invokes DR. There are reasons for not invoking DR, most of them around the fact that few people have true confidence in their ability to run in DR and even fewer have confidence that they can get back out of DR, but that’s a subject for another blog.

I have sat in a number of discussions over the years where the concept of building a redundant array of storage arrays has been discussed i.e stripe at the array level as opposed to the disk level. Of course, rebuild times become interesting but it does remove the array as a single point of failure.

But then there are the XIVs, Isilons and other clustered storage products which are arguably extremely similar to this concept; data is striped across multiple nodes. I won’t get into the argument about implementations but it does feel to me that this is really the way that storage arrays need to go. Scale-out ticks many boxes but does bring challenges with regards to metadata and the like.

Of course, you could just go down the route of running a clustered file-system on the servers and DAS but this does mean that they are going to have to cope with striping, parity and the likes. Still, with what I have seen in various roadmaps, I’m not betting against this as an approach either.

The monolithic storage array will continue for some time but ultimately, a more loosely coupled and more failure tolerant storage infrastructure will probably be in all our futures.

And I suppose I better find out if that engineer has resuscitated our array yet.

You know that I have somewhat mixed feelings about Object Storage; there is part of me which really believes that it is the future of scalable storage but there is another part of me that lives in the real world. This is the world where application vendors and developers are currently unwilling to rewrite their applications to support Object Storage; certainly whilst there is no shipping product supporting an agreed standard. And there are a whole bunch of applications which simply are not going to be re-written any time soon.

So for all their disadvantages; we’ll be stuck with POSIX filesystems for some time; developers understand how to code to them and applications retain a level of storage independence. You wouldn’t write an application to be reliant on a particular POSIX filesystem implementation so why would you tie yourself to an Object Store?

I was pleased to see this announcement from the guys at Scality; they are good guys and their product looks sound but no matter how sound your product is, you have to try to make it easier for yourself and your customers. Turning their product into a super-scalable filer is certainly a way to remove some of the barriers to adoption but will it be enough?

Of course, then there are the file-systems which are beginning to go the other way; a realisation that if you are already storing some kind of metadata, it might not be a huge stretch to turn it into a form of object store.

File-systems, especially those of the clustered kind and Object Storage seem to be converging rapidly. I think that this is only for the good.

Okay, so once more I take my life into my own hands and post about NetApp! I look forward to the hordes descending and telling me how terribly wrong I am and how I don’t understand anything!

So let’s get this straight, I think NetApp are a great company and they have done the storage world a great service in the way that they simplified administration and furthered the cause of IP storage, especially NAS. They led the way in taking commodity components and turning them into Enterprise class arrays; we should thank them for all of this!

But are NetApp having a bit of a blip in growth? Chris Mellor certainly seems to think so in his interpretation of the latest IDC Storage Tracker figures although there is a different take here; so it’s not entirely clear however I have some thoughts on all of this.

Having taken a straw-poll of some of my peers in the industry, large NetApp estates have such terrible utilisation figures that dedupe is pretty meaningless; 25-40% utilisation seems to be relatively common with reporting based around ‘effective capacity’ sometimes making it worse.

But I am beginning to see utilisation figures creep up; there is an increasing challenge to use this unused capacity and push it up. So existing NetApp customers might not be buying quite so much and are under increased pressure to use what they’ve already got. Has this impacted NetApp’s run-rate business?

And if customers can’t use the capacity that they’ve got; this is going to lead to some hard questions and there are now a plethora of vendors who have solid NAS products who can push NetApp very hard on capability and incredibly hard on price.

That and the confusion that OnTap 8 brought with it’s choice of modes has probably left NetApp a little wobbly.

Throw in the Engenio acquisition which confused their story even more and actually, it turns out that NetApp might not have purchased Engenio at the best possible time with them moving into an architectural refresh all of their own.

I think NetApp may continue to have a wobbly few quarters but that might not be such a bad thing. Great companies do wobble and learn from it; let’s hope that NetApp are a great company, I think they are…

VFCache is really yet more evidence that Storage Vendors are simply becoming software suppliers*; although ostensibly a hardware product; the smarts is down in the software layer and that is where the smarts are going to live. EMC are simply leveraging everything that they have learnt from PowerPath (Free PowerPath, Keep Fighting the Fight!) and using that to build on to introduce storage functionality on the server.

From the presentations we have seen so far, it seems that DeDupe is going to run on the server and not the card;  well, that’s my interpretation. Obviously this is going to have some interesting impact on CPU and memory utilisation meaning that EMC are going to have get this right or they risk the whole reason for putting cache closer to the server. Replication and cache consistency may also  be a server level function.

This does have some interesting implications; although it appears to be hardware product, how hard would it be for EMC to use any ‘flash’ technology which is installed in the server; do we have to have EMC hardware and could we use a Fusion-IO card or even a bog-standard SSD? What happens to the pricing?

EMC are already talking about accelerating any array; although it’ll be better with their own array but will they take this further and use anyone’s cache hardware? We’ll probably end up with another gargantuan certification matrix because EMC like those but it does seem possible. Perhaps Powerpath/Cache which allows a number of different vendor’s caching products to be used? This way EMC can monetise their software even further or perhaps, Powerpath/Cache comes free with VFCache but you need to pay to use it with third party cache products?

And what about file-based acceleration? Where do NAS and indeed Object storage fit into this? Do they? One of the biggest complaints or issues oft heard about object storage is that it can be slow, so would it benefit from some kind of caching; could cache hints be carried in the object metadata?

Also, it might be interesting to see how this could be integrated in the hypervisor stack? Perhaps a PowerPath/VE which support multiple vendor’s caching products?

Now EMC have validated PCIe local flash-cache concepts; we can start to move on and see where this takes us. And yes, someone other than EMC could have validated the concept but they didn’t; so let’s move on from there.

What are the other big vendors going to react with? IBM, HP, perhaps NetApp with SPAM (Server Performance Acceleration Module), HDS?

[* I’ve been thinking a lot about my recent experience with storage and whether storage teams may have more in common in with application teams than first thought; storage infrastructures tend to be more heterogeneous than other data centre infrastructures, certainly there is more vendor differentiation. ]