Are we heading to a Linux moment in the storage world where an open-source ‘product’ truly breaks out and causes the major vendors a headache?
I’ve had this conversation a few times recently with both vendors and end-users; the general feeling is that we are pretty close to it. What is needed is someone to do a Red-Hat and package some of the open-source products and take it on; make them pretty and simple to use. And then give it away..
Of course, Nexenta have already done this rather successfully and if I was looking for a bog-standard traditional dual-head filer product; I’d seriously consider them against the traditional filers.
But great product that it is, it hardly breaks new ground; well apart from price.
What I’m thinking is something which forces its way into the scalable space…block, file and object. Ceph is probably the technology that is closest to this and although it is pretty simple to get going; it is still a bit of science project for most. I’m not sure I’d want to manage a Ceph environment at scale yet; I’d certainly be nervous about running heavy production workloads on it.
Integrating it into a traditional mixed data-centre environment running Linux, Windows and a variety of virtualisation products would be a big challenge.
I’m looking at InkTank to do something but I’m not sure that they have the funding to push it to the level required.
Yet I think the storage market is ripe for this sort of disruption; especially in the object and ‘hyperscale’ space, the big vendors aren’t there quite yet.
Or perhaps a big vendor will finally realise that they can take the open-source building blocks and use that as a weapon..it may mean sacrificing some margin but they could guide the direction and gain some serious advantage. If I was already building commodity hardware, I’d be looking at building proper commodity storage.
There have been a number of tries in the past around to commoditize NAS with it but Linux/BSD have been pretty lagging behind in the block world. Linux storage target is out there but it’s just so featureless and problematic. You’d think that this would be a place for it to shine but just look at the linux volume manager… can’t to hardly match the features I had with Veritas years and years ago. You want to pvmove from a 100GB lun to 2x 50GB luns, you have to manually calculate starting and ending block offsets (for whatever reason you don’t want to move the entire volume with lvconvert)… ugggh.
I will say there are a number of truly interesting things you can do but they seem to hardly get beyond hobby land; just look through the Linux kernel and you’ll see some cool stuff but nobody outside of 2x guys in Antarctica seem to be using them.
Don’t think of me as a guy against Linux… as I’ve got a few thousands Linux servers that I wrangle with daily but everything is so slow feature wise in the storage world… the Sun will probably have turned into a red giant and consumed the earth before there is any stable enterprise systems that can match filesystems, volume managers and arrays out today.
Block is still problematic indeed but file and object is probably less so. And if you look at where the growth in the market is….it’s file and object.
Saying that “Linux doesn’t make a good substitute for NAS/SAN arrays” is like saying “A swiss army knife doesn’t make a good substitute for a machete”.
NAS and SAN have their uses. But they are increasingly not relevant to cloud architectures. The big vendors REALLY REALLY want to sell you their expensive dual-head NAS/SAN arrays for your cloud infrastructure, because they make huge margins on them.
Cloud infrastructures don’t need big expensive arrays. When most of your data is stored as objects and files, you want something distributed, scalable and resilient. I can’t think of a single array architecture that meets these three criteria – and if it does exist the cost involved is ridiculously high for building clouds.
Ceph is promising, just like Martin says – but I also have my doubts about the company behind it.
I have been hoping something was just around the corner for the last couple of years but nothing seems to happen.
Things like FreeNAS and Openfiler fit my needs but focus dwindled. OF is pretty much dead now. Gluster was my next hope as a replacement for OF but too me it looks like a couple of full versions away from being trustworthy. Ceph is even further behind in development/maturity. DIY or someone (if there is anyone) picking up the reins a la OF look like they are a couple years away from having the groundwork to work with (btrfs, enhanceIO/DM cache in the kernel), and they would still need something to scale out. I have read more than a few bad things about Nexenta to make me pause.
I have been watching the space “reasonably” closely. Just as the pieces seems to be coming into place, you look around at the rest of the market, and OSS looks so much more behind. While not completely relevant, look at how much “value” you can get from a Dell PV 3620i/Engenio with their latest firmware. Pretty powerful for something on the bottom end.
I love OSS and would be all over something that was trustworthy and had some competitive teeth (performance, features, community), but looks like I am going to be waiting a bit longer.
Don’t get me wrong I do believe that Linux works excellent for distributed data, object storage on Linux is here today very functional, etc. we looked very seriously about making our own object storage project internally using some of the existing opensource projects that have kicked out from Facebook, Google, Amazon and the like. Openstack has gotten quite a large following while the storage part isn’t heard from much but it’s there and quazi came from Rackspace’s cloud so it has some reasonable clout.
I don’t think the traditional storage vendors are afraid of any opensource object storage, they are afraid of the public versions (Amazon S3, Azure, etc), which I suppose would be a vendor that could be afraid of a free opensource private cloud competition. So to me the only real thing for traditional storage vendors to be afraid of is around block & NAS and Linux is just not there now, even as a good NFS server the file backend operations suck (ever expand a Linux filesystem and had it corrupt itself?)
For starters, until MS open sources SMB 3 (HAH!), or some Linux vendor licenses and then open sources their implementation of a real SMB 3 stack, Linux will always be a second-class citizen in the windows space. So it’s immediately relegated to block and Linux only… meaning good luck competing with NetApp and EMC. Especially at a time when MS is moving all of their products to supporting SMB 3, and in many cases actually providing a superior experience (Hyper-V).
Hell, even Nexenta is going to be in trouble going forward unless they work something out as the stack that Sun open sourced is already showing it’s age.
I would be happen with just block storage. Lots you can still do. Bringing the feature set of competing products into a block storage only Linux target would still be huge leaps forward from where we are now. Just give me “integrated” ZFS like features (caching, compression, dedupe etc) plus some forms of replication with a GUI and CLI and I would be happy (for a while anyways). It should all “just work”.
If that’s what you want, why do you need linux? You can already get that today, or most of it, with FreeBSD or any of the various open source Solaris versions.
A couple reasons.
– For server admin work, my experience is about 50% Windows and 50% RHEL. So I try to stick to RHEL variants and for the most part keep up with that community. I already do a lot of networking and VMware stuff along with my work as a dept manager so I am spread pretty thin as it is. 🙂
– I have yet to see anything in an enterprise (ish) package for the *BSD group that gives me the confidence to put anything important on them. I honestly haven’t taken a serious look in a year or more. Last time I did look, there were performance problems with ZFS and it seemed to be lagging behind the Solaris version.
– I started admin life with Solaris but never have been a big fan. My current employer has a bunch of ex Sun people and we do work for Oracle. Just an opinion but I don’t see much of a future for Solaris (or their storage and server line).
Just an opinion, but following it helps me sleep at night. 🙂 Nexenta seems to closest to what I want but having done some research last year, there were some issues some customers were having and support was lacking. That coupled with the troubles Oracle has been having (keeping ZFS engineers) and I haven’t bothered to look any deeper at it.
FreeBSD is at parity with the open versions of Solaris from a ZFS perspective. They’re actually a bit ahead in some features such as booting from a raid-z raidset, and behind on something like in-kernel CIFS (but so is Linux). Performance is not an issue that I’ve heard of. I don’t personally use it anywhere but in a lab, but I can tell you FreeNAS gets the job done, and you an get enterprise support via IX if you want – so it’s definitely “enterprise ready”.
I’m not talking about mainline solaris, I’m talking about the open source alternative – those built off Illumos. As someone who started on Redhat, moved to FreeBSD in the early 90’s and RedHat started locking things down, then moved to Solaris when it was opened up and the ZFS project began, I can tell you that Solaris can and does get the job done. Linux has yet to provide anything that can compete with dtrace, zfs, and crossbow. I’d suggest giving something like OmniOS a spin. I’m NOT a fan of Oracle and wouldn’t suggest anyone go down that path.
Finally, there is a ZFSonLinux project, but that is still behind the BSD/*laris variants, but catching up fast.
Disclosure – EMCer here.
‘Bod, you’re on to something 🙂 Working on it…. See you at EMC World!
[…] statement more than anything is most relevant here. Martin Glassborow recently wrote about the lack of Open Source storage solutions. Software is cheap and free for many other areas, but yet again, storage is special. Users […]