So you want to build yourself a storage cloud but you don’t have the readies to build one using one of the commercial products which are available. Well, don’t worry, there are open source alternatives which might allow you to get a taste of Scale Out without breaking the bank.
Gluster is one such open source alternative and is now part of OpenStack, the open source cloud computing platform being built by a number of developers and vendors.
Gluster is available as a commercial software appliance or you could simply download the packages and install it on a variety of Linux distributions including Ubuntu and Redhat-derived Linux distributions. I have recently built a small cluster using Scientific Linux 6.0 (Scientific Linux is my new favourite Redhat derived Linux and SL 6.0 is based on RHEL 6) and ESXi.
The initial set-up is pretty easy and it took me less than a couple of hours to stand-up a three node cluster and build a small environment. The documentation is clear and should be simple for anyone with a modicum of Linux knowledge to follow.
I will give people a couple of tips; if you do not want to play with IPtables, turn them off to get yourself up and running. And the latest version of Gluster requires rsync 3.0.7 for its geo-replication; there does not appear to be a RPM for RHEL 6.0 at present, just use the Fedora RPM and that appears to work fine.
Adding additional nodes is simple and I’ve quickly added a fourth virtual node non-disruptively; then it is simply a case of telling gluster to rebalance the files across the nodes.
But only supporting Linux means that if you want to serve files to other operating systems, this means utilising NFS and CIFS. There seems to be conflicting information on whether Gluster supports CTDB and the necessary locking; so at the present I am only exporting NFS from a single node with no fail-over support yet. My next experiment will be to see if I can get it configured as a true scale-out NAS solution.
I will let you know how I get on!!
Hey, nice overview. We are actually using CTDB for our geo-replication features in 3.2, but several customers just use NFS or CIFS to connect non-*nix OSes.
-John Mark
Gluster Community Guy
Thanks for popping by John; I’ve got CTDB working and clustered NFS working. It’s actually probably easier than getting CTDB working with GPFS.
It’d been even easier if I’d planned it out and thought about it as a complete exercise as opposed to building a Gluster environment and then putting CTDB and NFS on top. But hey, it’s all VMs at present, so I’ll probably blow them away and do them properly.
With Apple including XSAN in OSX, I think the interest in Clustered/Distributed file-systems will get pretty hot.
You can also use ucarp to setup IP failover for NFS or CIFS with Gluster. It is easier to setup than Samba CTDB.
Thanks, looks interesting. I’ll have a look at it; I can see a number of potential blog entries in the future on how to build open-source scale out storage.