Another year, another conference season sees me stuck on this side of the pond watching the press releases from afar, promising myself that I’ll watch the keynotes online or ‘on demand’ as people have it these days. I never find the time and have to catch up with the 140 character synopsis that regularly appear on Twitter.
I can already see the storage vendors pimping their stuff at NAB; especially the Object storage vendors who want to push their stuff. Yet, it still isn’t really happening….
I had a long chat recently with one of my peers who deals with the more usual side of IT; the IT world full of web-developers and the likes. He’d spent many months investigating Object Storage; putting together a proposition firmly targeted at the development community; Object APIs and the likes. S3 compatible, storage-on-demand built on solid technology.
And what has he ended up implementing? A bloody NFS/CIFS gateway into their shiny-new object storage because it turns outs what the developers really want is a POSIX file-system.
Sitting here on the broadcast/media side of the fence where we want gobs of storage provision quickly to store large objects with relatively intuitive metadata; we are finding the same thing. I’ve not gone down the route of putting in an Object storage solution because finding one which is supported across all the tools in today’s workflows is near impossible. So it seems that we are looking more and more to NFS to provide us with the sort of transparency we need to support complex digital workflows.
I regularly suggest that we put in feature requests to the tools vendors to at least support S3; the looks I generally get are one of quiet bemusement or outright hostility and mutterings about Amazon and Cloud.
Then again, look how long it has taken for NFS to gain general acceptance and for vendors to not demand ‘proper’ local file-systems. So give it 20 years or so and we’ll be rocking.
If I was an object storage vendor and I didn’t have my own gateway product; I’d be seriously considering buying/building one. I think it’s going to be a real struggle otherwise and it’s not the Operations teams who are your problem.
Me, I’d love for someone to put an object-storage gateway into the base operating system; I’d love to be able to mount an object-store and have it appear on my desktop. At least at that point, I might be able to con some of the tools to work with an object-store. If anyone has a desktop gateway which I can point at my own S3-like store, I’d love to have a play.
Well said- would you take a VM? Key to the object storage market in our opinion is leveraging gateways that allow today’s applications to work with object stores (ala CIFS/NFS), but keeps data in its native format. This way as applications evolve (and they will when the data is there), you can leverage the storage APIs directly and finally be freed from the CIFS/NFS bottleneck / NAS management nightmares.
We use webdrive or netdrive to mount our cleversafe object store on windows and mac.
S3? You meant CDMI right?
Martin,
We have exactly what you describe you’d love to see, and you can download everything in a fully functional time-unlimited max 2TB capacity version from caringo.com.
On my notebook I’m currently running all the following simultaneously in VM format:
* 8 CAStor v6.1 nodes with 4 virtual drives each, running a 5+2 erasure coding scheme
* a VM with our CFS (Content File Server) running on top of that CAStor cluster, serving out its fully POSIX compliant virtual file system over CIFS, NFS and WebDAV simultaneously. Use a free app like WebDAV Navigator to shoot an iPhone picture directly into a WebDAV folder, then watch it on Windows and OS X simultaneously through the CIFS connection, for example.
* a Windows 7 VM to show multiplatform (the notebook is a MacBook Pro Retina)
When copying in content from another machine through WebDAV, I’m seeing around 1.5 Gigabyte per minute – and that is with the whole kaboodle running on a 4.5 pound notebook! You can also just run CFS against our demo cluster at cas.caringo.com. It’s also important to know that CFS isn’t just another gateway – it is by far the most scalable of its kind, built to store hundreds of Millions or even Billions of files without the ugly hockeystick response time curves so typical for other file-system based offerings. The file-folder hierarchy of CFS is persisted in structured XML documents stored as objects in the CAStor cluster – therefore it scales nearly indefinitely.
Your dream is waiting for you, Martin! Just give it a try. I’m here to answer any questions you may have.
— Paul
[…] Object Paucity (Via Martin Glassborow aka @storagebod) […]
[…] is an answer I wrote to my dear friends StorageBod’s latest piece on Object Storage. As my response turned out quite a bit longer than initially intended, I thought […]
[…] was all ViPR was; it’d be pretty tedious. Still EMC obviously read my blog and obviously read this and rapidly turned it into a product or perhaps they simply talk to lots of people too. If […]