Storagebod Rotating Header Image

Cloud

Do you have an Umbrella?

Oct 5th, 2009
by Martin Glassborow.
No comments yet

Every now and then, we have 'interesting' conversations at work about morbid subjects; recently we had a discussion about another 9/11 and whether our systems could cope with it. Not the systems that you get to see but our internal systems and how we scale for peak load and what peak load actually means. Scaling web-servers is pretty easy but other systems don't necessarily scale so easily.

We had recently put a new system in and then Michael Jackson went and died; so that was a good test for the system, which coped pretty well. Tragic events are what generates traffic; that and England winning the World Cup but we have generally have to deal with tragedies not miracles.

But web-sites, phone-systems, networks can all fail when under extreme load; in fact, pretty much every form of utility is massively over-subscribed pretty much by design. If everyone got in their cars at the same time, the road network would melt-down. Flights are regularly over-sold, certainly by the budget providers. If everyone in the UK decides to make a cup of tea at the same time, the power-grid suffers.

This led me on to thinking about the Public Cloud; what level of over-subscription do the Public Cloud providers sustain in the forms of 'reserved instances' and how many instances can they actually support all coming up at one time. How much storage does Amazon actually have?

If you are going to bet your business on the Cloud, you probably ought to know and you probably ought to know what kind of events will cause you to burst and probably what will cause everyone else sharing the Cloud with you to burst. And what you are going to do if the Cloud does burst? Do you know where your umbrella is? Or at least the candles to dry you out.

What a Fraud!!

Oct 5th, 2009
by Martin Glassborow.
4 comments

Sometimes when I'm writing this blog, I feel a complete fraud! Actually I suspect most of the time that I am posting complete nonsense. Many of the technologies that I post on, I will never use in my current role.

1) Virtualisation – neither storage or server virtualisation impinge on my world. And it's currently hard to see that they might. Digital media production doesn't lend itself to virtualisation at present even the flag-wavers admit that they are not ready for such environments.

2) Deduplication – Digital media, enough said. Although Ocarina might have a play, digital media is the one place that deduplication does not really play.

3) Cloud – if we are not ready for Virtualisation, then we are not ready for Cloud. Even Cloud Storage currently has a limited presence in the large, performance oriented media space.

But even so, much of what is talked about still has relevance to me in what I do. I don't need to virtualise to utilise many of the process and procedures which are talked about and being developed. At times we all get caught up in the technology and believe that it is the technology that is something special but really it is only half the story.

Even if you don't see yourselves using the technology, you might find many of the processes and concepts relevant to you; for example, I may not use the general purpose Cloud Storage being provided by many people but I am still interested in how to build meta-data and what meta-data should be.

Arguably, a big chunk of what I am working on at the moment is building a specialised media-cloud which will serve content to a huge variety of applications. If all these applications could talk to the media cloud in a standard way; life would be alot easier (they don't), so I'm interested in the standards which are being discussed.

Virtualisation, actually in the media cloud, we will have very few servers but there may be times when we have to scale very quickly. So I can learn alot from the virtualisation guys on rapid provisioning.

So I'll keep posting on things that I know nothing about and hope that you guys don't catch on!

Important Announcement

Sep 29th, 2009
by Martin Glassborow.
4 comments

This is a very important announcement; there will be #Storagebeers on Wednesday 7th October following the first day of the IP Expo. IP Expo is being held 7th-8th October at Earls Court, London. It has a rather good list of keynote speakers and should be one of the better events in the UK this year.

Venue for drinks has yet to be finalised and any suggestions would be welcomed. Vendor sponsorship is always appreciated at such events but it will be a vendor neutral engagement i.e no fighting!!

And beer should never be virtualised or thin-provisioned!

I'll certainly tweet the venue once it has been agreed!! I look forward to seeing as many of you as possible!

Economic Truth

Sep 25th, 2009
by Martin Glassborow.
2 comments

Steve Duplessie posts on Cloud Economics and especially the economics of Cloud Storage, 20 TBs of storage from Amazon's S3 cloud will cost you $36,000 a year and that doesn't necessarily compare especially well with purchasing your own array. So do the economics of the Cloud storage scale, especially when we start talking about the 100s of terabytes of storage which many enterprises consume?

The problem is we don't really know how much it costs us per terabyte in total! There are no good published TCO models for storage; this is why the initiative started by Nick Pearce and Ian are so important. Go and read their blogs here and here on building a TCO model for storage; let's get this thing crowd-sourced and perhaps we can make the TCO costs of storage a little less cloudy.

And then we can start on the model for Cloud TCO…public vs private etc!!

Live Forever

Sep 10th, 2009
by Martin Glassborow.
6 comments

No not my favourite Oasis track, no it's something which I've been thinking about and a subject I've touched on a few times; once you've deployed a technology, how do you get out of it? Do you get out of it? And what impact does virtualisation have on this.

How many of us are running or are aware of business critical applications running on hardware which is no longer supported by the vendor? And the reasons? Often the reasons are that the application only runs on a particular operating system which is no longer supported and will not run on current generations on hardware.

Virtualisation may well allow you to refresh the hardware; for example if you insist in running applications under DOS; pretty much any of the x86 hypervisors will run DOS. This will allow you to run that business critical application on DOS for the forseeable future and will allow you to continually refresh the hardware underneath it.

Well, it will until the Hypervisor vendor calls time and decides that they no longer want to certify the hypervisor with DOS x.xx; oh well, what do you do? Obviously you shrug your shoulders and now run a back-level hypervisor with an unsupported Operating System which is running an application which by now, no-one has a clue on how it works!

Oh, you've migrated the application into a Public Cloud? Well, it didn't need much in the way of the resources and suited the Cloud perfectly. And now your Cloud provider has said that they are no-longer supporting or even allowing DOS instances to run; oh heck and now you can't get the hardware/software to run your application locally. 

So although virtualisation will allow you to get away with running legacy apps for a long time; don't assume that this means that they can 'Live Forever'! Virtualisation is not an excuse for not carrying out essential maintenance and keeping your estate up-to-date.

*that's 'Bring It On Down', just in case you were interested!

The Cry of the Grump!!

Sep 8th, 2009
by Martin Glassborow.
4 comments

A slightly plaintiff and frustrated tweet from grumpystorage aka @ianhf inspired this blog as did replies from various of the storage twitteratti.

Ian cries the lonely cry of the infrastructure architect

'Application
architects – please know & understand your infrastructure
requirements. Infrastructure isn't magic or telepathic!'

And of course, IaaS and PaaS are then held up as potential solutions to the problem. I've got bad news; they're not! Cloud Architectures are neither magic or telepathic, they still rely on application developers and architects understanding their infrastructure requirements.

Now we, in the infrastructure world can help them by educating in the questions that they both need to ask and also need to answer. Questions like, what availability does your application require? What is it's throughput? Have you designed an application which scales either horizontally or vertically?  Infrastructure Architects have always had to work in a consultative manner and drag the requirements from the Application teams.

All providers of infrastructure need to understand the importance of this; nothing will destroy the credibility of Cloud Architectures quicker than the inappropriate and unplanned deployment of applications into the Cloud.

I think there is a temptation that we represent the Cloud as Magic Smoke where things happen but just look at the backlash when Gmail goes down? Fortunately for Google; people have become so reliant on Gmail, beyond a bit of moaning, few people will really change but a Corporate taking their first steps into the Cloud who loses a key application or a key application under performs may well be more unforgiving. 

Push button deployment without the consultative input of the Infrastructure Architects guiding could end up in a world of pain and it won't be the application at fault, it will be the Infrastructure.

Get Involved.

Sep 7th, 2009
by Martin Glassborow.
No comments yet

As I discuss in my previous post here; what we are seeing with Cloud is very similar to what we saw with the PC revolution of the 80s. We see a de-centralisation of the control of the Corporate IT function and we see a very similar response to this movement from many people within the traditional IT functions.

But perhaps we can learn from history and ensure that this change is less painful and more productive. One of the most significant things about this change is that we can all influence it; the Cloud is not yet a fully fledged movement and we can all steer this ship; it is being debated, defined and developed in public. We can't stop it but we can influence it. Most of the commentators, influencers and nascent practitioners are accessible to all; most write blogs and most are active in all kinds of Social Media.

The Cloud that I am talking about is not the Cloud of Mash-Ups and Social Media, this is not the Cloud of start-ups; this is the Enterprise Cloud which will run our Enterprises in the future. This is a chance for us to get involved in a movement which will define the next ten years of our careers.

In fact if you have lived through the previous changes in computing such as the move to the PC and then to the Internet; you have a huge amount to bring to this discussion. I would encourage you all to get involved this time, don't let Cloud Computing be something which simply happens to you.

It’s all a case of History repeating…

Sep 4th, 2009
by Martin Glassborow.
No comments yet

If we look at the PC revolution of the 1980s, we can start to see some remarkable parallels with what might happen with Cloud computing. The PC revolution took a number of the existing big vendors by surprise; in fact most of the big vendors were so taken taken aback that they are no longer here.

The PC revolution allowed department heads to go and do their own thing; it decentralised the control of the Information Technology budget and allowed users to bypass their slow moving IT departments. Whilst IT departments were debating whether PCs etc were good things and the existing vendors were fighting rear-guard actions trying to maintain their historically high margins; new vendors and paradigms came to the market and people moved on.

Arguably, in the mid-90s and with the Internet revolution; IT departments finally got hold of things again and began to hold the whip-hand again. Larger Unix servers, big networks and an ever growing demand for storage and centralised information allowed IT to do things better and for the last decade, centralised IT functions have probably had control.

But the revolution which allowed 'us' to gain control has evolved and changed into something which could lead to us loosing control again. The growth of the Public Cloud allows department heads to yet again bypass centralised IT functions and to take control back again; yet again we see the inevitable debates by internal IT functions, the dismissal of the Cloud, similar objection with regards to Security, Availability, Control, Scalability etc all raise their heads as they did during the PC revolution.

Have we learned nothing? The customer is king and we need to maintain the focus on the customer; telling the customer that they are wrong will not work, it didn't then and it won't now. Internal IT departments need to ask why their customers are looking at the Public Cloud and try to replicate those services internally.

IT Suppliers need to embrace Cloud and look to how their products can play but also to understand what they need to change. Things are changing again and continuing to carry on as you have in the past is not going to work.

But what I see at the moment both in corporate IT departments and in IT vendors is a scarey amount of lip-service to the notion of Cloud; or sometimes an arrogance belief that this will all go away. It won't but it's not all doom and gloom, I'll suggest reasons why in a subsequent post!

Snake Oil

Sep 2nd, 2009
by Martin Glassborow.
6 comments

According to Kostadis, File Virtualisation is Snake Oil. I'm sure I can find at least a couple of vendors who would disagree with him!

I think it is the scale thing; Kostadis talks about tracking millions of files and moving them about; external meta-databases to keep track of the objects I mean files. Vendors like NetApp are going to have to grasp this nettle of scale and work with it.

For Cloud to work and to gain the scale it requires long term; this is going to have to happen. A file is just a specific type of object and Cloud File Stores have been designed in general to deal with millions of objects. Not all of the problems have been solved but these are not impossible problems to solve and it is not even improbable that they will be solved.

Also, I take issue with his comments about deduplication; file virtualisation can enable deduplication, not at the block level but at the file level. Your gains might not be as good but you can single instance files; these files might have been compressed using technology such as that provided by Ocarina.

The things you do potentially loose are those things which are storage-array based functionality but by externalising some of that capability and becoming truly virtual; your gains may outway that which is lost.

BTW, I still think DataMotion is cool; anything which allows data to move between storage-systems without outages is cool. The easier we can make this, the better! But there are many ways to skin a cat and there will be many useful solutions to the problems that we face.

Failure is an Option…

Sep 1st, 2009
by Martin Glassborow.
3 comments

I really like this comment by Ian on his blog as part of his questions/response to Val; this displays a different approach to engineering a service architecture and instead of focusing on the components; you focus on the service and how you make the service available.

"I see nothing relating to technology more aligned to a 'recovery
orientated architecture' where 'tin/software' failure is expected, and
as such component availability as a requirement is reduced in priority
(in favour of price point) and the architecture of the service deals
automatically with the regular failure of components."

Perhaps part of the Cloud is like routing for services and we simply design services that route round failure. I'm not sure exactly where this takes us but it's an interesting and powerful idea. 

← Earlier Posts Newer Posts →