Steven’s post and a twitter conversation between Storagezilla, W. Curtis Preston and others about archiving, backup, retention and deletion of data bring home some of the realities of data management and policy.
It is often easier to keep data than delete it; this is both a technical issue and an organisational issue. Many applications are not written with good data management in mind; often if you want to scare a developer, ask them how you age data out of their application? Ask them how you might bulk delete data, ask them how you move historical data to an archive and still have it available in some form for reporting purposes?
Organisationally, trying to get an answer out of a Business user how long that they want to keep their data around for is like getting blood from a stone. You will not get an answer and getting them to sign up to any kind of policy which you dictate is next to impossible as well.
So we end up in a situation where it is technically hard to either delete data or archive it and even if we could, we would never get anyone to agree to it anyway. So the default is to keep everything for ever and if you are doing this, the difference between archiving and backup becomes very blurred very quickly.
Now we all know that the right thing to do is to archive, set good retention periods, delete data and we can keep pushing this rock up the hill. And indeed, I would argue that we should but we also need a certain amount of pragmatism.
Archives and backups in many cases have become the same thing; so companies like EMC should not really be castigated for making products which enable this and make the best out of a bad job. But we should as the experts in the field, try where possible to do the right thing; there are cases where doing the right thing is the easiest thing to do, so do it there.
And when someone asks why their backups are taking longer and longer; why they are spending more and more on storage; why their databases appear to have data in which goes back to the dawn of time; you can nod sagely and say ‘Well, you see; backups are not archives but due to business realities, backups have become so but if you really want to move to a better solution, it will require this, this and this…’
I will bet that they will continue to treat backups as archives because to do anything else has become simply too hard. It’s the same old HSM story; it’s too hard…it requires thought and planning, neither of which is overly abundant in today’s IT.
