Monday, 8 June 2009

Which are you supporting? The systems or the customer?

Last weekend, the company that used to host this blog (4Uhosting) had a major meltdown, along with several other companies using the same datacentre. The UKGrid Greenheys DC had a major power outage, taking out many servers entirely and effectively causing many hosts to drop off the 'net completely.
A news update on the support page at had the following to say:
"We would like to make it very clear that this incident occured through no fault of ours. It could happen at any facility at any time. We rent floor space in UK datacentres to run our business and we do not expect this kind of thing to happen. Unfortunately, from time to time it does and we can neither predict, or prevent such problems."
Fair enough, if your entire data centre melts down and takes servers out with it, you can't expect to have a great deal of control over the situation and it comes down entirely to disaster recovery. They did, in fact, do a grand job of moving their entire operation to a different DC and getting it all back up and running in a few hours. Unfortunately, while the DC power grid melting might not be their fault, there is certainly backlash.

A lot of customers, upon discovering that their hosted domains and the 4uhosting site itself was offline, went on the internet to express their dismay. Many of them vowed to move their domains away to another hosting provider at the earliest opportunity. I was fortunate in that I found a correlation between the 4uhosting issue and the ukgrid issue so realised that it was a matter of time, but speculation was rife that 4uhosting had silently gone under and many people feared that they had lost hosting, data and possibly even their domains.

Even if half the customers threatening to up and leave actually do so now it's all back online, it represents a large chunk of business. So, the DC melting might not be your fault, but if a disaster situation arises and you don't inform your customers of what is happening, they will assume the worst. And that is your fault.

If your notification system consists of a single webserver hosted in a separate DC that can be used in place of the "real" servers in case of emergency, it would suffice. Give the customers something or they will turn their backs on you. Support is not just about getting the system up and running. It's also about making sure your customers know what's happening and don't abandon you because they don't feel that you are supporting them. It's a fickle thing; you can perform technical miracles and support your users' systems to the best of your ability, but if they don't know you're doing it, they only see the downtime. They only see the negative.

Surely a contingency plan for such a situation would cost less in the long run than the cost of lost business due to an actual disaster situation?

No comments:

Post a Comment