Customising your Disaster Recovery Plan
By Stuart Lecky
HCi Project Managers sometimes encounter clients who expect us to have a ready-made solution to their particular documentation problem. We explain that, unlike some other documentation consultancies, we’re not about the business of fitting square pegs into round holes, but rather we tailor the peg to fit the shape of their particular organisational hole. That’s why we do an audience/task analysis of the problem, in line with the International Standard for user documentation (ISO 15910). Its also why we produce a detailed documentation plan showing precisely what we’ll do, how long it will take and how much it will cost. There is always a need to tailor documentation solutions to individual needs.
There are similarities between organisations, particularly IT departments. They all have servers and desktops and operating systems and backup regimes, but there are always individual peculiarities that make organisations unique. These differences become most significant when we’re talking about a Disaster Recovery Plan (DRP), which by its nature must be tailored to the needs of individual organisations.
A DRP is the document that you have to have and hope never to have to use. It can be as big or as small as you like. How big and comprehensive you want to make it, or your budget will allow, depends upon how big your infrastructure is, of course, but also upon the amount of risk that you are willing, or able, to bear. Then there are other factors like standards and guidelines that you may be required to comply with, particularly if you’re a government department. Even then you can be faced with the problem of interpreting what can be somewhat sketchy guidelines.
The levels of risk, and the associated cost to cater for them, may well not become apparent until you have conducted an analysis of your needs and how they can be approached.
A “disaster” can encompass any number of scenarios. The obvious ones are fire and flood, which qualify as catastrophic. There are other catastrophic disasters (having lived in Newcastle NSW, earthquake springs immediately to my mind), but the upshot of a catastrophic disaster is that you have to start again. This can mean new premises, even if only temporary, new equipment, even if only leased, new desks, chairs, cabling, telecommunications links, the whole lot. Hopefully your offsite backups and personnel will have survived. Depending upon the criticality of your operations continuing you may have to employ temporary staff to fill gaps. Even if your staff lived through the catastrophic disaster, they might well be at home attempting to ensure that their house doesn’t fall down…any more.
Disasters can also be minor, but significant. The failure of a single server can have a major impact upon the business. If that server is key to the running of that business, getting it back up and running, or replaced, quickly can be of vital importance.
There’s also the added complication of the Business Continuity Plan (BCP), which is frequently mentioned in the same sentence as “DRP”. Ask one hundred people how a BCP differs from a DRP and you’ll probably get one hundred different answers. In truth the two are complementary and the differences qualitative. Some suggest that the BCP covers what needs to be restored to ensure that business can carry on in the event of a disaster, and that the DRP is a subset of that, containing specific details of how this recovery is achieved, particularly for key systems. Another school of thought suggests that the purpose of a BCP is to ensure that everything that an organisation does on a day-to-day basis to maintain and operate its systems be properly and comprehensively documented so that the systems can be run effectively thereby avoiding disaster.
Planning a DRP for a client can be a difficult process, particularly when that client is responsible for only part of its IT operations. When we were asked to do a DRP for a state government department, we faced just that problem. At some indeterminate point along a cable, responsibility for their systems changed to a central government agency that provides corporate IT services across many departments. But the department we were working with did have some responsibility for their own systems both at the their HQ locations and at various regional offices around the state. At the same time the central agency had some fingers in the pies of the regional locations so, not wanting to tread on anyone’s toes we needed to define where those boundaries lay and try to avoid stepping over them unless it was absolutely necessary.
This experience reminded us that we need to be very clear at all times who needs to do what for whom. Based on our experience, these are the primary things to look at when planning a DRP:
-
what are the core business functions?
-
who has responsibility for which areas in the event of a disaster?
-
what needs to be restored, and how is that to be achieved in the event of a disaster?
-
what procedures need to be carried out to effect recovery or to avoid disaster?
Core business functions, or more properly the systems that support them,
differ from organisation to organisation. In the government department mentioned above, we included
these things in the DRP:
-
databases (schemas and permissions as well as the data itself)
-
applications
-
network systems
-
communications systems (including e-mail)
-
intranet and internet sites (schemas)
-
servers, routers and switches (particularly configurations)
-
contact details for both staff and external suppliers
-
specific staff responsibilities
-
detailed backup and restoration procedures
-
backup regime details (what is backed up and when)
-
offsite storage and recovery details
-
details of configurations for servers (hopefully backed up)
Procedures include detail of any activity that needs to be carried out to effect recovery. Or procedures can also be preventative. For example, most computer rooms have a UPS system that kicks-in if the power fails. If power is not restored then there is normally around 30 minutes of stored power in the UPS to allow for an orderly shut-down of the servers, switches and routers. In the example of our government department other procedures that were documented included:
-
how head office could be fed directly from the centralised servers
-
reconfiguring transceiver ports and how to save that reconfiguration
-
restoring from the SAN
-
restoring servers (hardware and software)
-
installing data
-
getting Telstra to move the OnRamp service
Like many documentation projects, problems were encountered during the implementation phase. Some people wanted to expand the scope to include items that hadn’t been planned for, which was fine when they were DRP-related, but some went way beyond that scope. Others considered that some of the things in the scope were unnecessary. For example, procedures concerned with allowing access and granting permissions to systems were thought by some to be everyday procedures, not appropriate in the DRP. However the fact that they didn’t have these documented anyway and that, in the event of a disaster that required re-building the entire system they’d be needed, meant that they went ahead.
Ultimately the customer acquired a living document which, as long as it is maintained in line with changes and amendments to their systems, will serve them well in the unfortunate event that it has to be put into action.
First published August 2003
This article may be reproduced only with the permission of HCi (email HCi ). Copyright HCi, 2001-3.
