DisasterRecoveryOps for Beginners: A Primer with a Mini-Glossary

DisasterRecoveryOps for Beginners: A Primer with a Mini-Glossary
Tech How-To

Last monsoon, a friend of mine arrived at a railway station with a printed ticket, a charged phone and plenty of time, and still missed the train. A flooded underpass had cut off the usual route, and she stood on the pavement wondering whether to wait or to find another way. Every minute she spent hesitating was a minute she could not get back. That small story is a surprisingly good doorway into DisasterRecoveryOps, the practice of preparing technology services for their own flooded-underpass days. Should the phrase be unfamiliar, start here: the basics come first, and a short glossary closes the piece.

The idea in everyday words

Every organisation runs on systems that sometimes stop: a website, a booking tool, a customer database. DisasterRecoveryOps means planning how those systems come back, setting clear limits for how long that may take, and rehearsing the plan until it works under pressure. The Ops Atlas entry on DisasterRecoveryOps describes it as disaster recovery run as a continuous practice and notes that it grew out of work that began in 1981 and was later reshaped by official US guidance in 2002 and 2010.

Now return to the station. A sensible traveller keeps an alternative in mind before leaving home: a second route, a later train, a friend with a car. She also knows in advance at which moment she will give up on the first plan. Those are the same ingredients technology teams assemble.

Two promises: how long and how much

The first promise is about time. How long can the service be unavailable before the damage becomes serious? That limit is called the recovery time objective. The second promise is about freshness: how up to date must the restored information be? That limit is called the recovery point objective, and the Atlas, citing federal guidance, stresses that the word point is literal: it is a moment on the clock, not a volume of records.

For a travel site, the booking list may need to be restored to within a few minutes of the failure, while a gallery of old photos can be a day behind. Setting those limits separately for different systems is how teams avoid paying for protection they do not need.

Where the clock really starts

Beginners often picture recovery as a button press followed by a short wait. The Atlas pushes against that picture. It says the clock includes the time spent noticing and deciding, not only the time machines need to switch. In its example, people consumed more than half the hour before the technical switch even began, and the target was missed.

My friend at the station learned this the hard way. If she had agreed a cut-off time beforehand, say ten minutes of waiting and then the alternative, she would have caught the later connection. A cut-off decided on the spot always arrives late.

Choosing how to be ready

There is a ladder of options. At the bottom is keeping backups and restoring them after trouble; it is inexpensive and slow. Above that sits the pilot-light arrangement, in which a tiny live core waits to be scaled up. Next comes a warm standby, a scaled-down working copy, and then multiple sites that all serve visitors at the same time. Cost rises with each step, and so does speed.

A city traveller knows the same ladder: waiting for a bus, booking a cab on demand, keeping a pass for a metro, or owning a bike. Nobody uses the most expensive option for every trip, and sensible organisations do not either. Planning a whole city visit works best when you know how much time you have; our notes on how many days a visit to Chennai needs take that approach.

Mini-glossary for first-time readers

  • Failover: moving work from a failed system to a standby, often with a little recent data lost because it had not been copied yet.
  • Switchover: a planned, tidy move between systems with no data lost, used for rehearsals and maintenance.
  • Failback: the later journey back to the original system, which the Atlas counts as a second migration that needs a plan of its own.
  • Fencing: shutting the old system out of service so only one copy is making changes.
  • Tier: a priority level that decides how much recovery effort a system deserves.
  • Drill: a timed rehearsal that tests the plan, ideally with people and machines together.
  • Maximum tolerable downtime: a term from the NIST guide for the longest interruption an organisation could survive.

The order of the move

Failover has a sequence that beginners rarely guess. First fence the old system, then promote the copy of the data, and finally direct visitors to the new location. Reversing the order can leave two systems both accepting changes, which is far harder to clean up than a brief delay. A cautionary note from the sheet says that routing changes alone do not stop every client, because some background tasks keep talking to the old address.

Some real events worth knowing

The Atlas collects incidents that show why the theory matters. A 2021 fire at a hosting site in France caused some customers to lose backups held in the same location. In 2023, a water leak followed by a fire in a Paris-area cloud site powered down one building out of three, and the knock-on effects reached further than many customers had planned for. Even an internal coordination service lost its quorum. In July 2024 a faulty software update disabled millions of Windows machines, which showed that a standby elsewhere does not help when the problem travels with the update. These stories are summarised here at a high level, and the Atlas gives dates and details for readers who want them.

What a beginner can do this week

  1. Pick one service you rely on, such as a shared spreadsheet or a booking page.
  2. Decide how long you could manage without it and how old a restored copy could be.
  3. Name who would decide to switch to the fallback, and by what time.
  4. Try the fallback once, and write down where it stalled.

Doing that exercise on a small scale teaches the whole discipline in miniature. If you also want ideas for exploring the city once your own plans are sorted, the guide to heritage, culture and attractions in Chennai is a pleasant companion.

A note on wording

You may come across other labels for the same territory, such as short forms coined by service firms or vendor phrases built around the word resilience. The Atlas is candid that none of these is an established standard. If you meet one, ask what it promises in terms of time, data age and decision-making, and the substance will usually look familiar.

Why it matters to ordinary people

Behind every online ticket, hotel confirmation and payment, someone had to decide how much downtime was acceptable and how much data could be lost. Knowing that those decisions exist makes you a better customer, a better colleague and, if you work with technology, a better planner. The story from the railway station remains the shortest summary: have another route, set a deadline for abandoning the first one, and practise before the rain starts.