Disaster recovery lessons learned: 5 takeaways from 3,900+ disasters

January 14, 2019

By Sungard AS

We’ve helped businesses recover from more than 3,900 disasters in the last 40 years.

We’ve seen hurricanes, cyberattacks, floods, fires and plenty of power outages. We’ve also seen some unprepared companies be nearly wiped out by natural disasters, terrorist attacks, cyberattacks and more.

What differentiates companies capable of successfully responding to disasters from those who go out of business following them?

Here are five disaster recovery (DR) lessons we’ve learned from thousands of disasters that can help you prepare for the worst.

  1. Most disasters pose two major threats. Traditionally, the primary threat of a disaster has been displacement. Hurricanes, wild fires, flooding, power outages or even a car through the front of the building can displace your employees if they shut down your facilities. In this case, employees need an offsite recovery center where they have power, connectivity and access to a replication of your network and data to get back to work.

Increasingly, however, data loss is a major threat. Cyberattacks often happen weeks or months before anyone notices, which means malware can spread potentially corrupting your backups. Without a segmented network or multiple copies, your data is at risk.  

  1. Changes in production derail DR plans. You’re ready for a disaster. You have your DR plan fine-tuned. But when’s the last time you made changes to your environment? Often companies upgrade their compute, memory, or storage, or make other changes in production without corresponding adjustments in their DR plan.

Evaluating changes in production to see how they impact your recovery configuration is an easy way to avoid a slower, more complicated, or flat-out impossible recovery when the worst-case scenario hits. Whether you’re handling DR in house and need to be more vigilant, or you use a partner and need to shift your contract to ensure your new configuration is covered, make sure you account for every change in production.

  1. The cloud is manna for DR. Years ago, one of the best options for storing data to recover offsite was tape backup. Once you copied data to the tapes, you’d physically move them to an offsite storage location. If the data center went down, you would drive out to get the tapes and recover from them.

While tape backup may still be a viable option for some businesses, now we have the cloud. It’s much more seamless and faster than tapes, not to mention that storage replication across a dedicated network ensures site-to-site recovery and prevents data loss. It’s also more cost-effective, since you only pay for what you use, and it scales with your data.  

  1. Some recoveries are more difficult than others. Some organizations have one type of system, and some have five, seven or more. Not all computers are created equal, and if there’s a vast disparity in the types of processing they do, you might need multiple recovery systems as well.

Integrating those multiple systems in production creates daily tasks to ensure they stay integrated. When you go to recover, those same considerations must be brought into recovery.

  1. RTO is relative. The key question in disaster recovery is, “How long can you be down before you have to be back up and running?” The answer to that question varies widely by company and industry.

If your business is 100 percent online, you’ll have a low tolerance for downtime. If you’re a manufacturing company with a queue of production orders to work on over the next several weeks, and you lose the ability to queue up the next order, you might still have projects to work on for several days.

In any industry, office workers using workstations on top of desks could move to a different desk or different building and do the same tasks with minimal interruption.

Regulatory requirements for financial services and other industries might dictate RTO as well. It’s also important to remember that tiering your applications can get you back up and running faster.

Disasters can happen when you least expect it. You might not be able to prevent the storm, but you can prepare your company to handle the situation by taking the necessary precautions to keep your business up and running with minimal disruptions.

Paying attention to these five lessons is a good place to start. This is what the most resilient companies do, and it’s why they’ve stayed afloat while others have not.

Other Posts You Might Be Interested In