Why Business Resiliency Needs to Evolve

Why Business Resiliency Needs to Evolve

There is a saying, “Fool me once, shame on you; fool me twice, shame on me.” Nothing could be more accurate when planning to disaster-proof your business! My customer had a ‘bad player’ (hacker) in their systems for about 2 months. They slowly went through their network and started making configuration changes. One of the changes was duplicating the root admin account for their backups. When they initiated the encryption of their systems and demanded a ransom, the response team ignored it and started to recover their data. At the time, I suggested they perform an audit of all their administrative accounts because it seemed too easy for them to recover the data and the bad actor not to take further action. The audit was not performed, and a month later, the customer's day started with all their systems encrypted and a huge ransom demand.

They thought they addressed the threat but did not do their due diligence to investigate all the system changes. A full audit would have prevented them from being fooled twice.

 

If you have followed this series, we have discussed the Plan-Do-Check-Act lifecycle. We have covered why there is a need to plan(what is your risk appetite and identifying your risks), what to plan for (business impact analysis and drafting a plan), how to check your plan (exercise, testing, lessons learned and after actions ) and now we will discuss the Act part(Lessons learned, after actions, program changes) of the life-cycle.

After all, no one wants to be fooled twice, do they?

 

Program Changes or Program Maturity

Sometimes, this step is the hardest to get traction from leadership. After all, they just invested a bunch of resources to build a program and walked through the first three parts of the lifecycle. They read the exercise summary and related after-actions. They may not remember a past incident that could have cost the company, so to them, at this point, continuous improvement equates to continual costs.

At this point, a good consultant will make the case that maturing the program is not a one-and-done but a continuous improvement lifecycle. It takes educating the employees/responders and revamping plans as business functions, software and threats are identified. Eventually, each lifecycle requires fewer resources to refine it. Then, leadership will see even more come from the program, what I call the ‘ROI of BC/DR (Business Continuity/Disaster Recovery.)’

 

Program ROI

Too often, BC/DR programs are check-the-box audit-satisfying programs. I have walked away from more than a few clients who thought that since the program was built and the audit was off their backs, they could end it. As a consultant, I do not build programs, so I, or my team, must be the one to continue the program. Ideally, my role slowly diminishes and, if anything, is more of a check and balance over time.

When leadership fully invests in the maturing of your recovery program, it can transform into a program that provides an ROI. The ideal state of any risk program can ensure issues do not grow to become significant impacts, and those that impact the company minimize the pain (cost, reputation hits, legal, fiscal) to a manageable level. For example, when a past customer had a very mature program, their monitoring systems did identify a potential risk, in this case, a cyber threat.  The initial triage isolated the threat and triggered the escalation process, which convened a formal incident response team.

This specific threat was a ransomware threat. Since the initial threat response was quick, the potential damage was limited to a few systems. The ransom from the bad actors was 350 bitcoin. The response team ensured the data was stored on an immutable backup and was still recoverable, and other than the cost of addressing the impacted systems, they would be out nothing. They planned for this type of issue and practiced how they would respond. They had already identified what information they would need to know to make a well-informed decision.  Due to their program's thoroughness, they could decline ransom payments and fully recover because they had good processes and plans in place. The program had overtime paid for itself in this one response.

This same company was looking to make some strategic changes in its systems. The BC/DR program team I led was invited to the initial exploratory meetings. Within the first thirty minutes of the meeting, I identified several areas of potential improvements in the proposed changes, which would have saved millions of dollars. Subtle little changes, like embedding DR into the design of the system changes instead of going back and doing it after the fact, vetting third-party vendors to ensure they have acceptable recovery systems that could maintain the capacities needed and the ability to make the case for migrating to the cloud instead of a hybrid recovery system that would rely upon obsolete retired hardware.

Through my recommendations, I was also able to align the program with the interests of others through united goals. This ensured that a team would support my program as we could accomplish more united objectives and reduce resource overlap, thereby migrating to a more holistic and cultural existence.  

 

Initiating the Action

Assuming your team has leadership buy-in, you will need to collect the lessons learned and after actions, prioritize them, align them to the different lifecycle phases, and launch a new lifecycle kick-off meeting. Then, meet with leadership and get them to approve the policy changes needed, the areas where the program needs maturing, and how these changes will be implemented. Explain to them that we do not want to do the same thing year after year but take what has been built and build it better.

Ideally, your program will be able to ensure your company’s survival during any impact through the broad scope of planning, education, practice and continuous improvement. Action is not a completion of the lifecycle but the resetting of the lifecycle. It is time to reassess where the program is and identify what was done right and what could be done to improve the recovery planning. It sets up each cycle to improve and encapsulate the objectives of future planning, objectives, and goals.

The ACT part of the recovery planning lifecycle establishes the goals for success, commitment to maturity, and metrics to measure against.

 

Look for the next series of articles on ‘cyber resiliency: recognizing you may be the problem.’

James Knox is a resiliency expert with an innovative spirit who thrives when building meaningful solutions to various daily problems in the corporate world. He is an avid outdoorsman and loves extreme rock crawling, fishing, and hunting. As a survivalist, James has learned from necessity how to prepare for life’s bumps and thrive with practical and sensible solutions, supporting his family's self-sustaining lifestyle.