Lessons Learned from Ransomware Attacks

You could wait for a ransomware attack to teach you some hard truths about combatting these breaches. Or you could step up your game right now with hard-won lessons from organizations that have already been there. At the 2021 Secure Iowa Conference, two CEOs took the stage with a commitment to helping others learn from their ransomware experiences. In this post, you’ll step inside two organizations’ war rooms as they manage a ransomware attack—and share best practices we all can follow to stop these attacks, or at least limit the damage.

The Attacks

In June 2021, Des Moines Area Community College suffered a ransomware attack that made national news. The school, Iowa’s largest community college, has six campuses, 1,880 employees and more than 72,000 total students. The ransomware attack forced the closure of in-person classes for one week and online classes for two weeks. DMACC CEO Rob Denson joined the conference panel to discuss the school’s experience.

Rob Denson was appointed the fourth President of Des Moines Area Community College on November 1, 2003.
In addition to his DMACC position, he serves on the National Board of Gateway to College, a drop-out recovery program; the Governor's STEM Advisory Council and Executive Committee; the National STEM connector Innovation Task Force, and the Food and Ag Council; and, the National Leadership Council of Opportunity Nation. He also chairs the National STEM connector Higher Education Council and serves on the boards of Iowa Student Loan Liquidity, the Iowa Ag. Literacy Foundation, the Technology Association of Iowa, the Iowa Quality Center, the Agri-Business Association of Iowa, the Iowa Direct Caregivers Association, the Iowa Rural Development Council, the Greater Des Moines Partnership, the Iowa Innovation Council, and the Iowa Economic Development Authority.

In the summer of 2020, hackers launched a ransomware attack against EFCO, a Des Moines-based manufacturer that serves customers worldwide with its concrete forming and shoring products. EFCO President, CEO and Director Scott Walter joined the panel to tell his team’s story.

Scott Walter has been with EFCO since 2008 and in his current position since 2020. He is responsible for the strategic direction of the Company and oversees the management of manufacturing, sales, distribution, and finance. While with EFCO he has held positions in manufacturing and information technology.

Q: How were you first notified about the attack?

Rob: I was driving on vacation when I got a call that a student received a phishing message in a computer lab and gave up their credentials, which let the bad guys go in with Ryuk ransomware. I kept driving and got hourly updates from initial interactions with our insurance company.

Scott: Coming from IT, I was used to getting calls at night. And now being CEO (for only two months at that time), I was used to hearing about crises coming up at any time. This call came at 9pm. In hindsight, I think our initial reaction was an under reaction.

Q: What was your team’s first step?

Rob: We waited 24 hours for the insurance company to get everything place. We hadn’t done any practice runs, which I recommend you do. I hadn’t paid enough attention as CEO to all the crazy acronyms and company names. It was an unbelievable learning experience.

Scott: We worked through the night to shut down the network and stop the spread. Then we started working on identifying the extent of the attack and what recovery would look like. We met in that war room every day for a couple of weeks.

Q: What were your initial discoveries?

Rob: We found a ransom note on a computer in one of our satellite campuses. This group went searching for anything labeled “confidential” and found one of our VP’s files that had nothing in it but very old personnel data. In the end, we paid no ransom.

Scott: We found out that 50% of our servers were encrypted and wasted about a day trying to find the right vendor to help us out. Within 5 days (counting a weekend) I set up a sandbox with our dev team with our ERP system to run the business. We had 10 people taking calls from around the world to enter things into the ERP within that sandbox.

We kept a close eye on everyone’s energy level and ability to make decisions. You’re making critical decisions around the clock and looking for critical path to get back up and running.

Q: How was your cyber insurance experience?

Rob: We had great service, but our premium went from $30,000 last year to $100,000 this year. To not lose time in our next situation, we put the consultants we used on a retainer to stand by so that we don’t have to wait for insurance.

The business interruption consultants tagged our business loss at about $950K for the fall term due to students giving up on registration. It will be a great help if we can recover that money through our business interruption insurance.

Scott: When you have the whole company shut down, the damages are impossible to estimate and impossible to validate. We got minimal help there, I would say. 

Q: What were other business impacts?

Rob: They got our active directory, which I’d never even heard of. We didn’t have MFA, which would’ve sounded like an obscenity to me before this. We had thousands of e-mail addresses to put on MFA. That took a heck of a lot of effort.

We had to decide which systems were a priority to restore. The first thing we did was get payroll back up and get financial aid flowing back to our students, many of whom are low income. Then we went to registration systems.

Scott: In prioritizing systems and locations, we focused on the customer. We’re always shipping and returning equipment from customers every day. We had offline processes to handle that for a short period. Eventually, we’ll miss a billing cycle. Eventually, we’ll miss a payroll run. So that’s how we prioritized. We know which district offices do the most business and prioritized those first, knowing we’ll have to scan and clear every computer before it rejoins our network.

Q: What kind of media and notification situation did you face?

Rob: We contacted the FBI and had to notify the U.S. Department of Education and the Iowa Department of Education and our board. But most important was communicating to our own faculty, staff and students. We kept sending out e-mails and put up a daily note on our site, mainly reassuring people that very few names had been disclosed. The media was beating down the door, and the lawyers told us to just refer them to statements on our website.

Scott: We have a low profile in our city, and much of our operations are elsewhere, so it didn’t make the news. A blogger did pick up on our attack. They didn’t name us, but they gave us some new information because it was a new piece of malware we were facing, and they’d seen it on the dark web.

Law enforcement was first on my mind, but I was surprised to learn that our consultants said not to call the FBI. We notified customers, shareholders, employees—anyone who had information that may have been compromised. We didn’t know what servers the hackers had been on yet.

Q: What is your team saying about the breach now?

Rob: We’re on to dealing with the Delta variant. We did go to MFA. We’re doing more frequent passwords changes. Computers lock after 15 minutes of inactivity. But overall, we’re on to the next emergency.

Scott: It’s a significant event in our recent history, so it sticks in people’s minds. I had been part of starting an info security committee in the company a couple of years earlier, but we were weren’t disciplined about holding meetings and reporting to the board. The committee was reluctant to put in place security things that would make people’s jobs slightly harder and cause pushback. But now, nobody wants to go through this again.

We never knew exactly who attacked us or how they got in. That makes it hard to tell that to the team in a way they can understand it and in a way that’s applicable to their work and how they can do their part to protect it.

Q: What do you know now that you wish you’d known then?

Rob: You need to immediately get your IT people and your insurance on the horn. Identify your most likely consultants and reach out to them in advance. They can help test your system to confirm steps you’ve taken to become a harder target. 

Scott: Our experience validated that backups are gold. We had the option to not pay the ransom, and we lost very little data in the process. But I wish I would have known how important forensics would be. We needed a clean network to be reconnected and to scan for malware on all the machines. And forensics are also critical to knowing what was lost.

We also needed more focus on the prevention side of things. Minimally, we needed to be able to recover. But now we need to focus on prevention. We found out during our investigation that Microsoft Defender had detected this and was not configured to notify our IT department.

If this discussion gets you thinking about your own readiness for a ransomware attack, contact HBS today for a free consultation.