<img height="1" width="1" style="display:none" src="https://www.facebook.com/tr?id=1018517621595528&amp;ev=PageView&amp;noscript=1">
Stats & Trends
Jonathan Crowe
Aug 2017

How One Ransomware Attack Cost Erie County Medical Center $10 Million

Photo by Source

After a ransomware attack in April 2017 took down 6,000 computers, ECMC set about the grueling process of getting the hospital's systems and services back online. More than three months later, officials estimate the total cost of recovery has reached nearly $10 million.

It started in the early hours of Sunday, April 9th. It was 2am, a time when most organizations' offices sit silent and dark. That was far from the case at Erie County Medical Center, a 602-bed hospital in Buffalo, NY, however. As the region's Level 1 trauma center, the emergency room in particular would have been a whirlwind, packed with patients and staff treating anything from car crashes to major burns to gunshot wounds and the other types of extreme injuries that keep the department operating over-capacity an average of 12 hours a day.

In the midst of all that activity, it may have taken a few moments for staff to register the fact that strange message windows had silently popped up on computer screens all across the hospital. One second they weren't there, and it was just another hectic overnight weekend shift. The next, there they were — ransom notes announcing the hospital's files had been encrypted, and the only way to unlock them was to pay 24 Bitcoins, roughly $30,000.

ECMC-SamSam-ransom-screen.png

SamSam ransom screen that appeared on ECMC computers. Source: Buffalo News

An hour and a half later, the medical center's IT team made the decision to shut all computer systems down. From that point forward — for a period lasting more than six weeks — ECMC would be forced to meet all the modern-day demands of a major urban hospital by relying on low-tech, manual processes, some of which hadn't been used in 20 years.

Much more than simply encrypting files on ECMC's computers, what the attack had effectively done was blast the entire hospital back into the past.

No one expected this level of disruption

"It has been a huge task. We're rebuilding the entire computer system."

— Peter Cutler, ECMC VP of Communications and External Affairs

Taking into account advice from law enforcement authorities and security experts, ECMC management determined not to pay the ransom. In addition to not trusting the attackers to actually hand over the decryption keys (or trusting the decryption process to actually work), the hospital decided the only way to maintain the integrity of its systems and ensure the attackers hadn't made any additional malicious changes was to wipe all the potentially affected machines and restore each from from backups conducted before the initial compromise occurred. 

As ECMC's IT staff worked with external specialists around-the-clock to carry out that task and get the hospital's computer system back up and running, the rest of the staff grappled with the challenges of conducting their work offline. 

  • Patient notes were written on paper and circulated by hand for nearly four weeks until editing access to patient electronic health records (EHR) was restored.
  • For two weeks staff operated without full access to email and conducted patient registration manually.
  • For three weeks lab results and other communications were delivered via messengers. 
  • Electronic prescribing was unavailable for a full month.

Veteran staff were able to draw on their prior experience with the manual processes to ease the transition  — some doctors even brought in unused paper prescription pads to use — but for younger staff who had only worked with digital systems, the sudden shift was especially disruptive.

Practically everything that needed to be done took more time. And for a hospital specializing in urgent care and trauma, more time is the one thing no one ever has.

Timeline of ECMC's ransomware infection and recovery

Week of April 2 (initial compromise)

Sunday, April 9

      • 2:00am: Ransom screens appear on computers throughout ECMC announcing the hospital's files have been encrypted and demanding 24 Bitcoins (roughly $30,000) for the decryption keys.
      • 3:30am: ECMC IT staff shuts down all computer systems as a precaution to protect private patient information and prevent the infection from spreading.
      • 5:30am: ECMC management is notified and a cybersecurity consultant is called in.
      • 9:30am: Management team gathers and organizes a response plan originally developed for handling major power outages. It includes going back to paper record-keeping until systems can be restored.

Monday, April 10 (2nd day of recovery)

      • Acting on advice from experts and law enforcement, ECMC management decides not to pay the ransom.
      • The hospital begins to receive and distribute borrowed laptops to critical departments, starting with the emergency room and intensive care.
      • The IT team begins work wiping and restoring affected computers. There will be more than 6,000 of them, total.
      • Over the course of the next week, the Kaleida Health and Catholic Health hospital systems share information system specialists to assist in the recovery.
      • State police and the FBI are notified and aid in investigating the attack.  

Wednesday, April 19 (10th day of recovery)

      • The first of the 6,000+ affected computers that were wiped are re-distributed, with the emergency room and intensive care departments given priority. 

Monday, April 24 (15th day of recovery)

      • Staff can now access the hospital's electronic health record (EHR) system in view-only mode, but still can't make updates.
      • Electronic patient registration is restored for the hospital's emergency, ambulatory surgery, transplantation, dentistry, and direct admission departments. Financial systems are brought partially back online and employees are provided with temporary email.
      • Work begins on establishing a new hospital email system, reestablishing electronic communication with lab systems, fixing the bed coordinating system, and bringing more restored computers back online.

Friday, May 5 (26th day of recovery)

        • Staff can now update EHRs with new notes and begin transfering hand-written notes taken during the outage. 

Monday, May 8 (29th day of recovery)

        • Physicians can now communicate electronically with radiology, the lab, and other departments.  

Friday, May 12 (33rd day of recovery)

        • Electronic prescribing is restored.  

One attack, total cost: $10 million

In July, three months after the attack, ECMC estimated the total costs associated with the incident had reached nearly $10 million. 

According to ECMC officialsroughly $5 million had been spent on "computer hardware, software, and assistance needed in the response." The medical center estimated business losses and increased expenses such as overtime pay added up to an additional $5 million in damages. 

The good news for ECMC is the hospital may be able to recover nearly all of those losses thanks to an incredibly well-timed decision to increase its insurance coverage from $2 million to $10 million just five months prior to the attack.  

That $10 million aside, however, ECMC officials also estimate the medical center will spend an additional $250,000 - $400,000 a month on upgraded technology and employee education to reduce the risk and impact of cyber attacks moving forward.

While the hospital believes it can offset those costs by cutting back spending in other areas the fact remains the financial impact of the attack will continue to be felt for months to come.

5 lessons hospitals can learn from ECMC's battle with ransomware

"What happened to us was a wake-up call for the entire community."

— Thomas Quatroche Jr., CEO of ECMC

It's no secret healthcare is one of the most at-risk industries for ransomware attacks. In a June 2017 memo, the U.S. Department of Homeland Security categorized the sector as both "frequently targeted" and "highly vulnerable," and earlier in the year experts warned healthcare providers that they could expect attacks to double.  

With the risk of ransomware infections continuing to rise and the potential fallout from attacks becoming more and more severe, what lessons can other healthcare providers learn from ECMC's weeks-long ordeal?

1) The very first thing to do is secure Remote Desktop Protocol (RDP)

It's good that ECMC is investing in employee education to help their staff become more aware of ransomware, but the truth is the biggest uptick in successful ransomware attacks against hospitals isn't tied to users downloading malicious email attachments or visiting compromised websites — it's attributed to attackers breaking into networks via RDP (you can read about three more RDP attacks against hospitals here). 

As was the case with the ECMC infection, all attackers have to do is scan the Internet for systems with port 3389 exposed and then launch a brute-force attack to crack weak or default passwords to gain access and execution

The groups behind CrySiS, SamSam, Shade, Apocalypse, and other ransomware are all using RDP as an attack vector. From Q4 to January 2017, RDP attacks spreading CrySiS alone doubled.

That makes scanning hospital networks to determine whether any machines have port 3389 open a top priority. Tools like Nmap can make that process easy, and organizations should keep in mind attackers have access to these types of tools, as well. Here’s a tutorial from DigitalOcean that walks you through how to test your firewall configuration and see what your network looks like to an attacker.

2) An ounce of prevention is worth a pound of cure

Attacks like this one and WannaCry's infection of the UK's National Health Service (NHS) in May are potent examples of just how damaging and disruptive ransomware can be once it's set loose inside a hospital network. 

It's clear that, when it comes to ransomware, detecting and responding to attacks after the fact is no longer enough. Organizations need to prioritize preventing infections at the outset, before they have a chance to do damage. In addition to securing port 3389 (RDP) and port 445 (SMB), that also means supplementing standard antivirus solutions with newer forms of protection on the endpoint. 

To find how Barkly goes beyond antivirus to keep companies protected against the latest ransomware attacks, learn more about how our endpoint protection works.

3) Have a response plan ready

When ransomware infections happen, there isn't much time to react. And in a setting like a hospital emergency room, there's certainly no option to hit pause while you assess the situation and come up with a plan.

"One of the key things that got us through this is we have a plan in place and we practiced," said Dr. Jennifer Pugh, associate chief of service for emergency medicine. 

When ECMC's IT staff was alerted to the infection and made the call to shut down the hospital's computer systems, they knew that decision would have massive ripple effects on operations throughout the hospital. But they also knew there was a plan in place for keeping critical services up and running during major disasters and power outages. The plan was documented and hospital staff had been trained on it. 

That disaster preparedness played a key role in ensuring the disruption caused by the attack wasn't even worse. Now, the hospital has the incredibly unfortunate but also incredibly valuble experience of having that plan play out in real life, and it can make adjustments to ensure it's even better prepared moving forward. 

The key for other providers is to make sure they also have a practiced plan in place before they need it. 

4) Backup isn't a silver bullet, but it does give you options 

A key factor that enabled ECMC's decision not to pay the ransom was the fact the hospital had access to backups that hadn't been encrypted or corrupted during the infection. Having those backups didn't prevent the downtime or disruption caused by the attack, but they did ensure complete recovery would eventually be possible. 

5) Know what constitutes a data breach under HIPAA

According to ECMC officials, no patient data was compromised as a result of the infection, and in that regard the hospital can consider itself extremely lucky. 

In July 2016, the HHS Office for Civil Rights issued a statement that ransomware attacks constitute breaches subject to the HIPAA Breach Notification Rule, specifically because when electronic PHI is encrypted that means the information "was acquired (i.e., unauthorized individuals have taken possession or control of the information)."

It took months for this view to be adopted and enforced, but beginning in Q1 2017 there has been an influx of providers falling in line and reporting ransomware attacks as data breaches

In the eyes of regulators, ransomware attacks now constitute a data breach. That makes it a threat that can't be swept under the rug with backup or after-the-fact detection and response (EDR). 

The reason that's such a huge development is that it changes the way healthcare providers need to think about the damage and harm ransomware causes. It's no longer simply the risk of files becoming encrypted and inaccessible they need to worry about — though as this attack shows that can certainly cause widespread disruption — it's the risk of data exfiltration.

In the eyes of regulators, if providers suffer a ransomware infection, it's assumed that patient data has been compromised unless the provider can specifically prove otherwise. That burden of proof makes ransomware a threat that can't be swept under the rug with backups. All the more reason for providers to make blocking ransomware attacks (not just responding to them) a top priority.

Jonathan Crowe

Jonathan Crowe

Jonathan covers the latest threats and cybersecurity trends from a practical point of view.

blocks.svg

The True Cost of Ransomware

5 Companies, 5 Attacks, and the Reality of Recovery.

Get my eBook

Comments

Stay informed!

Get the latest security news, tips, and trends straight to your inbox.