The biggest story in the world of digital security this year (2020) has been the one that broke a few weeks ago. It was the Solorigate aka Solarwinds cyberattack that affected hundreds of corporations and the highest corridors of US Government. Over 18,000 locations and several thousand endpoints have been affected by this sophisticated attack, which was presumably by a state-sponsored hacker.
The attackers have infiltrated, maybe through espionage, the source code repository of a popular IT management suite called Orion from SolarWinds. After infiltrating the bad guys were able to sneak in a lightweight trojan code that would get triggered often to escalate privilege and plant additional malware in the target systems. Since the code was part of a well-trusted product, digital code-signed, no one suspected anything. The bad code was in computer systems for many months undetected till the hackers possibly made a mistake to target FireEye, an enterprise security platform. No one is sure of the extent of the attack, or what has been compromised and stolen. Right now, there is no reported threat because of this attack for consumers – but this story is likely to unfold more in 2021. Read here for a detailed analysis of this attack including code snippets from Microsoft (PDF of the Solorigate attack analysis).
It was a coincidence I listened this week to IEEE’s Software Engineering podcast episode 438 titled “Episode 438: Andy Powell on Lessons Learned from a Major Cyber Attack”. Andy Powell is the CISO (Chief Information Security Officer) for the world’s largest integrated shipping company Maersk that operate in 116 countries, in over 374 ports around the world. He was explaining another (equally scary at that time) cyberattack that happened in 2017, that crippled the shipping giant for days. It was called NotPetya – Wired magazine had written a detailed account of it here.
Andy walked through in detail what happened, how the affected Windows Active Directory (AD) brought down their entire network infrastructure consisting of 40,000 endpoints, thousands of apps and over 1000 IP phones. To get an unaffected copy of their AD, backups were inaccessible due to the network being down, they had to fly a person with a precious hard disk from one of their remote Active Directory server in Lagos, Nigeria which was offline due to a power outage for a few days before the attack. It took their partners IBM & Microsoft along with Maersk engineers over 9 days to get the basic system back.
After the attack, they had instituted many procedures for enhancing security, including many systems that can run offline like having Excel backup of critical data. And this one reminded of something we did in my company, nearly two decades back for a different problem of preventing data loss.
It was 1999 or so, we were the Technology & Development partner for a global web portal; in India, they ran a campaign for Diwali where any user could fill up a form online with a message and addresses, which will get printed on premium (physical) greeting cards and be shipped to their friends and family. It was a huge hit. On Day 2, we shipped an update directly to the production server and it included a database update query.
Unfortunately, the SQL update query first emptied the existing records before it did the schema change. You can imagine the chaos; we had lost several thousand entries from users. Luckily, after a few days of hard work, we were able to restore most of the data from our backups and SQL log files.
Following that, myself and the client manager instituted a policy – any application in which we were collecting (typed) data from users we will encrypt it to a long text and email it to a dedicated mailbox which we had provisioned in a third-party mail server with a good amount of storage. This was an inexpensive way for an offline backup. We planned to write a utility which when pressed to action will restore the emails into a database, but in the next few years we had this policy, we never had to resort to using the email backup!