Like beauty, excitement is in the eye of the beholder. What might have been mind-numbingly boring for some, was completely riveting to me. My colleague and I were at a remote water treatment facility with all of the sights, sounds, and smells that accompany this most important piece of critical infrastructure. We were in a large conference room with 20-30 heads of neighboring water authorities, with a classic “U” shaped conference set up with rows of chairs in the back. The side table had local doughnuts, juice, and coffee, and the lighting was more 70’s fluorescent than modern LED.
Two vendors representing a digital platform were delivering PowerPoint slides on the benefits of moving workloads to, and connecting their various operations with, the cloud. The uncomfortableness in the audience was palpable. After having presented the miraculous benefits of the cloud for 30 minutes or so, one of the vendors finally detected the audience's skepticism and asked a simple question, “How many of you see yourself moving portions of your operations to the cloud?” Crickets...Not a single person raised their hand. The vendor’s jaw dropped, and it took him a few moments to compose himself for the obvious two-word follow-up question, “Why not?” One grizzled veteran who was older than the combined age of the two vendor reps, answered, “Because we are air-gapped (with an implied, ‘you idiot’).” That was the end of the discussion for them, but just the beginning for me.
Air-gapping has been a traditional risk reduction approach, especially in utilities like power and water. The idea is that nothing is connected to the Internet, so all of the remote attack vectors that vex financial services and manufacturing companies today with ransomware, can’t jump that air gap. Phishing campaigns, man-in-the-middle attacks, SQL injection, web-based attacks, and social engineering are dead in their tracks. At least that’s the theory. And, in theory, as in the past, it’s a solid way to reduce risk. But, unfortunately, today’s post-Covid world is different.
In one study by a cybersecurity consulting firm, out of 237 assessments performed at air-gapped water authorities in the US, only six did not have any connectivity to the internet or a way to bridge the air gap. That means that 97% of organizations that THOUGHT they were air-gapped were not. They had a false sense of security and misplaced peace of mind. How is it possible that such a large majority were unaware of the posture of their facility?
Well, there were four main ways in which the air gap was defeated, oftentimes, unbeknownst to the SCADA systems manager or supervisor.
The first method was that, as a result of Covid, many vendors and some employees had set up jump hosts with VPN connections to gain remote access. Similarly, certain employees or vendors might set up a temporary hotspot on their phone to get software packages to perform updates or troubleshooting. This wasn’t always persistent remote access, but this connectivity was established quite frequently. In the mind of the water authority top brass, the system was still air-gapped, albeit not all of the time.
A second method is one in which vendors and staff that perform software updates and troubleshooting would either connect their laptops or removable USB sticks to the SCADA network machines, which can introduce malware, much like Stuxnet. One water authority told us about an integrator who had five different customers and would plug in the same laptop to whichever plant they visited. In the process, the integrator had infected their network with Conficker (a Windows-based worm) on three different occasions within a year.
A third way in eliminating the air gap is exemplified by what happened in the Oldsmar water attack in Florida. Remote access was accomplished, not through a jump host, but rather by installing Team Viewer or some other remote desktop tool. Unlike the intermittent methods like hotspots, and temporary jump hosts, this was persistent and almost led to lye poisoning of an entire community.
The last method of bypassing an air gap also involved persistent access. Some workstations and human machine interfaces (HMIs) employed “dual-homing.” Dual homing results from having multiple network interfaces on a single machine. In the case of water utilities, a workstation might have two NIC cards with one connected to the SCADA network and another connected to the business network. This configuration essentially bridges the air gap, so that any nefarious actor who gains access to the business network can run a scan and discover devices with legacy operating systems (e.g. Windows 7, XP, or 2000). It should be fairly obvious even to non-hackers that there is a very high probability that those operating systems are connected to the critical infrastructure controls architecture and not the business network. Enterprises typically upgrade their business PC’s every 3-5 years, and maybe as long as 10 years in the case of water and municipal services which are underfunded and understaffed. Older machines are almost always associated with the control network. And, once an adversary has identified the legacy hosts, they are off to the races by selecting any among hundreds of exploits to compromise that machine. It goes without saying that many legacy devices are not patchable, since the vendor has long eliminated support of those operating systems.
So, if almost all air-gapped water authorities aren’t really, what could the vendors and SCADA operations personnel do to meet their needs of ad hoc remote access, while at the same time having the benefits of an air-gapped network? And, what is it about an air-gapped network that is so attractive to utility operators?
Air gaps are appealing because there is no path for a remote adversary to use in performing OT system reconnaissance, the first step in their standard attack process (also known as the Cyber Kill Chain), unless they have physical access. Physical access is difficult to obtain from China, North Korea, Iran, or Russia. So, having an air gap greatly reduces the attack surface, which is a qualitative assessment of exposure and risk. However, not even an air gap addresses the issues related to a malicious insider or inadvertent vendor human error that does have physical access. In an ideal scenario, a water authority could configure their network in such a way that it is unscannable, or cloaked. Cloaking alone would be insufficient, but if combined with the ability to drop packets that lacked a digital signature or signing key from an authorized and authenticated employee or vendor, a utility could gain the benefits of an air gap in reduced attack surface while enforcing allow zero trust network access for employees and vendors. And, to take it one step further, this novel solution could execute policies that would give very fine-grained control and access to the least privileges needed by that employee or vendor. They would only be able to see and connect to those assets they are authorized and authenticated to see and connect to.
That would be the “nirvana” solution, that would give the benefits and functionality they need without the risks. It would be the best of both worlds, and one that could be described as a “Virtual Air Gap.”
Well, it just so happens that BlastShield can deliver exactly that. BlastShield is the world’s only peer-to-peer based software-defined perimeter purpose-built for OT, delivering zero trust networking to SCADA systems and industrial control systems (ICS). The nature of a software-defined perimeter as discussed in NIST 800-207 allows hosts to be “darkened” or cloaked from untrusted users and resources. Having a peer-to-peer architecture allows much faster performance and reliability. As any water authority person will tell you, their two most important priorities are to keep the tap flowing and prevent sewage backup. Availability and reliability trump everything else.
By using BlastShield, water authorities can get the availability they must have, while also gaining the convenience of secure remote access and network segmentation. And, the best part is they can deploy at their own pace, one lift or pump station at a time. Why? Because it deploys as an overlay, so the underlying network doesn’t have to be changed at all to gain this protection and connectivity.
Maybe after learning about the myth of the air gap and mechanisms to get the network you want with the network you have, you might find this as exciting as I do.
Start a free trial now using at: https://www.blastwave.com/free-trial
Experience the simplicity of BlastShield to secure your OT network and legacy infrastructure.