You read the water advisory. You nodded at every line. Then you forwarded it to someone else.
I did the same thing, and I want to talk about what happens to it next, because I do not think we are being honest about who this was designed for.
On July 30, the FBI and EPA issued a public service announcement about malicious actors gaining access to internet-facing programmable logic controllers at water and wastewater utilities. The technical account is unglamorous, but is worth reading in full. The actors accessed exposed devices, changed the IP addresses, and set passwords. Operators lost view and, in some cases, control. Reported effects included loss of pressure and flooding. The PSA notes that pressure loss can allow untreated groundwater to seep into pipes.
CISA has since said it observed the same activity against more than 100 internet-exposed systems in the water and wastewater sector. Rockwell controllers first, then Schneider Electric, then Siemens.
BlastWave’s CEO Tom Sego did a webinar with UTSI’s CEO Shaun Six on the water utility compromise on August 26th, and one of the slides was titled "What You Can Do On Monday," which outlined their suggested action items.
The FBI and EPA published their own list, and although it sounds reasonable (count the jobs, not the bullets):
- Disconnect the controllers from the public internet and broker access through a secure gateway.
- Secure the cellular modems used for field connectivity with strong authentication, up-to-date firmware, and logs that someone reviews.
- Consider a private APN (or 5G non-public network integration), cellular SD-WAN, ZTNA, or a site-to-site VPN.
- Set unique complex passwords on every device.
- Configure firewall rules and access control lists so only expected control-system devices can talk.
- Put the key switches back in run position and leave them there.
- Validate project files against known good logic using vendor integrity tools before you do.
- Review logs on every connected modem, HMI, and workstation for signs of lateral movement, and reimage anything that looks compromised.
- Maintain a rolling twelve-month end-of-life forecast, reviewed quarterly with owners and procurement, tracked by product, owner, location, and retirement date.
- Practice manual operation, with tested fail-safes, islanding, backups and standby systems.
That is not a checklist. That is a network engineer, an identity administrator, a configuration manager, a backup and recovery specialist, a log analyst, an incident responder, a procurement planner, and a continuity coordinator. Eight or nine distinct specialisms, each with its own tooling and its own failure modes.
And every one of them is correct. I would sign off on all of it.
The reality of implementation
The EPA's own figure states that more than 97% of the country's 156,000 public water systems are small, serving 10,000 people or fewer. The utilities in this campaign were largely municipal, largely rural, in places where one licensed operator is responsible for sampling, treatment, maintenance, regulatory compliance, and the after-hours callout when something alarms at two in the morning.
That operator is now also required to be the network engineer, identity administrator, log analyst, and procurement planner. Not because anyone made a conscious decision, but because the FBI/EPA list arrived and no one else was in the room.
This is a common failure in staffing shortages, and its impact is compounded because the ten unstaffed jobs are now vital. As a result, a single person is expected to be an expert in nine unrelated disciplines. Unsurprisingly, they will be mediocre in all nine; competence does not scale by assignment. You cannot make someone a specialist just by adding the specialty to their job title.
Honestly, the advisory is not wrong. It is, so far, unaddressed and, without considerable funding and headcount expansion, unaddressable. Which is why, a month after the guidance went out, CISA is still finding the same controllers sitting on the public internet, happily waving at every passing scan.
The attackers do not have the same problem
CISA has said the recent activity partly leaned on AI tooling, using publicly available information to build scripts against vulnerable Siemens controllers. The FBI PSA also noted something else: across several victims, similar network setups supplied by the same third parties gave the actors a way to multiply one success into many.
Put those two together, and you get an attacker who has an arbitrarily deep bench. Scanning is a job that never sleeps. Script generation is a job that costs nothing to duplicate. Finding the next utility that bought the same integrator package is a one-time job. None of it required a genius. It requires narrow, boring, repeatable work performed at a volume no municipality can match.
The defense/attack asymmetry solution everyone reaches for is “skill”. When one side can staff with AI to the size of the problem, while the other side has just one person who, on top of everything else, also has to take a chlorine residual reading this afternoon, skill alone will not cut it.
The only controls that survive a workforce of one
If you accept that skill alone is insufficient, the question changes. It stops being "what should the utility do" and becomes "which controls still work when nobody is watching them."
Most of the mitigation list depends on sustained human attention. Passwords get rotated by someone who remembers. ACLs stay correct because someone maintains them. Key switches go back to “run” because someone walked over and did it manually. Every one of those is a policy, and a policy is a promise that a specific person will keep doing a specific thing indefinitely, in a job with high turnover and no backup.
A smaller number of controls hold on their own, because they are properties of how the system is built rather than tasks somebody performs. Removing inbound exposure is the clearest one on the FBI's own list, and it is first for a reason. An asset that does not answer an unauthorized scan does not depend on anyone remembering anything. Reconnaissance is the first step in these intrusions, and it is the cheapest step to take away.
I want to be careful here because this is where many vendors overreach, and I will try not to do that. Removing exposure does not help against someone already inside with valid credentials. It does not tell you what is in your plant, nor does it excuse you from knowing. It is not a guarantee. Removing exposure is one of the few items on that list that keeps working on a Tuesday evening when the operator is home with their family.
There is one more line in the FBI advisory, and almost nobody quoted it. Explaining why the damage varied among victims, it lists the factors: what the controller was doing, which model it was, what function it supported, and the water utility's ability to switch to manual operation.
Manual operations. In the middle of a federal cyber advisory, what determined how bad it got was an operator who knew the facility well enough to run it by hand when the automation started lying or failing.
That is the single person the whole list was addressed to. We spend a great deal of money trying to replace them and almost none making sure there are two of them.
The advisory will be reissued the next time this happens, and the next one will be longer. Which is a shame, because it could be a far, far shorter list. Or avoided entirely.
FAQs
Why are water and wastewater utilities still vulnerable after federal cybersecurity guidance is issued?
Many water utilities lack the dedicated cybersecurity staff needed to implement and continuously maintain every recommended control. Small municipal and rural utilities may rely on a single operator who is already responsible for treatment, sampling, maintenance, regulatory compliance, and emergency response.
What did the FBI and EPA recommend for internet-facing water utility control systems?
The guidance recommends measures such as removing programmable logic controllers from direct internet exposure, securing cellular modems, strengthening authentication, restricting communications with firewall rules and ACLs, reviewing logs, validating controller logic, planning for end-of-life equipment, and maintaining tested manual operating procedures.
Why is internet exposure especially dangerous for PLCs and other OT devices?
Internet-facing OT devices can be discovered through automated scanning and reconnaissance. Once attackers identify an exposed controller, they may attempt to authenticate, change configurations, manipulate settings, or use information about one deployment to identify similar systems at other utilities.
Why is cybersecurity staffing a major challenge for small water utilities?
Effective implementation can require expertise in networking, identity management, configuration management, incident response, log analysis, backup and recovery, procurement, and operational continuity. Smaller utilities rarely have specialists for each function, leaving existing operators responsible for cybersecurity tasks outside their primary expertise.
Which OT cybersecurity controls require the least ongoing human intervention?
Architectural controls that reduce exposure can continue protecting systems without requiring constant attention. Removing unnecessary inbound internet access and preventing unauthorized systems from discovering OT assets can reduce reconnaissance opportunities without depending on someone continuously reviewing or updating a policy.
How can network cloaking help protect understaffed water utilities?
Network cloaking makes protected OT assets invisible and inaccessible to unauthorized users and devices. Because unauthorized scans cannot discover the protected systems, it can reduce the attack surface and eliminate a common reconnaissance step before an intrusion. BlastWave describes network cloaking as preventing direct scanning and enumeration of protected OT devices.
Does removing internet exposure eliminate every OT cybersecurity risk?
No. Removing exposure does not protect against every threat, including an attacker who has already gained authorized access or compromised a legitimate user or device. It should be part of a broader defense-in-depth strategy that includes access control, segmentation, secure remote access, backups, monitoring, and operational preparedness.
Why are manual operations important during a cyber incident?
If automation or remote visibility becomes unavailable, trained operators may need to run critical processes manually. Maintaining tested manual procedures, backups, fail-safes, and standby systems can help utilities continue providing essential services while compromised systems are isolated or restored.
How To Reduce OT Cyber Risk When Your Water Utility Has Limited Cybersecurity Staff
1. Identify every internet-facing OT asset
Inventory PLCs, HMIs, cellular modems, remote access systems, gateways, and other control-system components that can be reached from outside the facility.
2. Remove unnecessary direct internet exposure
Disconnect controllers and other OT devices from the public internet wherever direct connectivity is not required. Route legitimate remote access through a controlled security architecture instead of exposing individual devices.
3. Make protected OT systems undiscoverable
Use network cloaking or another architecture that prevents unauthorized users and devices from discovering protected systems through network scans. Reducing reconnaissance opportunities removes one of the easiest steps attackers can automate at scale.
4. Restrict communications to what operations actually require
Apply least-privilege access policies so only authorized users, devices, and applications can communicate with specific OT resources. Avoid broad network access simply because a user has successfully authenticated.
5. Secure remote access
Replace exposed or weak remote-access methods with authenticated, encrypted access that limits users to the systems they actually need. Where possible, eliminate password dependence and require verified identities and authorized devices.
6. Reduce opportunities for lateral movement
Segment critical OT systems so compromising one workstation, modem, or device does not automatically provide a path to the rest of the control environment.
7. Preserve known-good configurations and backups
Maintain validated controller logic, configuration files, system images, and backups so compromised or altered systems can be compared against trusted versions and restored when necessary.
8. Maintain a practical manual operations plan
Document how essential water or wastewater processes can operate when automation, communications, or remote visibility is unavailable. Regularly verify that operators know the procedures and that fail-safes, backups, and standby systems still function.
9. Prioritize controls that remain effective without constant attention
For utilities with limited staff, favor security controls that are enforced by architecture rather than those that depend entirely on someone remembering to review, update, or manually maintain them every day.
Jaguar Land Rover’s cyberattack shut production for five weeks. The lesson: limit blast radius with network cloaking, segmentation, and verified access to OT.
Explore the complete analysis of 23 OT attacks that defeated firewalls, VPNs, and air gaps.
Less than 8% of OT and ICS Companies Have Robust AI Deployment, According to new Nozomi Networks and BlastWave-Sponsored Takepoint Research
.svg)
.svg)
