Defense-in-depth leads cyber strategy for OT security

Some manufacturers believe IT departments can protect their infrastructures, but what happens when that fails or a saboteur is able to bypass the surrounding network and connect directly to the automation system? In this episode, editor in chief Mike Bacidore is joined by Luis Narvaez, who is the product marketing manager for industrial security at Siemens Digital Industries, U.S.A, to discuss network vulnerabilities and who's responsible for cybersecurity on the plant floor, as well as defense-in-depth strategies.

Transcript

Mike Bacidore: Hello, and welcome to today's episode of Control Intelligence. I'm Mike Bacidore, editor in chief of Control Design, and your host for today's podcast. In this episode, I'm joined by Luis Narvaez, who is the product marketing manager for industrial security at Siemens Digital Industries, U.S.A. We'll be talking about network vulnerabilities and who's responsible for cybersecurity on the plant floor, as well as defense-in-depth strategies.

Cyber attacks on the automation infrastructure of facilities in the United States are very real. Securing automation infrastructure can be a difficult task and is more difficult on aging infrastructures with no cybersecurity features and those that are running obsolete operating systems. Some manufacturers believe IT departments can protect their infrastructures, but what happens when that fails or a saboteur is able to bypass the surrounding network and connect directly to the automation system? At that point, the automation system must protect itself against unauthorized access, against malware, against theft of critical software algorithms, and against unauthorized modification. It must be able to detect that it's been manipulated, detect intrusion, and reports cyber activity.

Automation products have implemented cyber protection that when properly configured can provide protection for aging infrastructures. Luis Narvaez is the product marketing manager for industrial security at Siemens Digital Industries, U.S.A. And with more than 10 years of experience in industry manufacturing and engineering, his passion and knowledge of cybersecurity for OT systems makes him one of the most credible cybersecurity experts in industry. He graduated with a bachelor's degree in electrical engineering from the University of Central Florida and has worked in several roles, including controls engineer, systems engineer, and automation consultant.

Luis, welcome to Control Intelligence.

Luis Narvaez: Hey, Mike, thanks for having me on here. That was a really good intro on cybersecurity in the manufacturing space.

Mike: Great. Glad you liked it. So let's start with defense-in-depth. I know we've had discussions before about it and it's one of those concepts that everybody's heard it, but can you give it a little definition? What exactly is the defense-in-depth concept?

Luis: Yeah, sure. So, in general, defense-in-depth is, like you said, it's a term we hear a lot in the cybersecurity community especially as it relates to industrial control systems, manufacturing and automation space, right? And really, it's a term that we use to kind of describe a holistic cybersecurity protection concept. Holistic is another buzzword, I guess, that you hear a lot when it comes to cybersecurity, but, you know, to kind of put this plainly, I like to think of defense-in-depth as kind of like an onion with many layers of protection, right? And the more layers you can, kind of, add, the more difficult it is for a malicious attacker or adversary to basically get to what's at the center or the core of the onion, which, in this analogy would be your automation system that's running your process, your critical processes, or in many cases it could be a power plant, or substation, or whatever the case is.

Mike: So and that defense-and-depth concept that is fairly common within an IT environment say, but for OT, is that kind of a new concept, or is that something that is just as prevalent?

Luis: You know, I think there's a little bit different. So, first of all, I guess, to go a little bit further in-depth on the defense-in-depth concept, right, there are many facets to add those layers to defense and depth. And I should say, within Siemens, we, kind of, like to summarize it into, like, three core pillars, so to speak. And so, if looking from an outside perspective, going further inside the core of the onion, I guess, so to speak, we have plant security, which is intended to protect physical access to your industrial control system, your networks, the building, access control mechanisms, as well as like implementing policies and procedures to help protect against cyber attacks, common cyber attacks in the IT space.

The next layer would be network security. So securing your industrial networks. So that way, we can detect any kind of weird activity going on in the OT network space making it more difficult for attackers to get into the control system. And so, when I talk about things within the network security space, we're thinking things like encrypting communication, securing communication. Think of a phone tap or a wiretapping scheme, right? You know, you see these in, like, the spy movies and stuff like that, where somebody's wiretapping a line and listening in to the conversation, right? That's essentially what encrypted communication handles, things like that as well as network segmentation.

If you think about, going into a building that you've walked in the front door and all you see is this really long hallway with hundreds and hundreds of doors, and each door has a lock basically with a unique key for each door, it's going to be very difficult to find a specific item that you're looking for within that building, right? Because you not only have to know which specific door you're going into, you have to have the key to get into that door, you have to know which door it is, things like that. And so that's the idea of network segmentation.

And then the third pillar that we talk about with defense-in-depth as it relates to industrial cybersecurity is system integrity. And that's really, I like to refer to it as like the last line of defense. It shouldn't be the beginning of your cybersecurity program for your process, but you should do everything you can to implement some of these features that's available in your control components to protect it, or, again, make it more difficult for an attacker to get to the critical pieces. And that could be your PLC program, you know, the program that's running on the control of running the factory. It could be some screens that an operator is using and manipulating values on a screen. So, the operator sees one thing, but in the backend, it's doing something else.

I mean, most industrial control systems today offer some features that can help mitigate the effects of those kinds of attacks and a lot of aging control systems don't. And so that's why you kind of have to build those layers around those components as well. Going back to your original question, I kind of went off a little bit off tangent, but I thought it was a little bit important context, right? So going back to, I guess, essentially what you're asking, and correct me if I'm wrong, is, kind of, what are some of the similarities and differences between IT and OT security, right? 

Mike: Exactly.

Luis: Okay. So I think it, kind of, boils down to just like the basic protection goals for both parties. You know, for OT systems in a manufacturing priority, right, in a manufacturing environment, the number one priority is plant availability. Whenever a cyber attack happens or anything, right, you can't afford to stop production. In the case of like critical infrastructure, right? If you think about a water wastewater system or a water treatment plant or something, right? You cannot stop that process, or power generation plant. We can't afford to have a blackout for an undisclosed amount of time, right? So, there are some critical processes that just cannot stop operation.

From the normal manufacturing space, the implications could be millions of dollars lost in potential revenue, right, because you're not producing products, you're, kind of, at a standstill, and everybody's running around with their head cut off because they don't know how to get it back online, right, because some ransomware happened, right? So, the second protection goal or priority for OT security is integrity. Making sure that the data being produced by the control system, by the automation system is not affected, manipulated, is actually what's happening and trusted, right? And then the third priority, and this is kind of the order, right? And the third priority for OT cybersecurity or protection goal, as we named it, it is confidentiality. Is making sure that the data and the information or any kind of intellectual property, that is, you know, it could be the code that's running on the controller, it could be recipes for a given product that's being manufactured, whatever the case is, is not altered, manipulated, or stolen for whatever reason, right?

In an IT space, we basically flip the first and the third protection goals. So, in the IT space, the priority is confidentiality, protecting company files or secrets or anything like that, right? Obviously, you don't want any kind of malicious actor to get into your systems, know all your employee's passwords, or Social Security numbers, or banking information, right? That could be detrimental to the company, to the reputation and business and everything, right?

And then the second one, integrity, it's still kind of that second priority both for OT space and the IT space. And then on the availability, right? In the case where, I don't know if you remember a few years ago, right, when major retail brands would get hacked during the holidays or something like that, their credit card scams when they would steal the credit card information that's stored on company servers. You know, they would essentially just shut down the websites, right, or whatever. And they can afford to do that because it's not important for the business to have certain IT systems up and running in the event of a cyber attack versus on the OT space.

Again, you're producing goods, you're producing a service or something that could have implications to public health and safety whereas, if you take down a server on a website, a web server or something like that, "Okay, I can't check my Facebook post, but for a few hours."

Mike: But it's not going to affect the world.

Luis: Yeah, exactly. And so balancing the goals between both parties becomes a little bit tricky when you get both parties involved, but I do think they do have to play well with each other because the way technology is these days, there's a lot of kind of overlap between IT and OT, whereas... I don't know, you know, 20 years ago it probably wasn't so much that way. There was a lot of distinct separation between OT and IT.

Mike: More adversarial.

Luis: Yeah, there you go.

Mike: Yeah. It really was. And now, and these days there's so much more working together on a variety of issues just because those, I mean, some of that Ethernet is actually shared anyway.

Luis: Mm-Hmm. Yeah, there is...

Mike: So, there's similarities and the differences... Go ahead.

Luis: Yeah, exactly. I mean, that's what I was going to say was the similarities in just the technology alone has shifted in years. And even in the cyber security space, you're starting to see a lot of technology that's typically used in the IT space for detecting anomalies on the network traffic or being able to detect cyber threats and things like that. You're starting to see a lot of that technology bleeding into the manufacturing space, where vendors, who for many years, they do what they do very well in the IT space. They're having to adapt their products and their portfolios and their teams to then bring that into the OT space. And like I said, I think balancing those goals becomes a little bit tricky because in the case of like an incident response program, or process, or whatever, the priorities are shifted. You can't just unplug things on a factory floor and not know the consequences or not be prepared to handle the effects of that situation. 

Mike: It's much different. There's health, safety...

Luis: Absolutely.

Mike: ...productivity issues.

Luis: Yeah. There's absolutely different issues on the OT side. Mm-hmm.

Mike: So, in terms of providing that security for the OT space, and maybe even as you answer this, maybe even talk a little bit about IT versus OT and I'm sure most people are familiar with the two, although OT is a relatively new term, you know, maybe within the past few years when we used to have to explain it constantly. But in terms of securing the OT network, whose responsibility is it? I mean, does that fall on IT? Is it an operations and maintenance responsibility? Is it the responsibility of the product manufacturer or maybe the system integrator or is it everybody's responsibility?

Luis: Yeah. A lot of people might hate me for this answer, but, you know, and it sounds really cliché, but in simple terms, I think everybody is responsible in some way, shape, or form. When we talk about what the responsibilities are amongst the parties, it's a little bit different, obviously. As one working for a product manufacturer, I cannot say that it's my responsibility to maintain or to make sure that the systems that are installed on a customer site are always up to date. We, as a product manufacturer, we can provide the necessary tools and files to update the products, but it ultimately comes down to who's operating it, who's maintaining it to then perform what's needed to capacity, and things like that. From an integrated perspective, you're the one who's usually initially programming it or commissioning it or setting it up or servicing the equipment. And it, kind of, goes in that space too where, if the customer requests any specific security features or functions to be provided as a deliverable for a project, then the integrator needs to do their due diligence to ensure that the customer's happy and things like that.

On the other hand, the end-users need to specify those requirements. So, it's a little bit of a revolving circle. And that's, kind of, going back to the whole defense-in-depth concept. That's part of that holistic concept. It's that synergy between all the different parties involved, the product's suppliers, the system integrators, and the owners, and operators of the equipment. Everybody has to be involved with the process. If there's any vulnerabilities in the products themselves, the product manufacturer has to come up with patches and mitigations or firmware or software updates in order to fix those bugs or those vulnerabilities, right? And communicate that. We have an obligation to communicate that to our customers. And so it's a kind of holistic handshake.

Mike: Right, right.

Luis: If that's a term to use.

Mike: I like that. I'm going to start using that term now.

Luis: There you go.

Mike: So in terms of, I mean, yes, defense-in-depth itself, it's a huge undertaking that's best executed by multiple parties working together, you know, helping to develop these various layers of the onion, but what about for manufacturers? Are there any simple steps that manufacturers can take just to harden their OT?

Luis: Yeah. And I guess to be clear, I guess when we're talking about manufacturers, we're talking about the owners and operators if I understand your question, right? Okay.

Mike: Right. Yes.

Luis: Yeah. I mean, yep. So, you know, it's a common question. It's a fair question to ask. And when people ask me this, one of the most simplest things that come to mind is taking inventory of what you have. And depending on the size of the factory or the plant, this could be a pretty large undertaking or task. But I think that alone, It's a big first step to harden your industrial control system to put together a bigger cybersecurity program in the organization, right? You can't protect what you don't know is out there. And by understanding what you have installed on the factory floor, understanding the vulnerabilities that are associated with those devices, with those softwares and applications, you then can take the next step into applying any kind of mitigations to protect those devices or protect against those vulnerabilities. Basically, plugging the holes from the leak, right? But you can't plug holes unless you know where the holes are. So, I think that's usually my suggestion.

There's a lot of talk in the industry as well as on something called SBOM, software bill of materials. And I think it goes hand in hand with that kind of asset discovery, the discovery phase of understanding what's installed on the factory floor. You still have to figure out what you have there and then take the next step. And sometimes I will say, I mean, unfortunately, not unfortunately, but sometimes, the best course of action to protect against any kind of vulnerabilities is to modernize the equipment, to update the equipment. That's a little bit later down the path, right? We talked about what's one of the easiest, simple steps that people can do, but I'm just kind of laying out a little bit of a roadmap here, if you may, is once you have that, you know, sometimes the simplest course of action from there is just update to the latest, whatever widget it is that you need to update. And those usually will offer some more enhanced features or less vulnerabilities because of, you know, just newer technology and better performance components and things like that. It's almost a vulnerability in itself to have components that are obsolete. I mean, can you imagine the implications if a controller, a PLC that's controlling a critical process goes down because of some malware or some bug that was downloaded to it, fries the controller, and then the customer is in a situation where they have to recover from that. And they can't get any spares because it's an obsolete product.

Mike: Yeah. 

Luis: And sometimes, you know, that's a critical vulnerability in itself that affects the availability piece that we talked about. 

Mike: So, you've been mentioning these vulnerabilities quite a bit, and in order to plug that hole or in order to fix that problem, obviously, you have to be able to identify where those vulnerabilities might be. I mean, is there a place where someone can go to learn about product vulnerabilities for things that are already installed on the factory floor?

Luis: Yeah. I mean, most, well, there's a lot of product manufacturers, I'll say on the behalf of Siemens. I can't speak on behalf of everyone, but I know for Siemens, we regularly publish any kind of updates or advisories on product vulnerabilities through our openly available product CERT webpage. You can just go on Google, type in Siemens ProductCERT, C-E-R-T, and anything you want to know cybersecurity-wise on our products, you can find it on that page, certifications, certificates, known vulnerabilities, mitigations, things like that. Most product vendors will provide that service already free of charge or you might maybe need to have some sort of login to get that or whatever the case is.

The other place where a lot of that is communicated is usually with federal governments. So, in the case of the U.S. would be CISA. CSIA is a government agency that regularly publishes these vulnerabilities. These are usually communicated from the product vendors themselves to CISA and then, therefore a little bit more openly communicated as well. So, that's one place that people can go to get that information on what's vulnerable.

Oh, I'm sorry, on the product vulnerabilities. Now, a little side disclaimer. Unfortunately, sometimes there are vulnerabilities that are not published. And this is usually because you might be the first victim, right? And those are what we call kind of zero-day attacks or zero-day vulnerabilities. You know, you don't want to be the first, unfortunately, but it happens, obviously. You know, the sad truth is with a lot of the situation going on in the cyberspace, in the cyber protection space, and critical infrastructure being a target for a lot of cyber-attacks recently due to, you know, geopolitical tension, whatever the case is, I anticipate this being a lot more of the case. We're probably going to see a lot more of these types of vulnerabilities, these zero-day vulnerabilities moving forward. And it doesn't mean that products are not secure, right? 

Mike: Sure, of course.

But the good news is once there is a zero-day attack in those vulnerabilities, there is a place where they're reported and collected, whether it's to the product manufacturer or then reported onto CISA and those do become available. And certainly, staffs are probably already checking, but if not that's a great resource for them to know about for sure.

Luis: Mm-Hmm, yeah, absolutely. It's usually communicated pretty quickly. I mean, we just had a major, not we Siemens, I mean, like, the community, the industrial space, there was a major vulnerability reported just a couple weeks ago on some pretty popular industrial control brands. And I think it just shows the flexibility and how quick these cyber attacks are shifting focus, right? It's almost a moving target if you may.

Mike: Right, absolutely.

Luis: And remaining diligent and making sure that owners and operators of this equipment are doing their due diligence of protecting against those threats or vulnerabilities. And then, also being able to detect them and react and have a plan in place that you can react swiftly is very important.

Mike: Right. And that's why the defense-in-depth is so important just because you don't want just one layer of protection. So, once they infiltrate that vulnerability, they're home-free, you want additional layers in place. So, it takes some time, and in the meantime, you can re-shore up those other layers.

Luis: Yeah, absolutely. You know, and it's funny you brought that up. It used to be that there was usually like one layer, right? And another term you hear a lot in the industrial control system, cybersecurity space is insecure by design. And what that means is, what a lot of people refer that to is that the components that are running our factory floors, our production floors, and production systems, be it the controller or the display that the operators are interacting with, whatever the case is, the robots and all that, those products initially were not designed with security in mind.

And so when we talk about vulnerabilities, we're really talking about leveraging the functions that are already available in these products to cause some sort of malicious reaction. So, it's really not that they were designed for the purpose of hacking, right? They were obviously designed for the purpose of producing something and controlling production line or whatever the case is, but they just, you know, and traditionally haven't been designed in that way. And so what used to be the case was they would only be like that one layer and it would usually come in the form of some network firewall  and you'd have all these controllers just like on a network, right? So going back to what I mentioned earlier like you walk into this building and it's a long, long hallway, and you've got a hundred different doors to go into, imagine this same hallway and there's no doors, no walls, everything is visible. You can just walk up to whatever you want inside, you know, in this wide-open space. That's essentially what industrial networks used to look like. Just have one lock on the front door, one layer of that onion. And now we're starting to see a lot more people and technology implement security into the products, we're starting to see people implement security into the OT networks, and that defense-in-depth concept. And so, I went off tangent again.

Mike: Yeah. I mean, well, I mean, you know, it's a great tangent. Back then, there was, you know, 10 or 15 years ago, there was no Internet of Things or cloud-based algorithms. There was, you know, a firewall and you're using a dial-up telephone line to go in through the modem, and you had one-way access. Sometimes it was only going in or sometimes it was only going out. But certainly, with sensing capabilities and data exchange, now that game has completely changed.

Luis: Yeah. And it's changed very quickly.

Mike: Mm-hmm. Yeah. Yes, absolutely.

Luis: And the other thing I wanted to point out with that was that I mentioned insecure by design as a common term to describe, like, components, right, historic components. But even to this day, even though a lot of the technology is being implemented with more security features than they have been in the last 20 years or so, or 10 or 15 years really, the life cycle of these components are intended to last longer than what you would typically see in an IT environment. Like, can you imagine, you know, your home PC is, you know, I don't know, people are probably replacing their tablets or their PCs every two or three years now. After three years, it's running slow and you just, you know, it's got a new operating system, Windows 11 just came out and we thought we were done on Windows 10, and now your Windows 10 PC is obsolete.

Technology changes so fast, but in the automation space and the industrial space, it doesn't change as fast. And so that also, even though the technology, like I said, they're implementing a lot more security features into the product, they're still, I mean, if you got something that you installed 10 years ago, that's still considered a new line, a relatively new line. It's not even...

Mike: It's useful like for decades not years, right?

Luis: Exactly. So, you know, you're not looking to replace your equipment. That's just, you know, under 10 or 10 years old or anything. You're looking for this thing to run 20, 30 years.

Mike: Right. Right.

Luis: And so, it's secure for now, but in five years from now, it's probably considered, you know, a dinosaur in the hacking or cybersecurity community. And that's when that continuous monitoring and continuous patching comes into play and it's very difficult.

Mike: Yeah, that's a great point. Great point. All right. Well, this has been a delightful conversation. Really appreciate it. Anything else you wanted to add?

Luis: No, I think that was it. I mean, if anybody wants to reach out, feel free to reach out to me. I'm pretty active on LinkedIn. I'm always open for conversations. And again, Mike, I appreciate you for having me on this podcast.

Mike: Absolutely. Well, thank you to all of our listeners for joining us on Control Intelligence, the podcast for Control Design Magazine. And thanks of course, to Luis Narvaez of Siemens Digital Industries, U.S.A for his insights into cybersecurity strategies for OT. Thanks again, Luis.

Luis: Welcome.

Mike: If you've enjoyed this episode of "Control Intelligence", don't miss our older episodes and subscribe to find new podcasts in the future. You can find our podcast library at controldesign.com, or you can download all episodes via Apple Podcasts or Google Play.

For more, tune into Control Intelligence: The Podcast from Control Design.

About the Author

Control Intelligence: A Podcast from Control Design

Welcome to Control Intelligence, a podcast that goes deep inside the automation and technology that machine builders, system integrators and end users rely on to keep production humming efficiently.