Hello. My name is Nataliia Neshenko. I am a PhD candidate in Florida Atlantic University College of Engineering & Computer Science working with Professor Fuhrt and in collaboration with Professor Bou-Harb from the University of Texas in San Antonio. Today, I’m going to present our Behavioral-Based Forensics and Investigation Approach for Analyzing Attacks on Water Plants Using GANs.
I first will provide the background and motivation behind this work, go over the related research in the field and demonstrate the edit value of our work. From there, I’ll dive deeper into our approach and its benefits and evaluation strategies and present a couple of results to demonstrate this approach and elaborate on a future extension and conclude this presentation.
Water is the most valuable resource in the planet, and this is why economic efficiency and quality are top priorities for the water industry, while they still remain challenging. The consistent end-to-end digitalization of water facilities supported by technological vendors helps the water and wastewater industry meet these challenges. These industrial control systems which provide partial or fully automated control for critical infrastructure are now accessible through network communication and support economical, efficient operations.
But what appears to be a simple and efficient solution is not that simple. With [inaudible] environment the attack surface keeps expanding and if adversaries gain remote control over the automated operations, the fresh water supply and quality can be seriously compromised.
This is unfortunately not a theoretical threat and the first attack against network-connected water facilities was recorded decades ago in Australia, where around 127 people were put in danger by the polluted water due to a cyber attack. A cyber attack can also target water meters, and in order to manipulate the billing system and cause economic loss, which happened in Pennsylvania. A partially successful attempt to poison water was recently conducted in Florida.
And these are not isolated incidents, but rather examples of many other attacks. An increased number of cyber attacks against critical infrastructure of vulnerable or network-assessable automated system of critical infrastructure and water facilities and limited forensics investigation methods with asset attribution capabilities, specifically for ICS deployed in water plants motivated us conduct to conduct this research and address these challenges.
So we introduce an unsupervised method that can fingerprint system behavior under regular operations and infer attacks and attacked ICS assets for their further prioritized investigation.
Researchers around the world approach this problem from two different perspectives: control-theoretical and data-driven. Data-driven includes various machine learning approaches. And one of them is generative adversarial network, known as GAN. And there are also a couple of methods that can be employed for anomaly attribution or for ICS asset distribution.
Control-theoretical approaches assume the existence of mathematical models of underlying ICS. It basically takes ICS and converted it to this mathematical model and then leverages it for attack inference and their investigation. Is it computationally expensive? Yes. Is it practical? Well, ICS deployed in various water plants can have compatible structure, depending on their types.
For instance, domestic and industrial wastewater treatment require different filtering and different disinfectant technology, as well as advanced treatment such as oil separation, removing toxic or organic ingredients, and so on.
Furthermore, ICS assets are prone to continuous modification, which requires thorough maintenance of the mathematical model and regular updates. There are many other reasons that make maintaining this mathematical model a challenging task for control-theoretical approaches.
Given this, using the identical model-based target detection technique for ICS deployed in different plants seems to be unrealistic. Therefore, our approach generalizes the underlying system by employing a learning algorithm that operates on circulating ICS data.
Many machine-learning methods are employed to infer cyber attacks. Some examples of the employed methods include Deep Neural Networks, Support Vector Machines, Autoencoders, Dual Isolation Forests, and many others.
For many machine-learning methods, anomaly interpretation is a well known issue, while it is an essential future towards prioritization for cyber forensics, and also attack-related data is rarely widely available due to security and privacy reasons. Therefore, our approach, we kept this in mind when we designed our approach and our approach operates multivariate data and can be trained only in attack-free instances, and it also pinpoints attacked ICS assets.
One of the machine-learning algorithms that we will employ actually in our work is widely useful anomaly detection across different domains such as health care, public safety, finance, and cybersecurity. And these networks is known as generative-adversarial networks, GAN, and different variations of underlying architecture unlock the potential of this network to learn underlying data in depth. So it can grasp the inherited dependencies between variables, which is a desired feature for ICS deployed in water plants.
Our approach extends the application of GAN-based anomaly inference methods towards cyber forensics in this critical domain. For methods that solely rely on empirical data, future importance algorithms can serve as a medium for attack attribution. It’s a feature we call [inaudible] ICS asset.
Future importance refers to a range of techniques that run the input variables of the prediction model indicate the related significance of each feature for prediction. There are numerous ways driving these ranks and many algorithms sustain this purpose and notable examples include, but are not limited to, decision trees, statistical correlations and game-theoretical models.
Another method that appears in literature is to analyze reconstruction error and its change ratio and declare the assets with the highest ratio as assets under attack. However, given the stochastic nature of such techniques, obtained estimates are subject to some degree of uncertainty. To this end, we perceived that the fusion of independent outcomes produced by each method would likely render better normal contribution and concur the ambiguity of such attribution.
To summarize, we frame our contribution as follows. We complement existing method support in cyber forensics for water treatment plants, and to this end we introduce an unsupervised approach that infers maliciously-modified processes and pinpoints attack sensors; we leverage bidirectional GAN and recurrent and convolutional neural networks to fingerprint the behavior of ICS deployed in water plants, and to isolate anomalies which represent and attribute attack by pinpointing the exploited ICS assets, which is an essential function towards forensics prioritization.
And we evaluated the approach on empirical data collected by a small-scale water treatment plant. And the empirical evaluation demonstrated equivalent or better performance in comparison with state-of-the-art, and we also showed that the proposed approach can identify exploited ICS assets with a nearly 15% accuracy improvement over prior literature contributions.
There are three core components of our approach: It’s behavior fingerprinting, attack inference, and attack attribution. The first component learns to fingerprint the typical behavior of the system. And for that, we leverage a bidirectional, GAN-based learning approach, which we will discuss further.
The second component estimates anomaly score by comparing incoming data sequences for each sensor or actuator with a fingerprinted system behavior. And this score indicates any possible attack. The attribution function evaluates the contribution of each asset to the anomaly score and extracts the affected ICS asset to prioritize cyber forensics.
To fingerprint the system behavior on the normal operation, we employ a GAN-based model. And in classical formulation a GAN is a generative and discriminative deep-learning architecture that consists of two competing neural networks, known as generator and discriminator.
This first step generator received noise to learn its distribution and based on perceived distribution, generator produces data samples and passes them to discriminator. In its turn, the network discriminator data mines the distribution between real and fake data and it propagates the probability of data authenticity to generator.
Further, generator or the adapts its parameter based on received gradient information and passes new samples to discriminator. And they work together back and forth to meet their learning objectives. Our use for generator is to produce more realistic instances and for discriminator is to improve ability to distinguish fake data from the real.
Our architecture is different from classical formulation, and it was inspired by the model which was widely and successfully used for detection-anomalous images. And this architecture consists of an additional network known as encoder, and encoder maps incoming data sequences to attend variable space Z. And this architecture should learn to invert the generator G, and this this architecture actually proved under our investigation that it produced better results in comparison to classical formulation.
For attack inference we use generator and discriminator. First, network generator because network generator was trying to capture the complexity of the data distribution and to generate sensor measurements at a certain point of time, so we can take the attentive representation of incoming data obtained from encoder and reconstruct the expected ICS behavior by using generator.
The trained model then evaluates L1 norm distance, namely residual loss between actual and expected physical measurements for all ICS assets. We further utilized the ability of trained discriminator to distinguish the normal operational behavior of ICS, and D discriminator determines the distribution between fingerprinting behavior and latent representation of incoming data obtained from encoder. And it returned on the anomaly score, namely the discriminator loss. And we further obtained the regularity score for each time point which dissipated some or both of these losses.
It’s mentioned for data-driven, my thoughts on the attack attribution can be made with future selection algorithms. However, independent results can be misinterpreted. We perceived that fusion-independent outcomes produced by each method would likely render better anomaly attribution and concur the ambiguity of such attribution.
And for that, we use the notion of degree of belief that selected methods correctly identify the assets under attack. And this degree is computed based on the amount of related support representing a distance between predictions made by each combined method. And we declare an ICS asset is under attack if its score is in the 75th percentile.
For our approach, we selected three methods: Classification and Regression Trees; Shapley values, which is a game-theoretical approach; and KernelSHAP which is a model agnostic modified for deriving related future importance.
Now, our approach has several benefits: it works with multivariate data and can explore inherited dependencies between ICS assets, which is a desired feature for water plants and this leads to better anomaly inference. It’s also learns behavior of the underlying system, so it eliminates the need of mathematical models. It is trained on attack-free data, so it manifest its promising application for cyber forensics of ICS where attack-related data is rarely available due to security and privacy reasons. And it’s also pinpoints attacked ICS assets and contributes towards cybersecurity and forensics prioritization objectives.
To evaluate performance of our method, we use data that’s collected by a small-scale water treatment plant, similar to one that can be found in smart cities. The data consists of seven days of attack-free data and four days under different types of attacks. Attacks can vary in time, attack sensors, and attack process levels.
We seek an answer for the following question, for evaluation strategy, our evaluation strategies as follows. We seek our answers for several questions, and first of them, how accurately does the approach fingerprint the behavior of the underlying system by solely relying on empirical data?
And to date, we evaluate how firmly our model does so, we employ discrepancy measures known as energy distance, denoted as ED and maximum mean discrepancy, denoted as MMD to the actual and our reconstructed sequences. All of the ED and MMD measures indicate the lower discrepancies between two probabilities and suggest optimal stabilization of the high-quality of behavior fingerprinting.
There are two other questions: how can the proposed approach correctly infer attacks? How well it does so and how well it distinguishes attack sensors. And for that we use usual metrics: sensitivity, precision, and F1-score.
As a prerequisite for anomaly detection, the proposed approach should carefully model system behavior under regular operation. This finger shows the largest decrease in both metrics occurred in both metrics, which is maximum mean discrepancy in energy distance, occured after 30 seconds, and it is reasonable to suggest that the quality of generated data is stabilized after this epoch.
And we also employed t-SNE analysis and observed significant overlap between the original and perceived data, and the blue color indicates original data and orange color indicates perceived data. And this significant overlap indeed indicates that proposed model was able to grasp the system behavior quite closely. The proposed approach also demonstrates equivalent performance in state-of-the-art while asset distribution is significantly improved.
Now I’ll give you a couple of examples of inferred attack and pinpointed assets. The attack window here in this chart is denoted with a red color and the inference mechanism precisely isolated the incident and is denoted with this blue bar. However, irregularity score is higher after the actual attack which we call the recovery after attack and data set marks this recovery time is irregular operation.
However, it is essential to infer the stabilization period for further investigation of potential impact on the system signs, since it may take longer than a long time, sometimes hours for ICS to return to regular operation. The proposed methods correctly attribute attack to the respective assets, and it outperforms the previous proposed methods. In this table, both indicate the correctly identified ICS asset.
In the next scenario, the proposed approach also identified this ICS attack and irregularity score and correctly attributed the affected sensor while it raised a false alarm. Despite this, our methods outperform the previous contribution anyway.
To conclude, I’m aiming to support cyber forensics in the context of ICS deployed in smart water facilities. We compliment available contributions and introduce a supervised approach that infers cyber attacks and reveals the attacked assets for forensics prioritization.
The approach has three distinguishing characteristics: it operates with multivariate data, which is highly desired in ICS deployed in water treatment plants; it is trained using only attack-free instances, therefore avoiding the problem of from non-available attack-related data; and it identifies the attack points and thus can reduce forensics overhead in critical infrastructure.
For the future perspective, we plan to extend evaluation using different variants of ICS deployed in our real-world water facilities and investigate the effect of pattern instability, which can be found in water distribution system as variation of water consumption by consumer, and we also plan to investigate additional information fusion approaches to calibrate accuracy of asset distribution. And so finally, we would like to explore remediation methods, given the knowledge about the exact attack assets from this work.
Thank you for your interest in our work and for your time.
