Hardly anyone would love to watch what they designed to be sabotaged or even destroyed by outside influence. Therefore, it should come as no surprise that when technology capable of subverting machine learning models appeared, it started mainly causing frustration among ML developers.
However, as with most other attacks on emerging technology, historical or not, it is becoming evident that we can use adversarial attacks to benefit machine learning. Even more so, they can lead researchers of the entire AI field to a new level of understanding of the mechanisms behind it.
This article will review the five main reasons you should not look at Adversarial ML as an all-consuming evil and even start looking for it while working on a model with substantial real-world implications. Both technical and non-technical aspects of Adversarial ML will be considered part of the article’s scope.
What is Adversarial Machine Learning?
Adversarial Machine Learning aims to create the types of data instances (including text or images) that force the Machine Learning model to malfunction, either by providing a false prediction or causing it to break down. These examples are often designed to go virtually undetected by humans without raising suspicions, thus exploiting the data's numerical representations.
Machine Learning models are usually trained on the same statistical properties, and adversarial examples disrupt their performance by not following these properties.
One of the most famous examples of an adversarial attack is several successful experiments targeting self-driving car recognition models. For example, researchers could completely confuse the traffic sign recognition system persuading it to believe that the stop sign was a speed limit with simple physical manipulations.
Reason 1: Crash-Proof Model Design
In the traditional software design, producing a system that will not crash or perform unpredictably due to user input is vital. When such input exists, this will be a severe threat to the system’s security and sustainability. In a large-scale development, diverse input testing types are performed on a product before it is deemed safe enough for industrial use. The system should either know what to do with the input or not work with it at all.
However, this has mostly evaded the area of machine learning. One of the main culprits for this was an immense amount of out-of-sample possibilities where the model is not expected to perform by default. Such openness of input channels leads to feeding the malicious example with no mechanism to determine the potential usability of input before the model.
The appearance of adversarial technologies, which are by now not very hard to produce, caused the dream of strictly recognized-not recognized models to come crashing down, introducing a new term: “tricked to recognize.” These developments force us to ask why we should treat machine learning input differently from other human-accessed inputs?
The availability of adversarial examples and inbuilding a technology able to neutralize them will further help deal with unexpected out-of-sample inputs or attacks on the system since now it will know how to act.
Reason 2: Understanding the Consequences
With the Artificial Intelligence area growing into an extremely relevant field, opportunities to place a new algorithm in some decision-making processes only grow and become more ambitious. At the same time, the development pipeline is still very often circling the chain of “gather data-train-test-deploy.”
Did you ever know someone who wanted to build self-driving car software by themselves? They get car camera data, make a great model, install it on some prototypes, and already you have an autonomous vehicle you can drive around.
Well, adversarial machine learning shows us why this does not work at scale and in commercial applications. There are many downright terrifying examples of fooling the automatic car behavior with as much as a few carefully selected stickers on a traffic sign or faking a medical diagnosis with a normal-looking image. Adversarial Machine Learning makes us evaluate the decisions we are making, their importance, and an adequate amount of resources to protect them.
Reason 3: Gaining Customer Trust
Remember the last time you used a payment system in any purchasing situation. Did you, at the time, consider it to be safe or not? And even if you had doubts about it, you were more than likely concerned about the human aspect of the transaction.
This is how technology gains the reputation of being trustworthy. We usually have no second thoughts about making such operations because they managed to withstand numerous threats they faced over time. We know that it would take centuries to break into our most secure areas of life, understand why this is the case, and establish a pact of trust.
This sense of security and stability is what the general Artificial Intelligence area has been lacking. The average customer usually does not know nor understand how accurate and secure the AI system is and why it makes such decisions. Showing the system’s stability can help persuade those potential clients for whom security and performance stability are crucial.
Reason 4: Pushing Development of Explainability
Already a part of everyday decision-making, models are widely trusted to act without supervision. Unfortunately, this trust opens a new possibility for unfaithful users: to subvert “only” a trusted black box to enter the desired decision!
This reason is directly connected to the previous one and refers to the adversarial attack’s primary principle: if you have more knowledge about the system’s nature, you have an advantage. Thus, white or grey-box attacks are much more dangerous than black-box ones, as the attackers have to figure out the system’s configuration from scratch.
Motivation to understand the reasoning behind them almost forces us to inspect the model’s decision-making more closely. You can only defend an attack if you know what it aims for, which creates a new path to developing Explainable machine learning. Looking at the model as a white or black box now becomes crucial, and, more importantly, this helps us understand what the model is doing by itself.
Reason 5: Data Science Needs White Hats
For a long time, challenging the security of any digital system was vital to ensuring that it is well-defended from possible attacks. As a result, there already are many specialized algorithms for creating adversarial examples, with some of them even becoming a popular tool, such as the Fast Sign Gradient Method by Ian Goodfellow and others.
With such a great diversity of possible adversarial methods, new counter-methods are developed and continuously needed. This leads to an understanding that the models are vulnerable. However, it can prove decisive to stay a step ahead of the breaching threats in areas with higher security requirements. Therefore exploration of model vulnerabilities and their resolutions can become a new testing step for the model preparation.
Conclusions
Adversarial ML helps us understand why and how the model works by learning how it can be fooled. As a result, we increase the models’ stability in understanding unexpected situations and attacks when obtaining adversarial examples and make models safer. We can also make them more trustworthy and understandable for the customers. And who knows, maybe adversarial security has a future as an enormous field within cybersecurity?
.jpg)
