Name: Understand adversarial attacks by doing one yourself with this tool
Brand: thenextweb.com
SKU: thenextweb-com-63
Availability: OnlineOnly

critic

Understand adversarial attacks by doing one yourself with this tool

preview

This article was published on November 27, 2020In recent years, the media have been paying increasing attention to, input data such as images and audio that have been modified to manipulate the behavior of. Stickers pasted on stop signs that cause computer vision systems to mistake them for speed limits; glasses that fool facial recognition systems, turtles that get classified as rifles — these are just some of the many adversarial examples that have made the headlines in the past few years.There’s increasing concern about the, especially as machine learning systems continue to become an important component of many applications we use. AI researchers and security experts are engaging in various efforts to educate the public about adversarial attacks and create.Among these efforts is, an interactive tool that shows how adversarial attacks work. Released on GitHub last week, adversarial.js was developed by Kenny Song, a graduate student at the University of Tokyo doing research on the security of machine learning systems. Song hopes to demystify adversarial attacks and raise awareness about machine learning security through the project.Subscribe to our newsletter now for a weekly recap of our favorite AI stories in your inbox.Song has designed adversarial.js with simplicity in mind. It is written in Tensorflow.js, the JavaScript version of Google’s famous deep learning framework.“I wanted to make a lightweight demo that can run on a static webpage. Since everything is loaded as JavaScript on the page, it’s really easy for users to inspect the code and tinker around directly in the browser,” Song toldTechTalks.[Read:&nbsp;]Song has also launched athat hosts adversarial.js. To craft your own adversarial attack, you choose a target deep learning model and a sample image. You can run the image through theto see how it classifies it before applying the adversarial modifications.As adversarial.js displays, a well-trained machine learning model can predict the correct label of an image with very high accuracy.The next phase is to create the adversarial example. The goal here is to modify the image in a way that it doesn’t change to a human observer but causes the targeted machine learning model to change its output.After you choose a target label and an attack technique and click “Generate,” adversarial.js creates a new version of the image that is slightly modified. Depending on the technique you use, the modifications might be more or less visible to the naked eye. But to the target, the difference is tremendous.In our case, we are trying to fool the neural network into thinking our stop sign is a 120km/hr speed limit sign. Hypothetically, this would mean that a human driver would still stop when seeing the sign, but a self-driving car that uses neural networks to make sense of the world would dangerously speed past it.Adversarial attacks are not an exact science, and this is one of the things that adversarial.js displays very well. If you tinker with the tool a bit, you’ll see that in many cases, the adversarial techniques do not work consistently. In some cases, the perturbation does not result in the machine learning model changing its output to the desired class but instead causes it to lower its confidence in the main label.“I got interested in adversarial examples because they break our fantasy that neural networks have human-level perceptual intelligence,” Song says. “Beyond the immediate problems it brings, I think understanding this failure mode can help us develop more intelligent systems in the future.”Today, you can runrunning on your computer, phone, home security camera, smart fridge, smart speaker, and many other devices.Adversarial vulnerabilities make these machine learning systems. But they can also create security risks that we have yet to understand and deal with.“Today, there’s a good analogy to the early days of the internet,” Song says. “In the early days, people just focused on building cool applications with new technology, and assumed everyone else had good intentions. We’re in that phase of machine learning now.”Song warns that bad actors will find ways to take advantage of vulnerable machine learning systems that were designed for a “best-case, I.I.D. [independent and identically distributed], non-adversarial world.” And few people understand the risk landscape, partially due to the knowledge being locked in research literature. In fact, adversarial machine learning has been discussed among AI scientists since the early 2000s and there are already thousands of papers and technical articles on the topic.“Hopefully, this project can get people thinking about these risks, and motivate investing resources to address them,” Song says.“Adversarial examples are just one problem,” Song says, adding that there are more attack vectors like,, data reconstruction, or model extraction.There’s growing momentum across different sectors to create tools for improving the security of machine learning systems. In October, researchers from 13 organizations, including Microsoft, IBM, Nvidia, MITRE, released the, a framework to help developers and adopters of machine learning technology to identify possible vulnerabilities in their AI systems and patch them before malicious actors exploit them. IBM’s research unit is separately involved increating AI systems that are robust to adversarial perturbations. And the National Institute of Standards and Technology, the research arm of the U.S. Department of Commerce, has launched, an initiative to combat trojan attacks on machine learning systems.And independent researchers such as Song will continue to make their contributions to this evolving field. “I think there’s an important role for next-gen cybersecurity companies to define best practices in this space,” Song says.Greetings Humanoids! Did you know we have a newsletter all about AI? You can subscribe to it right here.

This article was published on November 27, 2020In recent years, the media have been paying increasing attention to, input data such as images and audio that have been modified to manipulate the behavior of. Stickers pasted on stop signs that cause computer vision systems to mistake them for speed limits; glasses that fool facial recognition systems, turtles that get classified as rifles — these are just some of the many adversarial examples that have made the headlines in the past few years.There’s increasing concern about the, especially as machine learning systems continue to become an...

public

Ben Dickson

The Next Web (TNW) is a website and annual series of conferences focused on new technology and start-up companies in Europe. The Next Web company was established in 2006 by co-founders Boris Veldhuijzen van Zanten and Patrick de Laive in Amsterdam, Netherlands, and a technology news website of the same name was started in 2009. On 5 March 2019, the Financial Times purchased a majority stake in TNW.

Understand adversarial attacks by doing one yourself with this tool

No article scores yet.

CRITIC REVIEWS

PUBLIC REVIEWS