Difference between revisions of "Offense - Adversarial Threats/Attacks"
(→Boolean Satisfiability (SAT) Problem/Satisfiability Modulo Theories (SMT): Z3 and Reluplex Solvers) |
(→Boolean Satisfiability (SAT) Problem/Satisfiability Modulo Theories (SMT) Solvers) |
||
| Line 43: | Line 43: | ||
* [http://ijcai13.org/files/tutorial_slides/tb1.pdf SAT in AI: high performance search methods with applications] | * [http://ijcai13.org/files/tutorial_slides/tb1.pdf SAT in AI: high performance search methods with applications] | ||
* [http://stanford.edu/~guyk/pub/CAV2017_R.pdf Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networks] | * [http://stanford.edu/~guyk/pub/CAV2017_R.pdf Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networks] | ||
| + | |||
| + | |||
| + | In what seems to be an endless back-and-forth between new adversarial attacks and new defenses against those attacks, we would like a means of formally verifying the robustness of machine learning algorithms to adversarial attacks. In the privacy domain, there is the idea of a differential privacy budget, which quantifies privacy over all possible attacks. In the following three papers, we see attempts at deriving an equivalent benchmark for security, one that will allow the evaluation of defenses against all possible attacks instead of just a specific one. [http://secml.github.io/class6/ Class 6: Measuring Robustness of ML Models] | ||
| + | |||
| + | * Nicholas Carlini, Guy Katz, Clark Barrett, David L. Dill. [http://arxiv.org/pdf/1709.10207.pdf Provably Minimally-Distorted Adversarial Examples] 20 Feb 2018 | ||
| + | |||
| + | * Guy Katz, Clark Barrett, David Dill, Kyle Julian, Mykel Kochenderfer. [http://arxiv.org/pdf/1702.01135.pdf Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networks] 19 May 2017 | ||
| + | |||
| + | * Tsui-Wei Weng, Huan Zhang, Pin-Yu Chen, Jinfeng Yi, Dong Su, Yupeng Gao, Cho-Jui Hsieh, Luca Daniel. [http://arxiv.org/pdf/1801.10578.pdf Evaluating the Robustness of Neural Networks: An Extreme Value Theory Approach] 31 Jan 2018 | ||
<youtube>DX3G4IoTNF0</youtube> | <youtube>DX3G4IoTNF0</youtube> | ||
Revision as of 16:21, 5 July 2018
______________________________________________________
- Cleverhans - library for benchmarking the vulnerability of machine learning models to adversarial examples blog
- Adversarial Machine Learning for Anti-Malware Software | nababora @ GitHub
- Deep-pwning/Metasploit | Clarence Chio
- EvadeML.org | University of Virginia
- AdversariaLib: An Open-source Library for the Security Evaluation of Machine Learning Algorithms Under Attack
- Pattern Recognition and Applications Lab (PRA Lab)
Adversarial examples are inputs to machine learning models that an attacker has intentionally designed to cause the model to make a mistake; they’re like optical illusions for machines. Myth: An attacker must have access to the model to generate adversarial examples. Fact: Adversarial examples generalize across models trained to perform the same task, even if those models have different architectures and were trained on a different training set. This means an attacker can train their own model, generate adversarial examples against it, and then deploy those adversarial examples against a model they do not have access to. -Deep Learning Adversarial Examples – Clarifying Misconceptions | Goodfellow et al.
Weaponizing Machine Learning
Boolean Satisfiability (SAT) Problem/Satisfiability Modulo Theories (SMT) Solvers
- Rise4Fun - automata concurrency design encoders infrastructure languages security synthesis testing verification language
- SAT in AI: high performance search methods with applications
- Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networks
In what seems to be an endless back-and-forth between new adversarial attacks and new defenses against those attacks, we would like a means of formally verifying the robustness of machine learning algorithms to adversarial attacks. In the privacy domain, there is the idea of a differential privacy budget, which quantifies privacy over all possible attacks. In the following three papers, we see attempts at deriving an equivalent benchmark for security, one that will allow the evaluation of defenses against all possible attacks instead of just a specific one. Class 6: Measuring Robustness of ML Models
- Nicholas Carlini, Guy Katz, Clark Barrett, David L. Dill. Provably Minimally-Distorted Adversarial Examples 20 Feb 2018
- Guy Katz, Clark Barrett, David Dill, Kyle Julian, Mykel Kochenderfer. Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networks 19 May 2017
- Tsui-Wei Weng, Huan Zhang, Pin-Yu Chen, Jinfeng Yi, Dong Su, Yupeng Gao, Cho-Jui Hsieh, Luca Daniel. Evaluating the Robustness of Neural Networks: An Extreme Value Theory Approach 31 Jan 2018
