Adversarial Machine Learning Against Voice Assistant Systems

Project Objective

This project aims to study the security of voice assistance systems under adversarial machine learning. The audio adversarial samples generated by adversarial learning algorithms can be played via a loudspeaker and recorded with the microphone of voice assistance systems so as to fool the machine learning models in the system. To make the adversarial samples robust under audio propagation, the room impulse response needs to be estimated and used in the adversarial sample generation process. Specifically, the room impulse response and adversarial attack scenarios can be conducted in digital domain or simulated for the over-the-air scenarios using Python or Matlab.

Tutorials

• Generating Adversarial Samples in Keras: ​https://medium.com/mindboard/generating-adversarial-samples-in-keras-tutorial-f881ac836246
• Tensorflow - Adversarial Example using FGSM: ​https://www.tensorflow.org/tutorials/generative/adversarial_fgsm
• Generating Adversarial Samples in Keras: ​https://medium.com/analytics-vidhya/implementing-adversarial-attacks-and-defenses-in-keras-tensorflow-2-0-cab6120c5715

Reading Material

• ​Hidden voice commands
• ​CommanderSong: A Systematic Approach for Practical Adversarial Voice Recognition
• ​Audio Adversarial Examples Targeted Attacks on Speech-to-Text
• ​Imperceptible, Robust, and Targeted Adversarial Examples for Automatic Speech Recognition
• ​Practical Adversarial Attacks Against Speaker Recognition Systems

Week 1 Activities

• Get ORBIT/COSMOS account and familiarize oneself with the testbed procedures

Week 2 Activities

• Get familiar with Python language.
  — Install Python environment
  — Use Jupyter Notebook to run Python code samples
  
• Learn the concept of deep learning and deep neural networks.
  — Slides: Neural Network Basics of Energy-Efficient Machine Learning System
  — Video tutorial (Optional): Neural Networks and Deep Learning by Andrew Ng (Recommended chapters: Week 2: Logistic Regression as a Neural Network, Week 3: Shallow Neural Network)

Week2 Tutorials

• Python tutorial: ​https://www.w3schools.com/python/
• How to run Python code: ​https://www.knowledgehut.com/blog/programming/run-python-scripts
• Jupyter notebook tutorial: ​https://www.dataquest.io/blog/jupyter-notebook-tutorial/
• Video tutorial (Optional): Neural Networks and Deep Learning: ​https://www.coursera.org/learn/neural-networks-deep-learning

Week 3 Activities

• Setup the TensorFlow environment and run the Python code sample for a basic neural network.
• Read the paper “X-Vectors: Robust DNN Embeddings for Speaker Recognition” (IEEE ICASSP 2018).

Week 4 Activities

• Understand the speaker recognition system (X-Vector) and time-delay neural network.
• Learn MFCC feature and extract the MFCC feature using TensorFlow.

Week 5 Activities

• Study the Python code samples for X-Vector and implement X-Vector.
• Learn how to use X-Vector and feed the extracted MFCC features into X-Vector.

Week 6 Activities

• Read the paper “Practical Adversarial Attacks Against Speaker Recognition Systems” (HotMobile 2020).
• Understand the untargeted and targeted attacks against speaker recognition systems.

Week 7 Activities

• Understand the Fast Gradient Sign Method (FGSM) for the untargeted attack.
• Study the code samples for Practical Adversarial Attacks Against Speaker Recognition Systems.

Week 8 Activities

• Develop an untargeted attack that can generate adversarial samples based on the sample code and tutorial.
• Evaluate the performance of the adversarial samples on the voice assistant system (X-Vector).

Week 9 Activities

• Debug and fine-tune the untargeted adversarial machine learning algorithm to achieve better performance.
• Develop a targeted attack that can spoof the X-Vector and misclassify the input audio signals as targeted speakers.

Week 10 Activities

• Debug and fine-tune the developed targeted attack method.
• If time allows, simulate the room impulse response (RIR) and integrate it into the developed attack methods.

Week 11 Activities

• Fine-tune the developed targeted and untargeted attack methods.
• Summarize and prepare for the open house presentation.