Version 5 (modified by 3 weeks ago) ( diff ) | ,
---|
Privacy Leakage Study and Protection for Virtual Reality Devices
Advisor: Dr. Yingying (Jennifer) Chen
Mentors: Changming LiGR, Honglu LiGR, Tianfang ZhangGR
Team: Dirk Catpo RiscoGR, Brandon (Jinu) SonUG, BrodyHS, Emily YaoHS
Project Overview
Augmented reality/virtual reality (AR/VR) is used for many applications and have been used for many purposes ranging from communicating and tourism, all the way to healthcare. Accessing the built-in motion sensors does not require user permissions, as most VR applications need to access this information in order to function. However, this introduces the possibility of privacy vulnerabilities: zero-permission motion sensors can be used in order to infer live speech, which is a problem when that speech may include sensitive information.
Project Goal
The purpose of this project is to extract motion data from AR/VR devices inertial measurement unit (IMU), and then input this data to a large language model (LLM) to predict what the user is doing
Weekly Updates
Week 1
Progress
- Read research paper [1] regarding an eavesdropping attack called Face-Mic
Next Week Goals
- We plan to meet with our mentors and get more information on the duties and expectations of our project
Week 2
Progress
- Read research paper [2] regarding LLMs comprehending the physical world
- Build a connection between research paper and also privacy concerns of AR/VR devices
Next Week Goals
- Get familiar with AR/VR device:
- Meta Quest
- How to use device
- Configure settings on host computer
- Extract motion data from IMU
- Connecting motion sensor application program interface (API) to access data
- Data processing method
Week 3
Progress
- Placeholder
Next Week Goals
- Placeholder
Week 4
Progress
- Placeholder
Next Week Goals
- Placeholder
Week 5
Progress
- Placeholder
Next Week Goals
- Placeholder
Week 6
Progress
- Placeholder
Next Week Goals
- Placeholder
Week 7
Progress
- Placeholder
Next Week Goals
- Placeholder
Links to Presentations
Week 1 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8 Final Presentation