Changes between Initial Version and Version 1 of Other/Summer/2024/pLS

Timestamp:

Jun 10, 2024, 3:00:40 PM (3 weeks ago)

Author:

dirky9000

Comment:

—

Other/Summer/2024/pLS

@@ +1 @@
+= Privacy Leakage Study and Protection for Virtual Reality Devices
+
+** Advisor:** [https://eceweb1.rutgers.edu/~daisylab/people.html# Dr. Yingying (Jennifer) Chen]
+
+**Mentors:** [https://eceweb1.rutgers.edu/~daisylab/people.html# Changming Li]^GR^, [https://eceweb1.rutgers.edu/~daisylab/people.html# Honglu Li]^GR^, and [https://eceweb1.rutgers.edu/~daisylab/people.html# Tianfang Zhang]^GR^
+
+**Team:** [https://www.linkedin.com/in/dirky9000 Dirk Catpo Risco]^GR^, [https://www.linkedin.com/in/genericnameforjinu/Brandon (Jinu) Son]^UG^, Brody^HS^, Emily Yao^HS^
+
+[https://google.com Final Poser With Results]
+
+
+== Project Overview
+Augmented reality/virtual reality (AR/VR) is used for many applications and have been used for many purposes ranging from communicating and tourism, all the way to healthcare. Accessing the built-in motion sensors does not require user permissions, as most VR applications need to access this information in order to function. However, this introduces the possibility of privacy vulnerabilities: zero-permission motion sensors can be used in order to infer live speech, which is a problem when that speech may include sensitive information. 
+
+== Project Goal
+The purpose of this project is to extract motion data from AR/VR devices inertial measurement unit (IMU), and then input this data to a large language model (LLM) to predict what the user is doing
+
+== Weekly Updates
+=== Week 1
+**Progress**
+   * Read research paper ![1] regarding an eavesdropping attack called Face-Mic 
+**Next Week Goals**
+   * We plan to meet with our mentors and get more information on the duties and expectations of our project
+ 
+=== Week 2
+**Progress**
+  * Read research paper ![2] regarding LLMs comprehending the physical world
+  * Build a connection between research paper and also privacy concerns of AR/VR devices
+
+**Next Week Goals**
+  * Get familiar with AR/VR device:
+    * Meta Quest
+    * How to use device
+    * Configure settings on host computer
+  * Extract motion data from IMU
+    * Connecting motion sensor application program interface (API) to access data
+    * Data processing method
+
+=== Week 3
+**Progress**
+  * Placeholder
+
+**Next Week Goals**
+  * Placeholder
+
+=== Week 4
+**Progress**
+  * Placeholder
+
+**Next Week Goals**
+   * Placeholder
+
+=== Week 5
+**Progress**
+  * Placeholder
+
+**Next Week Goals**
+   * Placeholder
+
+=== Week 6
+**Progress**
+  * Placeholder
+
+**Next Week Goals**
+  * Placeholder
+
+=== Week 7
+**Progress**
+  * Placeholder
+**Next Week Goals**
+  * Placeholder
+
+== Links to Presentations
+
+{{{#!html 
+<p style="display: inline-block;">
+<a href="https://google.com">Week 1</a> &nbsp 
+<a href="https://google.com">Week 2</a> &nbsp
+<a href="https://google.com">Week 3</a> &nbsp 
+<a href="https://google.com">Week 4</a> &nbsp
+<a href="https://google.com">Week 5</a> &nbsp
+<a href="https://google.com">Week 6</a> &nbsp
+<a href="https://google.com">Week 7</a> &nbsp
+<a href="https://google.com">Final Presentation</a> &nbsp 
+
+
+</p> 
+}}}
+   
+== References
+
+![1] Shi, C., Xu, X., Zhang, T., Walker, P., Wu, Y., Liu, J., Saxena, N., Chen, Y. and Yu, J., 2021, October. Face-Mic: inferring live speech and speaker identity via subtle facial dynamics captured by AR/VR motion sensors. In Proceedings of the 27th Annual International Conference on Mobile Computing and Networking (pp. 478-490).
+
+![2] Xu, H., Han, L., Yang, Q., Li, M. and Srivastava, M., 2024, February. Penetrative ai: Making llms comprehend the physical world. In Proceedings of the 25th International Workshop on Mobile Computing Systems and Applications (pp. 1-7).