= AR Mural =

  **WINLAB Summer Internship 2024**

  **Advisors:** Ivan Seskar, Jennifer Shane

  **Group Members:** Jovana Lekic, Nima Fallah, Nishi Agrawal, Suhani Sengupta, Sachin Kandan

== Project Description

This project aims to create a collaborative augmented reality application that will allow remote users to create a virtual mural collaboratively. This project will be made using the Unity game engine and deployed to AR platforms such as Microsoft Hololens.

In addition to creating the software that will manage the generation and rendering of the mural, another important component of this project is designing an infrastructure that will allow for very low latency collaboration. If time allows, one of the end goals of this project is to deploy it using an "edge cloud" architecture using the OpenAR Cloud project.

Students should get started with this project by familiarizing themselves with the Unity game engine and Hololens development tools.


== Weekly Progress

**Week 1**: Presentation Link: [https://docs.google.com/presentation/d/1e9WKNtZdiVpUDRzJE05nQMO2VyV6-vSEcMwTX7Ittqk/edit?usp=sharing]
* Got acquainted with Unity and HoloLens, and linked the two
* Using the Mixed Reality tool (MRTK) in Unity 
* Viewed last year's codebase

**Week 2**: Presentation Link: [https://docs.google.com/presentation/d/1skRqUdaybDfopb9Gdp5hkInJ3mrngNzjlyoGl3765RU/edit?usp=sharing]
* Worked on brush tool
* Moved codebase to Gitlab
* Found MRTK resources

**Week 3**: Presentation Link: [https://docs.google.com/presentation/d/12aLB9c_lplN2I6Wq8cHUoKUdi36zhQ3g9WWYOEJ5wso/edit?usp=sharing]
* Refined brush tool and added a select tool
* Added voice command for the brush tool
* Added a color palette
[[Image(colorPalette.gif, 640px)]]
[[Image(colorPaletteProof.gif, 640px)]]

**Week 4**: Presentation Link: [https://docs.google.com/presentation/d/1QMFfploRVZXWm-m6f_h3WRY83yJv5-BGWKhmnzGFggw/edit?usp=sharing]
* Started on smoothness, loop, and line buttons
* Added size slider functionality
* Made shape brushes
[[Image(loopTool.gif, 640px)]]
[[Image(lineTool.gif, 640px)]]

**Week 5**: Presentation Link: [https://docs.google.com/presentation/d/1aF3V1rku2ULFikMyMB7wBY5Ym2e4HuJrZPH0dsFyyCA/edit?usp=sharing]
* Finalized sharpness and size features
* Buttons specific to a drawing pop up when in select mode
* Created text tool
* Refactored code
[[Image(textToolWithoutInput.gif, 640px)]]
[[Image(proofOfConceptSelectMenu.gif, 640px)]]
[[Image(sharpnessSizeButtons.gif, 640px)]]
[[Image(sizeToolProofOfConcept.gif​, 640px)]]

**Week 6**: Presentation Link: [https://docs.google.com/presentation/d/1Gm3-1dNqIkTJQwAcrsrprOLf97ekPJRsANrg5AksCvs/edit?usp=sharing]
* Made text tool based on user input
* Refactored/documented existing codebase
* Created a shape tool and added two shapes - rectangular prism and ellipsoid
* Added voice commands for all tools
[[Image(shapeTool.gif, 640px)]]
[[Image(shapeToolRectangle.gif, 640px)]]
[[Image(shapeToolRectangle2.gif, 640px)]]
[[Image(deletingShapes.gif​, 640px)]]
[[Image(movingShapes.gif​, 640px)]]

**Week 7**: Presentation Link: [https://docs.google.com/presentation/d/1kXYXT7qoGsePtHjU1wCP2iUc-QgQiC257jsCKCRiE0g/edit?usp=sharing]
* Redesigned loop button
* Matched the line select buttons’ transparency to all other menus
* Install Holographic Remote App for Player Mode
* Created a custom server implementation on top of TCP protocols
* Worked on adding a tutorial feature and help feature integrated into the main scene

**Week 8**: Presentation Link: [https://docs.google.com/presentation/d/1C8ZR-LPOed49XZkATWWTwg8Brs8bPAiEkxrv13r1Kv8/edit?usp=sharing]
* Added functional muti-select move and delete options
* Fixed highlighting multiple objects
* Skeleton overlay works with webcam and prerecorded videos
[[Image(multiDelete.gif, 640px)]]
[[Image(multiSelect.gif, 640px)]]

**Week 9**: Presentation Link: [https://docs.google.com/presentation/d/115jVq8q7LLpU7HA_pZv0NUuvJcpaw3NaLt2u2qJNiJA/edit?usp=sharing]
* Added a color mixer
* MQTT Server works with add, delete, and move commands
* 3D tool
* Skeleton overlay application connected to the Hololens
* Tested 2-player and 3-player mode using not only the computer Unity Builds but also the Hololens
[[Image(colorMixer.gif, 640px)]]
[[Image(colorMixerProofOfConcept.gif, 640px)]]
[[Image(2PlayerProofOfConcept.gif, 640px)]]
[[Image(3PlayerProofOfConcept.gif, 640px)]]

[[Image(Untitled.png]]