Self Driving Vehicular Project

Team: Michelle Gutwein and Christopher Lee
Advisors: Ivan Seskar and Jennifer Shane
Project Description & Goals:
Build and train miniature autonomous cars to drive in a miniature city.
Technologies: ROS (Robot Operating System), Pytorch

---

Week 1

Read through the ​manual provided to us.

​Week 1 slides

Week 2

Learned what ROS is and how to use it
Also got to know the car, RASCAL

​Week 2 slides

Week 3

Read through all of the code in the ​Gitlab
Learned how to record and edit data on the web display

​Week 3 slides

Week 4

Learned how to train a model
Trained and tested the model on simple paths

​Week 4 slides

Week 5

Trained model on making right and left turns in the city environment
Started training on turns at the intersection
Created a new model called gps which will allow users to give the car commands on the direction to turn at an intersection before running the car

​Week 5 slides

Week 6

Collected new data on making wider right and left turns in the city environment
Continued training on turns at the intersection
Updated the documentation to be more clear

​Week 6 slides

Connecting to Rascal

Use Zerotier to connect directly:

1. Sign into Zerotier under the account rascalstonesdc-at-gmail.com password: si2022sdc
   
2. Connect to the network: rascalstonesdc's 1st network (ID: d3ecf5726d1a9fcd)
   
3. Now you can open a terminal and connect to the IP address of the rascal using the command: ssh rascal@10.244.114.238
   
4. Next, run the command: roslaunch rascal pure_pursuit.launch to launch all the nodes
   
5. In a web page type 10.244.114.238:5000 to open the web display where you can record and edit data for training
   

You can also use Zerotier to connect directly to the rascal server using the command: ssh rascaltrain@10.244.176.240
If this doesn't work for some reason follow these steps below:

Through ORBIT:

1. Make a reservation on your orbit account under the intersection bed
   
2. Open a new terminal and type ssh <your username>@console.intersection.orbit-lab.org
   
3. You should see the 'welcome to ORBIT-LAB' sign
   
4. Enter ssh rascaltrain@srv1
   

Collecting Data

To record data:

1. Press bagrecorder/enable on the web display to start recording
   
2. Use the joystick or keys on the keyboard (wasd) to move the car around
   
3. Press bagrecorder again to stop recording
   
4. Open data opt/upcar/dataProcessing and type python bag2AllData.py in the terminal to access all recorded sessions
   
5. Select the number that you want to download (should be the most recent one)
   
6. After it finishes downloading you can type the command: roslaunch rascal sim.launch which will upload that session to the web display
   
7. In the web display you can play the video to see what the camera recorded (you might need to refresh the page to see the session come up)
   
8. You can now edit the data that will be used for training
   
9. To save data, run the data_loader/smoothen command

Deleting Bad Data Segments

When portions of the recorded data are messy or irrelevant for training, they can be excluded to improve model performance

1. In the web display you can click replayer/pause-play to run through all of the data segments
   
2. To jump between different segments that the web display automatically divides the data into, click replayer/nextSeg or replayer/prevSeg
3. When you get to a segment you want to delete, pause the run through and hit replayer/editMode
   
4. Select the data you want to delete by using the replayer/jumpEnd and replayer/jumpStart buttons
   
5. Select replayer/editMode again to highlight the part that's going to be deleted
   
6. Click replayer/deleteSelection to delete the segment
   

Training Rascal

We used the fisheye model to train on

1. Open a new terminal and connect to the rascal server's IP address by using this command: ssh rascaltrain@10.244.176.240 (or use ORBIT)
   
2. Open /upcar/dataProcessing and type ./copyFromCar.sh which will get the files from the car and bring them over to the server
   
3. It will ask you to select a host. Choose 1) rascal@172.24.114.238 which is the car's IP address
   
4. If it asks for a password it's always si2022sdc
   
5. It will ask you which sessions you want to copy over (pick the session version of the files you want that says smooth)
   
6. If you go to /sessions and type ls your chosen files should be listed there
   
7. Go back to /upcar and move your data to /rascaltraining/data: mv dataProcessing/sessions/<your file name here> rascaltraining/data
   
8. If you go into /rascaltraining/data you should see your file listed
   
9. Go back to /rascaltraining and type the command: python trainModel.py -m fisheye —use_labels 1.0 -s labels
   
10. Now your model is being trained
    
11. To stop training press enter
    
12. Remember to stop training when the validation loss doesn't seem to be decreasing by a significant amount anymore to avoid overfitting
    
13. To check what number model you just created, go to /savedModels and type ls (yours will be the most recent number)
14. To push your newly made model to gitlab, type git add -f <your model number here>
    
15. Next type the command git commit -m "<give it a name>"
    
16. Next type the command git push origin main
    
17. Finally, back in the car's terminal go to /savedModels and type git pull origin main
    

You're now ready to start testing!

Testing Rascal

To test rascal:

1. Go to /upcar in the car's terminal (not the server)
   
2. Enter ./launch_with_model.sh pure_pursuit
   
3. Open up the web display
   
4. Run these commands in the web display:
   
   2. sim_camera/set_enable (set to true)
      
   3. ml/imgDisp (set to true)
      
   4. ml/0/enable (click enable)
      

Watch the car move on it's own!
Make sure it doesn't crash!