wiki:Other/Summer/2023/SelfDriving

Version 34 (modified by tochu, 16 months ago) ( diff )

—

Self Driving Vehicular Project

    Self Driving Vehicular Project

    WINLAB Summer Internship 2023

    Group Members: Adarsh Narayanan, Brandon Cheng, Christopher Sawicki, Joshua Menezes, Ruben Alias, Tommy Chu, Aleicia Zhu, Ranvith Adulla, Arya Chhabra, Suhani Sengupta

    Project Objective

    This project has several objectives involving the WINLAB miniature intersection.

    RASCAL
    (Robotic Autonomous Scale Car for Adaptive Learning)     		SCAMP
    (Self-guided Computer Assisted Mecanum Pathfinder)
    Continue work on the miniature autonomous car hardware project developed last summer (see the project page here: ​https://www.orbit-lab.org/wiki/Other/Summer/2022/Hardware).

    The goal for this summer is to finish prototyping the hardware and develop drivers to control the car with ROS.
    If time allows, once the ROS integration is complete, we will be able to develop and test machine learning models to control the car.     		Develop a remote-controllable car which can be used to mimic the path taken by a car in the real intersection.

    For this platform, mecanum wheels will be used to allow for real-time corrections to the path.
    The platform will be developed using Robot Operating System (ROS).
    Students will need to prototype hardware and develop software to extract the path from video of the real intersection, translate that into a path for the miniature intersection, and produce a set of commands for the car.

    *RASCAL pseudonyms: UpCar, BARTHOLOMEW

    *SCAMP pseudonyms: MIMIC, CORNELIUS

    Week 1

    ​Week 1 Slides

    RASCAL MIMIC
    Worked on getting RASCAL driving with a controller
    Was able to parallel park     		Started designing MIMIC hardware setup

    Week 2

    ​Week 2 Slides:

    RASCAL MIMIC
    Switched from PyCharm to Vim due to storage issues
    Started implementing ROS     		Started wiring with IMU, Jetson Nano, Teensy, four brushless
    motors, and two motor drivers

    Week 3

    ​Week 3 Slides:

    RASCAL MIMIC
    Started setting up Rosbags to collect data to
    eventually train an ML model     		Set up ROS and serial control

    Week 4

    ​Week 4 Slides:

    RASCAL MIMIC
    Created Flutter app to clean data from camera
    Set up scripts for converting .bag files to .csv     		Set up controls with PS3 controller to prevent motor stall
    Created web server to track and simulate car movement

    Week 5

    ​Week 5 Slides:

    RASCAL MIMIC
    Worked on creating imitation learning model to drive autonomously Created an electrical schematic
    Started working on PID for motor control

    Week 6

    ​Week 6 Slides:

    RASCAL MIMIC
    Started designing and testing for hardware issues with v2 (BARTHOLOMEWTWO)
    Set up IMU for better odometry and position tracking     		Moves precise distances and angles for path following

    Week 7

    ​Week 7 Slides

    RASCAL MIMIC
    Finished designing BARTHOLOMEWTWO and successfully ran the imitation learning model Able to follow a path drawn on the web server

    Week 8

    ​Week 8 Slides

    RASCAL MIMIC
    Tested pure pursuit and normalized ML inputs Implement full omni-drive motion for path following

    Week 9

    [ Week 9 Slides]

    Week 10

    [ Week 10 Slides]

    Project Setup Instructions

    Set up SSH key for Gitlab

    • If you clone the repository with HTTPS, then you don't need to do this. However, then you will need to input your username and password every time you want to fetch or push from gitlab. To get around this, you can generate an SSH key pair and upload the public key to gitlab.
    • navigate to your .ssh directory by doing "cd ~/.ssh"
    • run the command "ssh-keygen".
    • It will prompt you to name the file. For example, "gitlab".
    • It will then prompt for a passphrase for the key. Leave it empty if you don't want to put in a password every time you use it, or put one for security if you prefer.
    • Two files called "gitlab" and "gitlab.pub" will be created in the ~/.ssh directory now. We need to upload the public key to gitlab.
    • Navigate to your .ssh directory and use the command "cat gitlab.pub" to print the contents of the file into the terminal. Copy all of the file, including the "sha-rsa" at the beginning and the username at the end.
    • Login to gitlab (the winlab specific gitlab! ​https://gitlab.orbit-lab.org) and click on your profile picture on the top right.
    • Go to SSH keys and add the public key in the text box
    • Now go to ~/.ssh and add a file called "config" by doing "touch config". Edit this file (with nano or vim) to look like this:

    Host gitlab.orbit-lab.org

    PreferredAuthentications publickey IdentityFile ~/.ssh/gitlab

    Project instructions:

    • Clone the desired repository. You can click "clone" on the repository page for the desired repository on Gitlab (​https://gitlab.orbit-lab.org/self-driving-car-2023). For example, "git clone git@…:self-driving-car-2023/upcar.git"
    • Make sure you set up ROS in your .bashrc file. You need to have "source /opt/ros/noetic/setup.bash" in it
    • You need to add the setup file for the catkin workspace of the project in your .bashrc file. To do this, first take note of the absolute path to the project directory. You will need to add "source path_to_catkin_ws/catkin_ws/devel/setup.bash" (replace path_to_catkin_ws with the actual absolute path on your system.)
    • One you are done changing the setup files, logout and login.
    • Install the joy package by following this tutorial: ​http://wiki.ros.org/joy/Tutorials/ConfiguringALinuxJoystick
      • The user needs to have input permission for ROS to read data from joystick inputs. To set this, do "sudo usermod -aG input username". -aG specifies adding an additional group (input) to a user (username)
        • Changing groups requires the user to log out and in for effects to take place
    • Install the realsense package (Add Link!)

    ROS setup instructions:

    • Follow these instructions to set up ROS Noetic: ​http://wiki.ros.org/noetic/Installation/Ubuntu
      • Note: any time you edit a startup file, like .bashrc, you need to logout and login (on VM you might need to power it off and on) for the effects to take place.
      • ROS Noetic is used for Ubuntu 20.04 (on RASCAL's Intel UpBoard), ROS Melodic is used for Ubuntu 18.04 (on SCAMP's and new RASCAL's Jetson Nano)

    Using ROS:

    • To run a ROS node: use "rosrun package nodename". For example, "rosrun joy joy_node" to run the joystick package.
      • roscore must be running first by typing "roscore" in the terminal.
      • To run multiple processes at once, you can use Tmux (terminal multiplexer). More advanced uses can be found online, but the basic usage is as follows:
    • To run a ROS launch file, use "roslaunch package launchfile.launch". For example, "roslaunch rascal sim.launch".
    • on the first launch, or any time you change build files (CMakeLists, package.xml) you will need to navigate to the catkin workspace directory and run "catkin_make".
    • To create a new package: ​http://wiki.ros.org/ROS/Tutorials/CreatingPackage
    • To create a new python file that can be run through rosrun: Create the file somewhere within the package's src directory. Make sure the shebang is in the beginning ("#!/usr/bin/env python"). Then add the file to the package CMakeLists.txt in catkin_install_python.
    • To be able to import python modules from within the same package: make sure catkin_python_setup() is uncommented in the package's CMakeLists.txt. Then reference the directory structure visible in the rascal package. Make sure to include the "setup.py" file in the package's directory.

    VM setup instructions:

    To run the simulation on a VM:

    • Switch to the simulation branch by using "git checkout simulation" (in the upcar repository)
    • Joystick needs to be configured in virtualbox. If you are using mac, the VirtualBox must be run as sudo.
    • The joystick needs to be set. navigate to /dev/input. if you do "ls -a" in here, you should see js0 and js1 which are added by virtualbox by default, and then js2 if your controller is configured correctly. Run "sudo jstest /dev/input/js2" to test if it is working.
    • You need to open a port to be able to view the web display on the host machine of the VM.
      • Open virtualbox and go to the settings for the VM.
      • Go to network → Adapter 1 → make sure it is on NAT
      • click advanced → port forwarding → add a new port forwarding rule
      • Fill the following fields (you may need to scroll right): name: flask, Host port: 5001, Guest port: 5000
      • note: you can use any unused port instead of 5001. 5000 is used for the guest port as it is the default for flask.
      • you may need to restart the VM.
    • run simlaunch.sh
    • To view the website, go to 127.0.0.1:5001 on your web browser

    Other Notes:

    • For cars built on the Jetson Nano (ie SCAMP and new RASCAL), the use of ROS Melodic requires that any python files need to be marked as executable. To do this, run "sudo chmod +x path/to/file.py". This adds execute permission (+x) to the file.

    Instructions for MIMIC (aka SCAMP)

    Clone repository into /opt

    Teensy 4.0 Setup:
    1. copy the links for the downloadable tar files for arduino-cli and teensy_loader_cli

    → ​https://github.com/arduino/arduino-cli/releases

    → ​https://github.com/PaulStoffregen/teensy_loader_cli/releases

    1. Use wget to download the tar files and extract them with tar -xvf {filename}
    • Follow the instructions on the teensy_loader_cli github (Run sudo apt-get install libusb-dev for Ubuntu, and make to compile to an executable)
    1. Move the two executable files to "/bin" to add it to path (makes it so that the executable can be run from any directory)

    → This allows the ./teensy-tool.sh file in mimic/teensy to compile and upload to the teensy

    1. Refer to ​https://arduino.github.io/arduino-cli/0.33/getting-started/#adding-3rd-party-cores to add Teensy support to arduino-cli

    → You may have to create a file called arduino-cli.yaml in ~/.arduino15 with information about additional urls, format is provided in the link above

    → ​https://www.pjrc.com/teensy/td_download.html

    → ​https://www.pjrc.com/teensy/package_teensy_index.json

    1. Follow the instructions to create the 00-teensy.rules file:

    → ​https://www.pjrc.com/teensy/00-teensy.rules

    1. Use arduino-cli board list command to find the port of the Teensy

    → Modify the sketch.yaml file in the ./teensy directory to makes sure default_port and default_fqbn matches what is listed from arduino-cli board list

    1. Upload teensy code by running ./teensy-tool -bu in mimic/teensy/ to build and upload the code

    → additionally, you can add the -m flag to open the monitor as well

    ROS Melodic Setup:

    Uses ROS Melodic for Ubuntu 18.04 on the Jetson Nano

    1. Refer to ​http://wiki.ros.org/melodic/Installation/Ubuntu for install
    1. Add the following lines to ~/.bashrc
    • source/opt/ros/melodic/setup.bash
    • source/opt/mimic/devel/setup.bash
    1. cd into mimic repo and run catkin_make to create ROS environment
    1. Run ROS nodes by running ./roslaunch.sh in mimic/

    → check ports for teensy for serial communication → check joystick connection

    1. Making new nodes/msg

    → List out .py files and .msg files as needed in the CMakeLists.txt file located in the package folder (ie test_pkg) → Run catkin_make to refresh environment messages/nodes

    Integration with Web Display:
    1. Use ip a to find the ip of the Jetson Nano
    1. Open a web brower and go to {ip}:5000 to view
    Troubleshooting
    1. "SerialException: Device reports readiness to read but returned no data"

    ROS will not be able to communicate with serial if there is another service using the port, i.e. Serial Monitor

    • Use lsof to check with programs are using the port lsof /dev/ttyACM0
    • Kill the program that is using the port kill {PID}
    1. "socket.error: [Errno 48] Address already in use"

    Flask server was not properly stopped before starting it again

    • Close the connection sudo fuser -k 5000/tcp
    Note: See TracWiki for help on using the wiki.