Smart Intersection

    Smart Intersection - daily traffic flow

    WINLAB Summer Internship 2020

    Group Members: Bryan Zhu, Kevin Zhang, Nicholas Meegan

    Project Website

    Gitlab Repositories

    DeepStream and YOLOv3 Application:

    OpenCV- Add Bounding Boxes to Video:

    Project Objective

    The goal of this project is to create a method for estimating the statistics for vehicle count/traffic flow into one intersection in New York City. As an example, record videos of the northbound traffic on Amsterdam Avenue, as vehicles are entering the 120th St./Amsterdam Av. intersection. Using YOLOv3 deep learning model, detect and count vehicles as they approach/enter the intersection from south, making sure that there is no double-counting. Use 180 second long video fragments (approximately two traffic light cycles), and repeat up to half a dozen times a day, for a number of workweek/weekend days during the same times of each day. Compare the vehicle count (traffic flow) as a function of the time of the day. Utilize NVIDIA DeepStream deployed on COSMOS GPU compute servers to run the model. The method should be generalizable/expandable to any direction of vehicle movement, when appropriate camera views are available.

    Week 1 Activities

    • Get ORBIT/COSMOS account and familiarize oneself with the testbed procedures
    • Learn about YOLOv3 deep learning models for object detection
    • Read about NVIDIA DeepStream
    • Explore the image (set of computing tools) available on COSMOS, which uses DeepStream and can deploy YOLOv3
    • Brainstorm about vehicle counting/traffic flow estimation methodology

    Week 1 Weekly Meeting Presentation:

    Week 2 Activities

    • Understand the concepts of object detection in 3D Point Cloud
    • Gain an understanding of NVIDIA’s DeepStream SDK
    • Get comfortable deploying YOLOv3 on the COSMOS testbed
    • Use existing datasets to play around with DeepStream and YOLOv3

    Week 2 Weekly Meeting Presentation:

    Week 2 Team Meeting Presentation:

    Week 3 Activities

    • Investigate existing RGB-D (RGB + depth map) object detectors whose models we can immediately put to use for inference
    • Look into existing 3D Point Cloud object detection implementations
    • Learn how to run DeepStream's YOLOv3 implementation
    • Investigate DeepStream Python bindings for use with YOLO

    Week 3 Weekly Meeting Presentation:

    Week 3 Team Meeting Presentation:

    Week 4 Activities

    • Investigate YOLOv4 and its use with TensorRT
    • Look into getting output/data processing based on the outputs from DeepStream
    • Look into the DeepStream tracker to build on top of
    • Build a presentation slide set to inform the intern class about DeepStream and YOLOv3

    DeepStream and YOLOv3 Overview + Demonstration Presentation Slides:

    Video recording DeepStream and YOLOv3 Overview + Demonstration Presentation:

    Week 4 Weekly Meeting Presentation:

    Week 4 Team Meeting Presentation:

    Week 5 Activities

    • Keep trying to get YOLOv4 running as a DeepStream app
    • Augment DeepStream YOLO inference output with bounding box class confidence scores
    • Begin setup of a pub/sub system for inference output
    • Investigate ways of recombining inference output with input video stream (NTP/OpenCV)

    Week 5 Weekly Meeting Presentation:

    Week 5 Team Meeting Presentation:

    Week 6 Activities

    • Implement a publisher within the DeepStream app using high-level C bindings for ZeroMQ provided by CZMQ
    • Attempt to run the DeepStream app on a live video stream
    • Investigate ways to sync video frames and inferred bounding boxes on different machines synced via NTP (Network Time Protocol)
    • Continue working with OpenCV in order to add bounding box information to the input video stream

    Week 6 Weekly Meeting Presentation:

    Week 6 Team Meeting Presentation:

    Week 7 Activities

    • Start the implementation of the subscriber class (Download the ZeroMQ Library (ZMQPP) + Baseline/barebones necessities)
    • Continue developing the publisher class in ZeroMQ
    • Continue working on adding bounding boxes on video stream through the use of multi-threading and synchronization schemes (mutexes, condition variables)

    Week 7 Weekly Meeting Presentation:

    Week 7 Team Meeting Presentation:

    Week 8 Activities

    • Implement any remaining features to the Smart Intersection Project
    • Finish integrating all code together
    • Prepare the final presentation and final poster

    Project Review Presentation:

    Open House Final Presentation:

    Last modified 3 weeks ago Last modified on Jul 24, 2020, 8:02:22 PM
    Note: See TracWiki for help on using the wiki.