Real-Time Cyber Physical Systems Application on MobilityFirst

Github Repo

​https://github.com/MrHohn/opencv-CPS

Introduction

Most Cyber Physical Systems (​http://en.wikipedia.org/wiki/Cyber-physical_system) are characterized by stringent network requirements both in terms of latency (i.e. less than 100 msec response time) and scalability (i.e. Trillion-order scalability for CPS devices/objects ).

MobilityFirst, thorugh its name based Virtual Network Architecture used to provide Application Specific Routing to services and applications, can be exploited in the attempt of assisting these applications to meet their requirements.

The goal of the project is to implement a CPS Application based on computer vision (i.e. object recognition) that will be used in conjunction with MobilityFirst’s Virtual Network to showcase the benefits of the network service.

Preliminary Goal

Outline of the Project

Tasks

• Part A: Cyber Physical System (Higher Priority Tasks)
  • Get familiar with camera system available
  • Implement application that transmits video over the network in standard format. Requirements:
    • Control of frame per second transmitted over the network
    • Potentially start with transmitting single frames (i.e. still pictures)
  • Implement server application for object recognition. Standard libraries are available. Random references:
    • ​http://dl.acm.org/citation.cfm?id=2742663
    • ​http://synrg.csl.illinois.edu/papers/overlay.pdf
  • Collect training set of objects/buildings
  • Implement simple graphical interface to display processing results
• Part B: MobilityFirst
  • Get familiar with MobilityFirst and its prototype
  • Run basic experiments on Orbit
  • Get familiar with MobilityFirst Network API
  • Test basic applications using MobilityFirst’s VN with ASR
• Integration
  • CBS application port to the MobilityFirst API. Two options:
    • Through IP-to-MF proxy and vice-versa
    • Replacement of network logic through native MF implementation (preferred option)
  • Test on Orbit MF topology
  • Run on top of MobilityFirst VN with ASR

Image Processing

We implement SURF algorithm to process object identify and match.

The core idea of SURF algorithm can be summarized to the followings:

1.Use Hessian Matrix and Scale Space to calculate key points of an image
2.Use SURFDescriptorExtractor to find feature vectors and to complete related calculations.
3.Use BruteForce to match feature vectors within two images.
4.Use drawMatches to draw matched key points.

From the perspective of image processing each picture is consisted of n × m pixels which can obtain its corresponding determinant value of Hessian Matrix. Given a point p = (x,y) within an image [1] the Hessian matrix H(p, σ) at point and scale σ, is defined as follows:
where etc. are the second-order derivatives of the grayscale image.

After the calculations of determinant value of each pixels a scale-space that contains repeatedly smoothed images through a Gaussian filter will be constructed. A scale-space can be regarded as an image pyramid built with multiple layers of an image with distinct scales. Determinant value of each pixel will be compared with that of other surrounding 26 pixels consisting of 9 upward ones, 9 downward ones and 8 encompassing ones. The maximum one will become a key point. From a circular region around the key point, a square region will be built to find its orientation and extract the SURF descriptor in order to construct its feature vector. After the calculations of all feature vectors of two images by following the above method, BruteForce algorithm matches feature vectors by calculating the shortest Euler distance of each of two vectors in all vectors. Each two feature vectors with the shortest distance is a pair of matched vector. At last we use drawMatches to draw matched key points.

Weekly Summary

Week 1

 Familiarizing with Mobility First.

 Completing Orbit Tutorials.

Week 2

 Explore Object recognition algorithms

 Development of Image processing with basic functionality

 Implement camera sampling apps with integrated webcam and mjpg-streamer

Week 3*

 Further development of Image processing and camera apps

Week 4*

 Complete the integration of client and server part, the system supports real-time video transmission and image matching on Linux

Week 5-6*

 implemented mobility first network on real video transmission to replace TCP/IP protocol

 add message distribution API on mobility first code so that it will assign to each connecting socket a thread number as an identification

 revise image processing code to support multiple clients while considering on memory optimization

 start to learn the concept of fog computing

• To be updated

Future Goals

 Incorporation with Google glass for live object recognition and information display

 Build CPS project on Android platform

 Client side low resolution object verification for coherent information display

Team

     Karthikeyan Ganesan                  Wuyang Zhang                          Zihong Zheng                        Shantanu Ghosh                             Avi Cooper

Presentation Slides

Week 1 (​Presentation)

Week 2 (​Presentation)

Week 3 (​Presentation)

Week 4 (​Presentation)

Week 5 (​Presentation)

Week 6 (​Presentation)

Week 7 (​Presentation)

Week 8 (​Presentation)

Week 9 (​Presentation)

Week 10 (​Presentation)

Week 11 (​Presentation)