= Dynamic Video Encoding = [[TOC(Other/Summer/2015*, depth=3)]] == Introduction == Live-streaming video from a device to the Internet in an efficient manner can be a tricky task. The concept of video-streaming itself is simple; the device records whatever is going on and then streams the video file information live to a video-hosting website. The problem arises with the nature of internet/data/WiFi connections. Having an unstable connection will result in a low, partially corrupted, and blurry video file on the streaming server. The original video file can always be uploaded later on to wherever the video was streamed to, but just simply re-uploading the same video file is a waste of bandwidth and time. Bits and pieces of the video file were already sent to the streaming server during the live-streaming process; why not use the information previously sent to figure out and upload whatever is missing? That's the goal and name of our project: Dynamic Video Encoding. == Goals == * Automation of bash script procedures via Ruby scripts. * Dynamic adaptive streaming over HTTP (DASH). * Streamline DASH encoding * DASH video streaming tests with variable bitrate control == Background Information == === Anatomy of a Video File === 1. Container * Defined by file extension * Holds various audio/video streams * Specifies contained encoding scheme a. Encoding scheme defines algorithm(s) used to encode/decode video 2. Content stream * Contains encoded audio/video streams * Limits video use to only specific codecs === What is a CODEC? === 1. Encoder-Decoder * Algorithms defined in compression standard * Can sometimes encode/decode several compression algorithms * Popular compression algorithms include JPEG/MPEG family === H.264 Compression Algorithm === 1. Defined in MPEG-4 Part 10: Advanced Video Coding * Implements many common image/video compression techniques a. Discrete Cosine Transform (DCT) b. Motion Compensation === Scalable Video Coding === 1. Uses various methods to split a high quality bit stream into component layers * Temporary Scalability * Spatial Scalability * SNR Scalability 2. Decoder recombines selected layers to form output video stream [[Image(Scalable Video Coding.png)]] === Network Emulator Test Results === 1. Original File [[Image(original.gif)]] 2. 5% Packet Loss Simulation [[Image(5Loss.gif)]] 3. 8 Megabit Connection Simulation [[Image(8mbit.gif)]] 4. 12 Megabit Connection Simulation [[Image(12mbit.gif)]] === DASH Multi-Bitrate Encoding === * DASH - Dynamic Adaptive Streaming over HTTP * Encodes single video into multiple resolutions/bitrates * Video files are segmented in certain time intervals * Manifest file describes resolutions and bitrates of separately encoded files * Client and server monitor network connections and switch video files as necessary === DASH Content Generation === * Minimal existing software support for encoding/playing DASH content * Encode multiple copies of videos using ffmpeg * Generate manifest files using MP4Box and manually combine === Bitrate Profiles === 1. Camera Bandwidth Profile [[Image(Camerabandwidth.png)]] 2. Client Bandwidth Profile [[Image(Clientbandwidth.png)]] 3. Complete Bandwidth Profile [[Image(Completebandwidth.PNG)]] By analyzing both the camera and client bandwidth profiles, we can hopefully create a better stream quality by dynamically adapting to the network conditions and improving the quality where possible. === Video Encoding Algorithms === 1. Throughput Estimation Method * Averages past bandwidth estimates and creates a smoothed throughput estimation curve [[Image(ThroughputEstimation2.png)]] * Standard deviation determined by recent deviation from expected throughput [[Image(ThroughputEstimation.png)]] 2. DASH Adaptation Algorithm * The algorithm measures average download time of each segments * It then builds an adaptation decision based on this download time and the average bitrate of the whole video [[Image(DashAlgorithm.png)]] [[Image(DashAlgorithm2.png)]] * The function maxbw(s_i) returns the max bandwidth for that specific segment i === GPAC === 1. What is GPAC? * Open source multimedia framework * Contains a multimedia player (MP4Client / Osmo4) as well as a multimedia packager called MP4Box * Written in C * Can be manipulated through the command-line, GUI, or browser plugins 2. Why use GPAC? * The multimedia player MP4Client is a command-line player for GPAC, allowing it to be easily used on the Linux OS and via any bashscript shell * The multimedia package MP4Box is a powerful tool that can be used for manipulating many video file formats such as .avi, .mpg, .ts, and most importantly, .mp4 * MP4Box can be used for conversion, splitting, combining, etc. * The most important part of GPAC, however, is that it supports DASH 3. Sample Experiment Rig [[Image(ExperimentRig.png)]] 4. GPAC Script * Script first initializes the streaming * It then limits and measures the bandwidth in parallel [[Image(GPACScript.png)]] * Results [[Image(GPACScriptResults.png)]] * The results were not quite as expected, with the bandwidth tests averages going above the bandwidth limit during some portions of the streaming, indicating a flaw either in the program or in the data == Presentations == Here are the links to our weekly presentations, presented during the duration of the summer of 2015 1. [https://docs.google.com/presentation/d/11NXayP5FTk8Zb_Hg6HNxipzxb9B5ysqpQLPrAuOZ0y4/edit?usp=sharing Week 1] 2. [https://docs.google.com/presentation/d/1NUXqK_Oj7tEsTWk6g4ZrAf0V8c513zKC0qFxbeT0eyI/edit?usp=sharing Week 2] 3. [https://docs.google.com/presentation/d/1V_FEJVmEBHWto1r_Q7mM_S63sd9R9vBZktVwbPmxn1I/edit?usp=sharing Week 3] 4. [https://docs.google.com/presentation/d/1zMEvO-eAi1Ozlhq_hdjcFyKbnZSAtwa5D3Uujm03FUM/edit?usp=sharing Week 4] 5. Week 5 - No presentation this week 6. [https://docs.google.com/presentation/d/1nFRTQEgH0WlzTLsQAJA_GO_RKKYVYOw9jfJyFgs7p2s/edit?usp=sharing Week 6] 7. [https://docs.google.com/presentation/d/11of2tEdHthjiy4h4j1ucvC8WbGUI74nfrWKXD-ysESI/edit?usp=sharing Week 7] 8. [https://docs.google.com/presentation/d/13WjH0Njo7zN4ZwaORXVXpTO82h2sfNOrcdDAmAxRUTs/edit?usp=sharing Week 8] 9. [https://docs.google.com/presentation/d/1ZHMPViesOrtnQ0keRjldtsHPMLlzx490V4xnlh4uSy8/edit?usp=sharing Week 9] 10. [https://docs.google.com/presentation/d/12fLlCRWCIolcrdnPsSYjGcxEn5Hj0bGuMLL73jICK44/edit?usp=sharing Week 10] 11. [https://docs.google.com/presentation/d/1UnO__C1YJBgrNL3rDagprc-eBgv_LZ7cLHuXOQawnSg/edit?usp=sharing Week 11] == People == * Professor Anand Sarwate * Professor Roy Yates * Danny Ayoub * Kush Oza [[Image(Danny.jpg)]] [[Image(Kush.jpg)]]