Documentos de Académico
Documentos de Profesional
Documentos de Cultura
Quality Assessment
Dat Truong Tieu+, Binh Cong Nguyen+ , Linh Manh Le+, Quan Minh Pham+, Quang Tri Nguyen* and Dung Trung Vo+
+
Faculty of Electrical and Electronics Engineering
University of Technology, VNU-HCM
Ho Chi Minh City, Viet Nam
*
Personify Inc
Chicago, Illinois, USA
Abstract— To understand better what ha ppens when video streaming thus play a very important role in v ideo
streaming takes place, this paper introduces a framework to transmission.
simulate a real-time video streaming over wireless channels. The
system is divided into many modules and is simulated with In [1], the effect when blocks are missed in a single frame
different tools. DUMMYN ET is used for the network simulator during transmission is studied. That paper simu lates a single-
while FFMPEG is for coder/encoder an d sender/receiver module. hop wireless testbed with IEEE802.11n W LAN configuration.
The video quality is measured by spatial (SS IM and PS NR) If blocks in an intra-frame are lost due to one or more packages
metrics. To permit automatic characteristic, a system controller is lost, not only the quality of this intra-frame is affected but
with JENKINS is used to control the encoder, sender, network, also the quality of the inter-frames predicted based on this
receiver and decoder. A script at this server can be run to intra-frame is affected. This case is studied in [2] where this
simulate automaticly the system with many test cases for the error propagation is investigated with package loss. If the
practical network setting. The simulation results show that the artifact happens at fixed location for a long time, it becomes
simulated system can estimate accurately the quality of the very annoying. [3] also studies and predicts the visibility of
videos transmitted over wireless network. package loss for MPEG-2 videos. If both the original and
received videos are available, the quality metrics can be
Keywords— video streaming; quality assessment; FFMPEG;
calculated by a frame-based method. Metric with frame -based
DUMMYNET; JENKINS; PSNR; SSIM
methods are PSNR [4] and SSIM [5]. Not many metrics are
applied in real framework for video streaming yet. If the
I. INT RODUCTION original v ideos are not available at the decoder, optical flow [1]
Video streaming plays a very important role in our daily can be used to estimate the temporal quality of the videos.
lives and has become very popular these days. There exists a Based on a framework with video quality assessment, a
very diverse range of different video communication and specific solution to enhance their performances can be verified
streaming applications from video-on-demand channels like and be compared to other solutions. One application examp le is
YouTube, Netflix to live sports broadcasting or interactive on- that if the transmission bandwidth is dropped or is not stable,
line games and video conferencing like Skype. However, our the codec can reduce the speed by downsampling the frames or
knowledge about those kinds of applications’ behaviors in the increase the compression ratio. The video is thus in smaller
real world is quite not much. Therefore, to understand video bandwidth and is easier to be transmission over the channel
streaming better, it is necessary to build a framework for without any interruption. For these purposes, a novel
assessing their behaviors. framework is proposed in this paper to simulate the whole
Videos which are delivered to the users by transmission are process of video streaming in the real system. The system
suffered from the imperfect of the channel, such as package includes different modules which are implemented by specific
loss, delay, bandwidth drop… For v ideos, their quality is tools. Especially, the quality assessment module is done for
perceived by the viewers in a d ifferent scope than images’ spatial metrics.
quality. For images, the spatial quality is a concern. For The rest of the paper is organized as follow: section II
videos, the temporal quality of frames over time is also describes the architecture of the proposed framework for
considered. In a higher level, annoyance to video viewers can simulating the process of video streaming, section III, IV, V
be caused by one missing block or one low quality frame or and VI will describe in more details all the components of the
temporal quality inconsistent. Quality metric on v ideo framework. In section VII, measurement of the proposal
streaming is thus the key measurement not only to let viewers framework performances under some network conditions is
verifying the video quality but also to permit v ideo reported. Then, the conclusions and future works are
broadcasters enhancing their network. Quality metrics on video mentioned in section VIII.
II. PROPOSED FRAMEWORK ß Initialize the sender:
Fig. 1 describes the proposed framework, which has 5 main rtmp ://192.168.0.110/live/stream
components: Encoder/Sender module, Rece iver/Decoder
module, Network Emu lator module, Server module and Video ß Encode and send to server:
Quality Assessment module. ffmpeg -i input.mp 4 -c:v lib x264 -f flv
First, at the Encoder/Sender, the input video is encoded into rtmp ://localhost/live/stream
a bit stream. Then the bit stream is chopped into packets and ß Receive and decode:
sent to the Server. After that, the Server continues streaming
the packets through the Network Emulator, which emulates a ffmpeg -re -y -i rt mp ://localhost/live/stream live=1
plenty of network issues in the real world such as propagation output.flv
Available This work was done during the time the authors were at
Bandw idth 500kbps 35.105 0.94443 Personify Inc.
200kbps REFERENCES
100kbps 26.4432 0.80223 [1] An (Jack) Chan, Amit Pande, Eilwoo Baik and Prasant Mohapatra,
“Temporal Quality Assessment for Mobile Videos”, MobiCom’12,
August 22–26, 2012, Istanbul, T urkey.
500kbps 35.105 0.94443 [2] Nick Feamster And Hari Balakrishnan, “Packet Loss Recovery For
Streaming Video”, 12th International Packet Video Workshop,
150ms Pittsburgh, PA, April 2002
100kbps 26.4432 0.80223
[3] Kanumuri, S.; Cosman, P.C.; Reibman, A.R.; Vaishampayan, V.A.,
Propagation "Modeling packet-loss visibility in MPEG-2 video," Multimedia, IEEE
Delay Transactions on , vol.8, no.2, pp.341,355, April 2006
500kbps 35.105 0.94443
doi: 10.1109/TMM.2005.864343
50ms [4] Q. Huynh-Thu and M. Ghanbari. Scope of validity of PSNR in
100kbps 26.4432 0.80223 image/video quality assessment. Electronics Letters, 44(13):800 –801,
19 2008.
[5] Z. Wang, L. Lu, and A. Bovik. Video quality assessment based on
structural distortion measurement. Signal Processing: Image
Apparently, based on Table I above, it can be seen that the Communication, 19(2):121–132, February 2004.
network characteristics does not affect the quality of the [6] https://www.ffmpeg.org/
output videos. All the PSNR values of the output videos with [7] http://info.iet.unipi.it/~luigi/dummynet/
the same bitrate are similar. The same thing happens to the [8] https://www.adobe.com/support/flashmediaserver/downloads_updaters.h
SSIM values. The reason is because of the chosen streaming tml
protocol, which is RTMP. RTM P is a TCP-based protocol, [9] https://jenkins-ci.org/
which obviously has flow control and erro r control. Therefore, [10] Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “ Image
the output videos almost have its best form co mparing to the quality assessment: From error measurement to structural similarity,”
input videos. The only thing that affects the quality of service IEEE Trans. Image Processing, vol. 13, Jan. 2004.
here is just the video bitrate. Further check with the PSNR and [11] S. Chikkerur, V. Sundaram, M. Reisslein and L. J. Karam, “Objective
Video Quality Assessment Methods: A Classification, Review, and
SSIM at the sender verifies the robustness of the PSNR and Performance Comparison”, IEEE Trans. Broadcasting, vol. 57, No. 2,
SSIM values calculated at the receiver using the proposed June 2011.
Video Quality Assessment (VQA) module.