Application Note - Video MOS On YouTube
Application Note - Video MOS On YouTube
Application Note - Video MOS On YouTube
(2)
Application Note
8002667883 01
The software contained in this product makes use of several valuable open source software packages. For information, see the
"Open Source Acknowledgment" on the user documentation CD-ROM (included in delivery).
Rohde & Schwarz would like to thank the open source community for their valuable contribution to embedded computing.
SwissQual AG
Allmendweg 8, 4528 Zuchwil, Switzerland
Phone: +41 32 686 65 65
Fax:+41 32 686 65 66
E-mail: info@swissqual.com
Internet: http://www.swissqual.com/
Printed in Germany Subject to change Data without tolerance limits is not binding.
R&S is a registered trademark of Rohde & Schwarz GmbH & Co. KG.
Trade names are trademarks of the owners.
SwissQual has made every effort to ensure that eventual instructions contained in the document are adequate and free of errors and
omissions. SwissQual will, if necessary, explain issues which may not be covered by the documents. SwissQuals liability for any
errors in the documents is limited to the correction of errors and the aforementioned advisory services.
Copyright 2000 - 2015 SwissQual AG. All rights reserved.
No part of this publication may be copied, distributed, transmitted, transcribed, stored in a retrieval system, or translated into any
human or computer language without the prior written permission of SwissQual AG.
Confidential materials.
All information in this document is regarded as commercial valuable, protected and privileged intellectual property, and is provided
under the terms of existing Non-Disclosure Agreements or as commercial-in-confidence material.
When you refer to a SwissQual technology or product, you must acknowledge the respective text or logo trademark somewhere in
your text.
SwissQual, Seven.Five, SQuad, QualiPoc, NetQual, VQuad, Diversity as well as the following logos are registered trademarks of SwissQual AG.
Diversity ExplorerTM, Diversity RangerTM, Diversity UnattendedTM, NiNA+TM, NiNATM, NQAgentTM, NQCommTM, NQDITM, NQTMTM,
NQViewTM, NQWebTM, QPControlTM, QPViewTM, QualiPoc FreeriderTM, QualiPoc iQTM, QualiPoc MobileTM, QualiPoc StaticTM, QualiWatch-MTM, QualiWatch-STM, SystemInspectorTM, TestManagerTM, VMonTM, VQuad-HDTM are trademarks of SwissQual AG.
Contents
Contents
1 Introduction............................................................................................ 5
1.1
1.2
Measurement Considerations......................................................................................6
2.2
2.3
4.2
4.3
4.4
4.5
Compatibility............................................................................................................... 18
Annex.................................................................................................... 19
A Customer Support Center................................................................... 19
Contents
Introduction
A Short History of YouTube
1 Introduction
YouTube is moving away from a pure video-streaming platform that streams uploaded
videos to an application on a smartphone or a PC. YouTube has become a complex
and interactive portal that provides additional information prior and parallel the video.
The video itself is delivered in different qualities according to access technology. YouTube encrypts video transmission and uses DASH now to react dynamically to available network capacity by reducing or increasing the video bitrate. This approach leads
to variable video quality and targets the best trade-off between compression and freezing, that is, preferring a slightly blurry video to a stalled video.
Consequently, testing and measuring the YouTube video streaming service has
become more complicated. The change of the YouTube session structure moves away
from the sequential step-by-step buildup as assumed in standard measuring procedures. As a result, some trigger-points as defined by ETSI are either no longer accessible through encryption or now have a different meaning due to the different session
structure. SwissQual now focuses on events that are visible to a viewer to provide a
more customer oriented, flexible, and future-proof solution.
Bitstream encryption is not the only challenge. YouTube also scales down the resolution and changes the spatial compression of a video based on access technology and
network capacity. The YouTube client requests video chunks in a quality that is appropriate to the current channel capacity. Pre-buffering is scaled up in high capacity situations and video chunks are re-loaded when the network improves. All of these changes
render the mean video bitrate obsolete as an indicator of video quality. Hence a high
quality algorithm that considers all of these impacts is required.
SwissQual has integrated a real visual quality measure that combines bitstream information with image information to consider what the viewer actually sees. The implemented measure is the ITU-T J.343.1 standard from 2014, which is the first imagebased no reference model that has been standardized in ITU-T. The ITU-T J.343.1
algorithm is available on QualiPoc Android and has been designed for video streaming
services that use encrypted streams. This lean algorithm combines image and IP
header information to predict Video Mean Opinion Scores (MOS) and is perfectly tailored for use on smartphones.
By introducing this Video MOS for YouTube video streaming, SwissQual emphasizes
and proves its leadership in applied research and Android development in order to
handle the latest technology trends.
Introduction
Measurement Considerations
Since networks could not deliver constant throughputs, YouTube, along with other
video on demand services, implemented a progressive download strategy to avoid
interruptions in the video stream. This strategy created a buffer which stored the partially downloaded video stream before the playout started and to bridge gaps that would
have occurred in the absence of new packets. The determination of buffer length continues to be a topic of optimization.
YouTube soon became interesting to professional and semi-professional content providers and its content grew rapidly. The YouTube platform was extended to include
additional information, such as advertisements and the listing of the most watched videos.
Technological advances since YouTube's inception have resulted in more people viewing its content on mobile YouTube apps than on traditional PC player base.
YouTube has been moving away from streaming single MP4 files, where the entire clip
was encoded with at a specific bitrate, to hosting the same video clips at different resolutions, including other transmission technologies. YouTube recently introduced
Dynamic Adaptive Streaming over HTTP (DASH) and has started a transition to other
video codecs.
YouTube still provides the older formats and continues to support players and smartphone Apps that have not been updated for years. That is, each video is available in
several formats and based on the requesting application, YouTube streams the matching format. Even today, RTSP streams from the early days are accessible, but in a
reduced number of resolutions. This approach leads to the fact that for the same
requested video, physically very different streams are delivered to different player or
mobile App versions.
User client
720p
Video upload
(for example,
HD720)
480p
Transcoding
Downstream
(for example,
360)
Player
360p
Display
240p
YouTube
720p
Video upload
(for example,
HD720)
480p
WLAN
LTE
Transcoding
Player
360p
240p
Re-scaling to
phone display
2G / 3G
Figure 2-2: Schematic of YouTube selecting a fixed resolution streaming based on access technology
The first resolution that was offered to mobile users was 240p, which was later
increased to 360p. LTE clients can also receive 480p and even HD content on request.
DASH
Control
Mobile user client
'YouTube App'
720p
Video upload
(for example,
HD720)
480p
Transcoding
Player
360p
Re-scaling to
phone display
240p
Switching to this approach, the YouTube client on a PC or a mobile can use network
conditions to decide whether the next piece of video should be requested in a lower
resolution to ensure the transmission in time or to request more pieces in a higher
quality and higher bitrate to fill up the buffer under good network conditions.
The YouTube algorithm considers many features to achieve an optimal perceived quality. It can control which video quality is requested and for how long as well as the buffer
management.
The goal is to find the best trade-off between the following considerations:
Short pre-buffering and re-buffering to ensure that video playback starts in a short
time
Avoid freezing
This trade-off is illustrated in Figure 2-4 which shows the buffer status and the reload in
case of improved network conditions.
No Freezing
Low compression quality
Low network load
Network capacity
Potentially Freezing
High compression quality
High network load
480p
Minimized Freezing
Best compression quality
Optimal network load
Network capacity
Variable bitrate
720p
480p
360p
360p
240p
Figure 2-4: Principle transmission of low, high and adaptive bitrate
The use of a constant low rate will minimize freezing since a low rate can pass even
under low capacity; however, the perceived video quality is low due to the strong compression even in high capacity situations.
A constant high rate delivers good visual quality if the network is able to transport the
stream, but there is a high risk of freezing and a high network load in low capacity
infrastructure.
Finally, the adaptation of the bitrate also minimizes freezing and delivers the best possible quality. There also means to further restrict the network load for low capacity networks.
The use of DASH possibilities is just starting. Current YouTube mobile Apps request a
resolution based on access technology only and do not rapidly switch the resolution in
10
response to changing network conditions. However, this is expected soon and all capabilities are on board already.
11
12
As a result, a reliable estimation of video quality is only possible if the quality measure
analyses the video frames that are actually displayed and takes into account information from the decoded images. This is the reason why a video-quality measure that
examines the received bitstream and the image information is now integrated in the
SwissQual YouTube testing approach.
13
Video
Source
Encoding
Transcoding
MPX
IP
Decoder
Player
Meta information
J.343.1
No information about
reference video required
Lastly, J.341.1 is not dependent on one video resolution and contrary to previous video
quality measures, J.343.1 can produce quality scores on any resolution up to 1080p
HD content. This algorithm is perfectly suited for the variable bitrates and resolutions
that YouTube now uses.
14
be used precisely in a mobile streaming use case on a smartphone. Despite the video
resolution delivered by the YouTube server, the native video drivers on the smartphone
scale the received video frames up to the phones resolution. This is what the user
sees. The video quality algorithm does the same, that is, it receives a video in a certain
resolution and rates the video as if it were scaled to 1080p. This fixed upscaling allows
you compare video in different delivered resolutions and from smartphones with different display sizes.
Server
YouTube
Network
DASH
Smartphone
Mobile user client
'YouTube App'
Control
720p
Video upload
(for example,
HD720)
480p
Transcoding
Player
360p
Re-scaling to
phone display
240p
Upscaling to 1080p
Image analysis
MOS
modelling
The J.343.1 algorithm calculates and stores the Video MOS every few seconds and
stored. Since a YouTube session is longer than a few seconds, a series of Video MOS
values that describe the quality profile for the stream are available by the end of the
stream. This series show how the Video MOS values change over time and how it is
adapting to network conditions.
It has to be mentioned that this division of the stream into shorter parts for evaluation is
made in a very sophisticated way. It can be imagined as subsequently cutting the video
every few seconds instead of hard cuts every 10 seconds. The evaluation points are
chosen with some flexibility based on the content. Preferred MOS evaluation points are
when the resolution changes or a scene cuts to another scene. In case there is a preferred cutting point in this range, the next evaluation starts before a full synch frame is
cut.
15
720p
480p
360p
240p
High
MOS
Avg MOS
Low
Figure 4-3: Principle of long-term MOS profile over a stream
As a next integration step, the weighted average Video MOS for the entire stream is
provided and can be used for direct quality comparison of different YouTube streaming
sessions.
16
allows it to differentiate between freezing (no new frames arriving) and still images
(new frames are arriving but the content is the same). This differentiation is unique for
no-reference models and enables safe freezing detection.
Another special effect in streaming video is frames without visible content, that is, a
sequence of black/very dark/mono-color frames or frames with barely visible content.
Such frames are marked but are not counted as an impairment since they belong to
the video content. They are even taken into account in the analysis of the frame rate or
the inter-frame difference.
The mono-color frames can also be white, for example, when very bright light leads to
white frames or for the blue frames introduced for synchronization in SwissQual reference videos.
Typical on YouTube
240p
~3.0
2.0 - 2.8
360p
~3.6
2.5 - 3.3
480p
4.3
3.6 - 4.1
720p
4.6
3.8 - 4.2
1080p
4.8
4.2 - 4.5
If an arbitrary video from YouTube is selected, the resolution of the content and the
image might not match the resolution. Quite often widescreen videos are provided that
contain black bars at the top and bottom as known from TV. Therefore, even if a video
is flagged as 360p, the image height might only be 290 lines. This obviously influences
the video MOS and the stream will receive a score closer to a 240p video than a 360p
video.
17
4.5 Compatibility
The support of Video MOS on YouTube depends on the low-layer video driver and its
accessibility as well of the Android version of the phone. As a result, not all SwissQual
QualiPoc and Diversity 15.3 devices can provide Video MOS values.
For a list of compatible phones see the Diversity and QualiPoc 15.3 Release Notes or
contact our support center.
Please consult Chapter A, "Customer Support Center", on page 19 for the best
method to contact us.
18
Annex
A Customer Support Center
Technical support where and when you need it
For quick, expert help with any Rohde & Schwarz equipment, contact one of our Customer Support Centers. A team of highly qualified engineers provides telephone support and will work with you to find a solution to your query on any aspect of the operation, programming or applications of Rohde & Schwarz equipment.
Up-to-date information and upgrades
To keep your instrument up-to-date and to be informed about new application notes
related to your instrument, please send an e-mail to the Customer Support Center stating your instrument and your wish. We will take care that you will get the right information.
Europe, Africa, Middle East
North America
Latin America
Phone +1-410-910-7988
customersupport.la@rohde-schwarz.com
Asia/Pacific
Phone +65 65 13 04 88
customersupport.asia@rohde-schwarz.com
China
Phone +86-800-810-8228 /
+86-400-650-5896
customersupport.china@rohde-schwarz.com
19