Perceptual Video Quality

Author

Stefan Winkler

Introduction

Evaluating and optimizing the quality of digital imaging systems with respect to the capture, display, storage and transmission of visual information is one of the biggest challenges in the field of image and video processing.
In particular, lossy compression methods and errors or losses during transmission introduce distortions whose visibility depends highly on the content. Subjective experiments, which to date are the only widely recognized method of determining the actual perceived quality, are complex and time-consuming, both in their preparation and execution. Basic distortion measures like mean-squared error (MSE) or peak signal-to-noise ratio (PSNR) on the other hand may be simple and very popular, but they do not correlate well with perceived quality.
These problems necessitate advanced automatic methods for video quality assessment. Ideally, such a quality assessment system would perceive and measure video impairments just like a human being. Two approaches are possible:

1. The "psychophysical approach", where metric design is based on models of the human visual system. Such metrics try to incorporate aspects of human vision deemed relevant for quality, such as color perception, contrast sensitivity and pattern masking, to name a few. Due to their generality, these metrics can usually be used in a wide range of video applications.
2. The "engineering approach", where metrics make certain assumptions about the types of artifacts that are introduced by a specific compression technology or transmission link. Such metrics look for the strength of these distortions in the video and use these measurements to estimate overall quality.

Quality metrics can be further classified into the following categories:

• Full-reference metrics do a frame-by-frame comparison between a reference video and the video under test; they require the entire reference video to be available, usually in uncompressed form, which is quite an important restriction on the usability of such metrics.
• No-reference metrics look only at the video under test and have no need of reference information. This makes it possible to measure video quality of any video, anywhere in an existing compression and transmission system.
• Reduced-reference metrics lie between these two extremes. They usually extract a number of features from the reference video (e.g. amount of motion, spatial detail), and the comparison with the video under test is then based only on those features.

Current and future research

Quality assessment for television applications has become quite well established. Video playback on a PC (multimedia applications), video streaming over packet networks such as the Internet and over wireless links to mobile handsets (e.g. UMTS) is an entirely different matter. These applications comprise a wider range of frame sizes, frame rates and bitrates, and thus exhibit a much wider range of distortions. Network conditions (e.g. congestion, packet loss, bit errors) are largely different from the ones occurring in TV transmission.
In this multimedia/streaming framework, we investigate the quality of typical streaming content, codecs (MPEG-4, Motion JPEG 2000, Real Media, Windows Media), typical bitrates and network conditions. We study the design of subjective experiments, comparing different presentation and assessment methods, and analyzing their reliability. At the same time, we use the data obtained in these experiments to test and improve the performance of existing quality metrics and to develop novel metrics for these applications. Our focus is mainly on low-complexity, no-reference metrics for jerkiness, blockiness, blurriness and noise artifacts. We use these metrics in a variety of settings, including:

• Evaluation and comparison of codecs;
• End-to-end testing of video transmission systems;
• Online quality monitoring and control.

Publications

Collaborations

Sabine Süsstrunk
Signal Processing Lab, EPFL
Genista Corp., Tokyo

Funding

EPFL internal research grant