Assessing the Perceptual Quality of VoIP Transmissions

This web page contains contributions, on how to rate the service quality of VoIP systems. The contributions consists of publications, software and measurement data:

1. Predicting the Perceptual Service Quality Using a Trace of VoIP Packets
   
2. A Perceptual Quality Model for Adaptive VoIP Applications
3. Listening-Only Test Results for Non-random Packet Losses and Playout Rescheduling
4. Mongolia: An Auditory Testing Environment to Study the Importance of a VoIP Packet

Predicting the Perceptual Service Quality
Using a Trace of VoIP Packets

Christian Hoene, Holger Karl, Adam Wolisz

Abstract:

On this web page we present an instrumental approach on how to assess the perceptual quality of voice transmissions in IP-based communication networks. Our approach is end-to-end and uses combinations of common codecs, loss concealment algorithms, playout schedulers, and ITU's quality assessment algorithms E-Model and PESQ. It is the first method that takes into account the impact of playout rescheduling and non-random packet loss distributions. Non-random packet losses occur if a rate-distortion optimized multimedia streaming algorithm forwards packets dependent on the packets' importance.

We provide an open-source software package in which our approach is implemented. It evaluates both experimental and simulated VoIP systems. It includes various playout schedulers for Voice over IP (VoIP) described by Schulrzinne, Ramjee, Moon, Lin and others. Also, the two most common speech encoding schemes (G.711 and G.729) are supported. Finally, two quality models (ITU's PESQ and E-Model) predict the perceptual end-to-end transmission quality.

Keywords:

Publications:

• C. Hoene, S. Wiethölter, and A. Wolisz, "Predicting the Perceptual Service Quality Using a Trace of VoIP Packets", In Proceedings of Fifth International Workshop on Quality of future Internet Services (QofIS'04), Barcelona, Spain, September 2004.
  (PDF)

• C. Hoene, H. Karl, and A. Wolisz, "A Perceptual Quality Model for Adaptive VoIP Applications", In Proceedings of International Symposium on Performance Evaluation of Computer and Telecommunication Systems (SPECTS'04), San Jose, California, USA, July 2004, Paper won the Best Paper Award of the conference.
  (PDF)

• C.Hoene and E.Dulamsuren-Lalla, "Predicting Performance of PESQ in Case of Single Frame Losses", In Proc. MESAQIN 2004, Prague, CZ, June 2004 MESAQIN 2004 - Measurement of Speech and Audio Quality in Networks, (Slides).
  (PDF)

Software:

Version 2 (September 27th, 2004): Source Code Manual

A Perceptual Quality Model for Adaptive VoIP Applications

Christian Hoene, Holger Karl, Adam Wolisz

Abstract:

Quality models predict the perceptual quality of services as they calculate subjective ratings from measured parameters. In this paper we present a new quality model that evaluates VoIP telephone calls in order to control their transmission at run-time. In addition to packet loss rate, coding mode and delay it takes into account the impairments due to change in the transmission configuration (e.g. switching the coding mode or re-scheduling the playout time). It is also computation efficient and open source.

To demonstrate its potential, we apply our model to select the ideal coding and packet rate in bandwidth-limited environments. Furthermore we decide, based on model predictions, whether to delay the playout of speech frames after delay spikes. Delay spikes often occur after congestion and cause packets to arrive too late. We show a considerable improvement in perceptual speech quality.

Keywords:

Internet telephony, adaptive VoIP, perceptual quality model, E-model, PESQ, packetization, delay spikes.

Publication: