MOS Calculation

This article is intended to give the reader a complete description of how Accedian’s Skylight suite of products calculate the Mean Opinion Score (MOS), a general and widely-used metric, to state the quality for Voice over IP (VoIP).

Skylight sensor: control and Measurement Data

The Accedian Skylight measurement system collects measurement data using active performance monitoring sessions injected by the Skylight sensor: control. The data is periodically pushed to the Skylight orchestrator for storage and analysis. The Skylight sensor: controls measure per-packet data, but distill this into result records (RR) that are sent upstream to the Skylight orchestrator.

For VoIP measurements, the Skylight sensor: controls emulate telephone voice packet streams. For such sessions, the Mean Opinion Score (MOS) can be calculated based on the RR data on the Skylight orchestrator.

Result records are created using a fixed configurable measurement interval, by default every 30 seconds.

The calculation of MOS follows the appropriate ITU standards using the E-model [2]. However, several simplifications have been made since many of the parameters in the E- model are based on acoustical and analog models that are not relevant in a computer networking transmission environment.

Playout Buffers

Since the Skylight sensor: control is not actually playing out the audio, it emulates a playout buffer (jitter buffer) to calculate a quality measure for the audio. Such a playout buffer absorbs jitter while not delaying the data more than necessary. If the playout buffer is too large, the end-to-end (E2E) delay will be higher and thus affect the MOS value negatively. If the playout buffer is too short, packets will be lost and also affect the MOS value negatively.

During calculation of MOS value, the Skylight orchestrator assumes a number of different playout buffers based on the delay and jitter values of a measurement interval. The reasoning is that a playout buffer adapts to a changing delay and jitter and that such an algorithm is implemented in a hypothetical system. The playout buffers in the Skylight system are in a sense optimal in that they are computed in retrospect when the delay and jitter values are known. A real jitter buffer does not have such knowledge but must act on packets when they arrive at the receiver.

The MOS and R-value metrics that are exported from Skylight orchestrator via CSV and XML feeds do not include any added virtual packet loss due to jitter buffer overflow but represents the pure MOS value of the network itself.

A packet trace showing the end-to-end delay of individual packets. The median, 96%, 98% and 99% delay percentiles are also shown.

There are several ways to calculate the length of the playout buffers. One simple way is to use the delay percentiles of the traffic and shape the playout buffers based on that. All packets are then delayed to that percentile, while packets arriving later are discarded.

Assume for example the distribution shown in the above figure, where the 99% delay percentile is
102ms. The playout buffer is then designed so that all packets arriving with a shorter delay are stored until 102ms have passed. 1% of the packets, those that arrive after 102ms, are dropped.
Another method is to use a delay jitter measure combined with the mean or median to create a playout buffer. This is the method used in the Skylight sensor: controls. The jitter as defined in [1] and the percentiles of the jitter is computed. The (hypothetical) playout buffer is then set to delay all packets uniformly to the median plus a jitter percentile. Additionally, the packet encoding time is added. Packet encoding time is the packet processing time, i.e., a 50Hz stream has 20ms packet encoding time. Packet loss is also increased with lower jitter percentiles as follows:

• 99% jitter percentile: Latency is set to the median plus the 99% jitter percentile. Packet loss is increased with 0%.
• 98% jitter percentile: latency is set to the median plus the 98% jitter percentile. Packet loss is increased with 2%.
• 96% jitter percentile: latency is set to the median plus the 96% jitter percentile. Packet loss is increased with 4%.

The 96%, 98% and 99% limits are sometimes referred to as bad, good and best quality.

Further, the loss is computed as absolute loss plus reorder divided by the number of received packets plus lost minus duplicated packets.

General

The E-model [2] defines the rating factor R as:
𝑅 = 𝑅0 − 𝐼𝑠 − 𝐼𝑑 − 𝐼𝑒−𝑒𝑓𝑓 + 𝐴
where

• 𝑅0 is the basic signal to noise ration
• 𝐼𝑠 is the simultaneous impairment factor
• 𝐼𝑑 is the delay impairment factor
• 𝐼𝑒−𝑒𝑓𝑓 is the effective equipment impairment factor
• 𝐴 is the advantage factor

R to MOS mapping

𝑀𝑂𝑆 can be calculated from 𝑅 using [2], appendix B:
𝑀𝑂𝑆 = 1 + 0.035𝑅 + 𝑅(𝑅 − 60)(100 − 𝑅)7 ∗ 10−6 In the range 0 < 𝑅 < 100. This function is shown in the above figure.

For advanced codecs, an R-value range of 0-129 is used, the mapping towards MOS of Rvalues larger than 100 is fixed to 4.5.

The rest of this article describes how each component of equation 1 is computed.

The Basic Signal-To-Noise Ratio

The basic signal-to-noise ratio 𝑅0 is set to 129 following [2] default values and was updated in 2015 according to the latest version of the ITU-T model catering for wideband codecs. This value was previously set to 93.2. This also has the effect that the constant used in the formula to calculate 𝐼𝑒−𝑒𝑓𝑓 has increased from 95 to 130.8 with the introduction of wideband codec support. In the Accedian Skylight system, 𝑅0 can be preset to another fixed value.

The Simultaneous Impairment Factor

𝐼𝑠 includes loudness factors, sidetones and quantizing distorsion. 𝐼𝑠 is set by default to 0. 𝐼𝑠 is also a configured value.
It should be noted that 𝐼𝑠 is 1.42 using the default values of [2], but are by most systems set to 0.

The Delay Impairment Factor

𝐼𝑑 is dependent on talker and listener echoes and absolute delay:
𝐼𝑑 = 𝐼𝑑𝑡𝑒 + 𝐼𝑑𝑙𝑒 + 𝐼𝑑𝑑

The echo delay factors 𝐼𝑑𝑡𝑒 and 𝐼𝑑𝑙𝑒 I are configurable and set to 0 by default.

Values when absolute delay in ms (𝑇𝑎) is changed, all other values kept cons

The dependency of absolute delay, 𝐼𝑑𝑑 follows [2]. It is 0 for absolute delay (𝑇𝑎) values up to 100𝑚𝑠 and increases slowly for values above 100𝑚𝑠. Figure 3 shows the effect of 𝐼𝑑𝑑 on 𝑅 when 𝑇𝑎 is varied, all other factors are kept constant.
The computation of 𝐼𝑑𝑑 is given below in C-code for 𝑇𝑎 values above 100𝑚𝑠: double X, Id;
X = (log(Ta100.0))/log(2);
Idd = 25* (pow(1+pow(X,6),1.0/6.0) -3* pow(1+pow(X/3.0,6),1.0/6.0)+2);

In the Skylight measurement system, 𝑇𝑎 is approximated over the measurement interval (default 30s). The MOS calculation assumes a fixed jitter buffer that drops all packets with a 𝑇𝑎 higher than a set of limits.

Note that this means that no jitter value is directly involved in the calculation of 𝐼𝑑 .

The Effective Equipment Impairment Factor

𝑒 − 𝑒𝑓𝑓 dependency on packet loss for a selected set of codecs at random loss

𝑒 − 𝑒𝑓𝑓 is dependent on equipment which includes codecs and effects of losses on codecs.

where the components of equation 4 are as follows:

• 𝐼𝑒 is the Equipment impairment factor,
• 𝑃𝑝𝑙 is the packet-loss probability,
• 𝐵𝑝𝑙 is the packet-loss robustness factor,
• 𝐵𝑢𝑟𝑠𝑡𝑅 is Burst Ratio.

Bpl and 𝐼𝑒 values for a selected set of codecs

The values of Ie are codec dependent and set according to the standard [3]. A selected set of values are shown in Table 1. Note however, that these values are valid for normal conditions and may be different under error conditions or large packet sizes. The standard is not complete in this area, so the base cases shown in Table 1 are always used in the system.

The packet loss probability Ppl is the percentage of packets lost during a measurement interval added to the packets loss due to the jitter buffer described in Chapter 2. For example, if the 98% delay quantile is selected, and the packet loss during an interval is 1:2%, then Ppl = 3.2%

Bpl is also taken from the standard [3]. The values are as shown in the above table for some codecs.

Again, the standard is somewhat unclear on some cases, so these are the values used.

Finally, 𝐵𝑢𝑟𝑠𝑡𝑅 is a burst measure which is 0 at random loss. 𝐵𝑢𝑟𝑠𝑡𝑅 is however currently not reported by the Skylight sensor: controls.

The Advantage Factor

The advantage factor is set to 0 by default but can be preset to another fixed value.

Reconfiguration of Constants
All three constants used in the MOS and R calculations are configurable via SkyLIGHT Director advanced CLI settings but it is not recommended to change these parameters unless advised so by Accedian. The constant names and default values are shown below:

• MOS_R0 = 129.0 - Base signal-to-noise ratio
• MOS_Is = 0 - Simultaneous impairment factor
• MOS_A = 0 - Advantage factor

Examples

General

In this section, MOS values for some codecs are presented. For each codec, two graphs are shown. One where the loss is zero and the end-to-end (E2E) delay is varied delay, and one where the delay is constant, but the loss is varied.

The MOS value is computed for four different playout buffers:

• Max – The playout-buffer is adapted to the packet with the longest delay
• 99% – The playout-buffer is adapted to the 99th delay percentile. That is, one percent
• of the packets are dropped by the playout buffer since they arrive too late
• 98% – The playout-buffer is adapted to the 98th delay percentile
• 96% – The playout-buffer is adapted to the 96th delay percentile

In the following section graphs for these codecs are shown:

• Graphs for G711 with loss concealment
• Graphs for G723.1
• Graphs for G729.a
• Graphs for GSMEfr

Codec – G.711

G.711 delay dependency

G.711 loss dependency

Codec – G.723.1

G.723.1 delay dependency

G.723.1 loss dependency

Codec – G.729.a

G.729.a delay dependency

G.729.a loss dependency

Codec – GSME Full-Rate

GSMEfr delay dependency

GSMEfr loss dependency