How good is good enough? PBXL IP Voice quality over the 3G/LTE/Public Wi-Fi networks (KDDI/AU version)
At PBXL we conduct various quality tests and experiments of the products we offer. While we always start by using our own products ourselves, or “eating our own dog food”, we also conduct specific tests to mimic customer situations.
Over the course of a couple months we have conducted various tests to determine the best operating conditions for mobile client. These include phone operating system (iPhone vs. Android), network type (3G, LTE, Wifi), as well as location and time of day.
Typical call quality issues that we are looking at are dropped calls, calls sounding like they were made speaking under water, “metallic sounds”, and excess noise.
The Goals:
- Determine which conditions provide the clearest and least dropped or ‘glitched’ calls
- Provide the PBXL engineering support team with data and experience to better help troubleshoot customer problems
The Set Up:
- 1 PBXL engineer visiting 5 different cafes near subway stations in central business districts of Tokyo
- At each location a call was made from cell phone to office phone, via the PBXL VOIP client of course
- A voicemail was left on the office phone
- “Original signal”: a second iPhone5 records the speaker as they talked on the cell phone to provide a copy of the call without any influences of cellular transmission
- MOS: Mean Opinion Score, a common test in telephony networks, is used to analyze the voicemails
The Scale (human perception to sound quality):
| MOS | Quality | Impairment |
|---|---|---|
| 5 | Excellent | Imperceptible |
| 4 | Good | Perceptible but not annoying |
| 3 | Fair | Slightly annoying |
| 2 | Poor | Annoying |
| 1 | Bad | Very Annoying |
| 0 | Terrible | Overwhelming, covering content |
Judged, measured and weighed:
- Voicemail recordings were rated and glitches, drops, perceived voice quality, clarity etc. were noted
- “Original signal” files were used in the analysis to gauge the natural background noise quality (talking, background music, cars etc.) thus, separating this noise from the noise due to cellular transmission
The Results:
So what myths and rumors about VoIP networks were proven or debunked? And what networks were found wanting? Look forward to the next blog to see the results and analysis of these tests!
