Quality of Experience With CloudCheck: Perception vs. Reality

Senior executives of 362 companies from different industries were recently asked about their customer experience.  As one might expect, 80% of them said that their company provides an outstanding customer experience.  Unfortunately, only 8% of their customers concurred with those positive assessments [1]. The executives’ responses were based on service quality metrics they use to gauge  the satisfaction level of their subscribers.  This disparity between the executive perspective and customer view of experience highlights the difference between assessments based on the service quality versus actual customers’ perception of their experience.

The Importance of Quality-of-Experience

Because of such discrepancies in customer perception of service quality, Quality-of-Experience (QoE) has become one of the most important indicators that measure the customers’ actual satisfaction with their service.  An emphasis on QoE has become one of the key differentiators in the marketplace between service providers in this socially-connected era.   This is important to service providers as high satisfaction equates to higher customer retention as well as lower operational expenses, not to mention the marketing and the positive brand perception promoted by vocal customers.


What About Quality-of-Service (QoS)?

QoS is an important metric that measures the level of the service.   As defined by one source that defines QoS in the telecommunications and computer networking arena, QoS “refers to traffic prioritization and resource reservation control mechanisms rather than the achieved service quality. Quality of service is the ability to provide different priority to different applications, users, or data flows, or to guarantee a certain level of performance to a data flow[1]. Many technical knobs exist in the Customer Premise Equipment (CPE) to manage access and control the performance of the different stations inside the home. QoS is focused on evaluating these controls, verifying their functionality and performance while adjusting priorities and resources to guarantee the IPTV performance or the VoIP quality, for example.

Computing QoE

QoS is one of the components that is used to measure and improve QoE.  However, QoE could be very subjective and not necessarily linked directly to the quality of the service rendered. QoE represents the aggregation of the “total experience” of the individual customers and their perception of their service.  This could be affected by uncontrolled or environmental factors.  In residential networks, QoE is an end-to-end measurement that includes both the broadband and Wi-Fi.  It is evaluated using customer data such as calls, customer surveys, NPS, and churn among other factors.   Hence, computing QoE is challenging because QoE is:

  • Not quantifiable
  • Subjective
  • Inconsistent
  • Expensive
  • Hard to track
  • Reactive

To solve the QoE puzzle, one must:

  • Identify technical parameters that relate to the customer experience
  • Quantify
  • Measure as a whole, but pinpoint the root cause
  • Correlate
  • Calibrate

In a previous blog, ASSIA discussed the importance of measuring throughput.  The first step to quantifying QoE lies in measuring and separating the broadband throughput from the Wi-Fi throughput for every station connected to a given Wi-Fi network.

While throughput is the most relevant technical parameter to QoE, it is not the only one. Connectivity and latency combined with throughput provide greater insight into a subscriber’s sense of their connection and perception of their experience.   However, such metrics can also be adversely impacted by external factors related to coverage, interference, congestion, and hardware configuration. Incorporating a multi-variant model that combines these different measurements and diagnostics to generate a single QoE score that measures the customer experience offers tremendous benefit.  ASSIA’s CloudCheck provides an overall QoE score in addition to quality scores for both Wi-Fi and broadband.

CloudCheck QoE Index

Once this quantifiable index for QoE is calculated, the model is continually tuned and validated.

CloudCheck provides a machine learning engine in the cloud server that uses external data sources such as call data, customer surveys, dispatch records to:

  • Correlate QoE with customer input data to continually validate the QoE score.
  • Calibrate periodically to adjust for seasonality and other changes in the network over time.
  • Categorize performance levels such as turning the numbers into a Red, Amber, Green (RAG) index.

Learning Engine

CloudCheck’s QoE Index and its machine learning engine offers several benefits to the service providers:

  • Quantifies QoE into a single technical automatically-generated metric.
  • Incorporates periodic and automated calibration to maintain accuracy.
  • Uses a predictive model that tracks the performance trends and signals potential customer dissatisfaction at the network level which would eventually translate into more complaints and higher churn.
  • Presents a similar measurement per individual line and links any degradation to its main driver(s).

The same approach used to conceive the QoE score can be used to derive more specialized quality scores that focus on specific services such as IPTV, VoIP, channels (Wi-Fi vs. broadband), and hardware (gateways, mesh, APs, extenders …). In a future blog, we will look more closely at these quality scores and their use cases.

[1] http://www.bain.com/publications/articles/how-some-banks-turn-clients-into-advocates.aspx

[2] https://en.wikipedia.org/wiki/Quality_of_service#End-to-end_quality_of_service