The Service Data and Strategy & Analytics teams are partnering to deliver the Service Experience Metric Suite (SEMS) which provides high-value insights to better understand and quantify EG Service experiences. Because these metrics are very specific, this suite will allow us to accurately and comprehensively capture the traveler and partner experience.
On this page, you'll find a list of metrics currently available or in planning/progress to improve service outcomes, inform product management direction, and so on. Below, you'll also find links to pages with in-depth explanations of these models.
Please note: If you have worked with previous models (e.g., CES v3), all have been renamed for clarity and this page is the authoritative source going forward.
Lastly, if you believe you have a use-case to leverage some of these metrics, want to propose metrics, or just have questions, please reach out to us in the dedicated Slack channel: #service-experience-metric-suite
Our industry uses many metrics (e.g., Net Promoter Score, Customer Effort Score) to quantify experiences. Expedia adopts and aligns with industry best practice metrics, and drives decisions and investments based upon them. These metrics, however, are sometimes abstract and subjective, hence their usefulness is limited.
For us to maintain our position as an industry leader, we need to expand our approach. We should define new metrics tied to specific outcomes driven by AI or statistical models powered by our rich datasets; the Service Experience Metric Suite will encompass all of these metrics. Our latest addition is the Repeat-Purchase Likelihood (RPL) model, which is an evolution addressing some limitations regarding the Detractor Likelihood (DL) and Repeat Contact Likelihood (RCL) models. Though industry uses metrics like Customer Effort Score (CES) to quantify effort which impacts outcomes, the metrics we are developing directly quantify outcomes which are far more powerful.
The SEMS will grow as we generate new ideas, acquire new datasets, and provide new experiences. What we build and provide will also be informed by you, so please reach out to us to discuss insights you want to unlock.
| Metric | Data Grain | Details | Current Use Cases | Status |
|---|---|---|---|---|
| Agent Quality Index (AQI) | Agent | A statistical model scoring an agent's ability to provide an excellent customer experience. This metric feeds into the Agent Quality Scorecard (AQS). Input variables
|
|
Live in Production |
| Agent Quality Model Score (AQM) | Agent | A machine learning model that predicts agent quality as measured by repeat booking rate. This metric will feed into the Agent Quality Scorecard (AQS). Input variables
Output values
|
|
Live in Production |
| Detractor Likelihood (DL) currently named CES, but to be updated in Q4 | Conversation Itinerary Customer |
An AI-based model trained to predict the likelihood of a customer being a detractor regarding Net Promoter Score (NPS). Input variables
Output values
Model Performance Metrics
|
|
Live in Production |
| Predict Product (UPP) | Conversation | A machine learning model that predicts a traveler product based on transcript text, when latestvaprod= UNKNOWN. Output values
|
|
Live in Production |
| Repeat Contact Likelihood (RCL) | Conversation | An AI-based model that measures the platform efficiency by predicting the likelihood of an agent-assisted follow-up from a customer after 72-hrs when conversation is completed. Input variables
Output values
Model Performance Metrics
|
|
Live in Production |
| Repeat-Purchase Likelihood (RPL) | Customer + Itinerary | An AI-based model trained to predict the likelihood of a repeat-purchase, meaning a customer makes a repeat purchase within a 90-day window from an initial purchase. Input variables
Output values
Model Performance Metrics
|
|
Live in Production |
| Customer Effort Score (CES) | Conversation Journey |
Industry-standard metric that is a weighted average of all effort drivers to quantify the customer's effort during a service interaction. Input variables
Output values
|
Requested but not yet available | |
| Schedule Quality Index (SQI) | Agent Forecast | A metric that measures how well staffing levels were met when compared to established requirements for a given time frame. . Input variables
Output values
|
|
In development |
| World Class Experience Compliance (WCEC) | Expedia Group is focused on delivering a World Class Experience, and doing so requires us to adhere to SLAs across many dimensions. This model will quantify strict and comprehensive SLA compliance using variables representing our commitments to Travelers and Partners. Input variables
|
Requested but not yet available |