What is Conversational Quality (CQ)

by Eugene Mandel
May 18, 2023

Introducing CQ

Conversational Quality (CQ) is a metric that evaluates the customer’s perspective of their interaction. It takes into account the agent’s conversation management skills as well as factors beyond the agent’s control, such as company policies and products. The goal of CQ is to determine whether the customer feels that their issue has been resolved, and that they have been treated fairly and taken care of.

We developed CQ because we identified a gap in the traditional methods used by teams to measure customer outcomes and experience quality. While metrics like CSAT (Customer Satisfaction) and NPS (Net Promoter Score) have their value, they provide an incomplete picture. This is because survey-based scores suffer from low coverage and survey bias. Typically, only the most dissatisfied or satisfied customers respond to surveys, and the act of being surveyed can influence their answers. As a result, Customer Service leaders are now seeking inferred customer experience metrics that go beyond traditional survey-based data in order to make more informed decisions about optimizing customer support.

CQ of an Individual Conversation

CQ answers the question: “How did this conversation go?”

There are three key components that contribute to the CQ of a conversation:

1. Conversation Path

Conversation path is the shortest answer to the question “How did this conversation end?”.

We currently define four paths, but more specific paths may be identified in the future:

“Golden path”: The customer thanked the agent and acknowledged that their issue was addressed. This is a positive outcome.
“No news is good news”: The customer abandoned the conversation after receiving an answer from the agent. We don’t have direct evidence of satisfaction or dissatisfaction. This is a neutral-to-weak-signal-positive outcome.
“Rough landing”: The customer was dissatisfied with the agent’s answer at the end of the conversation. This is a negative outcome.
“Failure to launch”: The conversation ended before reaching any of the above outcomes. The customer abandoned it during the authentication or clarification stage.

Note: To follow one of the first three paths, the conversation must reach the solution stage, beyond initial greetings, verification, and other conversation “rituals.” The agent must provide a solution or answer.

2. The intensity of the customer sentiment in the end of the conversation

The level of positive or negative sentiment expressed by the customer at the end of the conversation impacts the CQ. For example, one must recognize there is a difference between a simple “Thanks” and an enthusiastic “Really appreciate it – you have been super helpful – thank you!!!”

3. Reasons to Review

The presence of negative conversational attributes affects the CQ score. We start with the following issues:

A threat to file a complaint to a regulator
A threat to post a bad review
Request to speak to a manager
Customer expresses feeling abandoned
Explicit frustration
Customer feels misled
Customer complains about wasting time

Only reasons to review appearing after the agent’s first message are considered for determining CQ. Over time, we aim to identify issues directed specifically at the agent’s conduct, as some threats or complaints may be related to factors like policy that are beyond the agent’s control.

CQ of a conversation is using the five-star scale. Our users are familiar with this scale from many places, from Yelp’s restaurant reviews to various e-commerce sites.

CQ is measured on a five-star scale, which users are familiar with from various platforms like Yelp and e-commerce sites. The primary factor in CQ is the conversation path, while the presence of negative attributes and the intensity of customer sentiment provide secondary considerations.

Path: “Golden path”
No negative attributes

Path: “Golden path”
Some negative attributes are present

Path: “No news is good news”
No negative attributes

Path: “Rough landing” with mild negativity or Path: “No news is good news” with negative attributes

Path: “Rough landing” with strong negativity (either in negative attributes or in the final sentiment)

Note: Conversations following the “Failure to launch” path do not receive a CQ score as they do not reach the solution phase.

CQ of an agent/team/organization

Agent CQ is calculated based on the CQ scores of all conversations the agent had during a specific period. For example, “John’s CQ for last week is 86.” It represents how well the agent maximized customer satisfaction during that time.

Agent CQ ranges from 0 to 100 and is determined by the percentages of conversations rated with one-star, two-star, and so on during the specified period.

CQ can be computed for any set of conversations, such as those handled by a specific team over a period or conversations with a particular intent.