Monitor

Summary

The script outlines the monitoring phase of an optimization process in a CCAI Insights console, highlighting various dashboards for tracking conversations, topics, and agent performance. It emphasizes the importance of monitoring metrics such as conversation volume, topic distribution, and customer satisfaction to improve call center operations. Users can filter data to analyze trends and agent efficiency, providing actionable insights for optimizing customer interactions.

Takeaways

• 📊 The CCAI Insights console provides various dashboards to monitor signals in the Optimization process, including Conversation hub and topic models.
• 🔍 Conversations created dashboard shows daily traffic through Insights, while Conversation history provides metrics like average duration and top topics.
• 📈 Topic models dashboard displays topic distribution over time, allowing for the identification of trends or less frequent topics.
• 🔑 Conversation highlights dashboard tracks the popularity of keywords or phrases, with options to view Smart Highlights or Custom Highlights.
• 🗣 Generative FAQ dashboard monitors customer questions and how they are answered by the agent, indicating question popularity and response efficiency.
• 📞 Call Center Operations dashboard in Looker offers an overview of metrics like call numbers, containment, and sentiment.
• 📈 The dashboard also includes metrics for customer segmentation, topic volume, and conversation volume to understand customer behavior and preferences.
• 🤖 Virtual Agent Performance dashboard, when integrated, assesses the performance of virtual agents against live agents for specific tasks.
• 👥 Agent Performance dashboard focuses on live agents, offering insights into waiting times, follow-up needs, and agent-specific metrics like CSAT.
• 📝 Detailed filters can be applied to any dashboard or graph to view more granular data, enhancing the monitoring process.
• 📊 Monitoring metrics such as conversation handling, call duration, sentiment, and customer ratings can be tailored to specific agents for performance analysis.

Q & A

What is the purpose of the Monitor phase in the Optimization process?

The Monitor phase in the Optimization process is focused on tracking and analyzing various signals and metrics to assess the performance and effectiveness of the conversational AI system.

How can I view the number of conversations created per day in CCAI Insights console?

You can view the number of conversations created per day through the 'Conversations created' dashboard in the Conversation hub of the CCAI Insights console.

What metrics are displayed in the Conversation history dashboard?

The Conversation history dashboard displays high-level metrics such as the total number of conversations, their average duration, average turns per conversation, and the top three conversation topics.

How can I filter conversations by primary topic in the Conversation history dashboard?

You can use filters in the Conversation history dashboard to get a list of conversations that share the same primary topic, allowing you to analyze metrics like the number of turns or the time of occurrence.

What information does the topic models dashboard provide?

The topic models dashboard shows the distribution of topics over time, allowing you to identify spikes in specific topics or determine which topics receive the most or least amount of traffic.

How can I highlight specific topics in the topic models dashboard?

By clicking on 'View Legend', you can open a sub-menu with a list of identified topics and select one or more topics to highlight their distribution in the graph.

What does the Conversation highlights dashboard monitor?

The Conversation highlights dashboard monitors the popularity of keywords or key phrases in your conversational data, displaying each tracked phrase over time along with their count.

What is the purpose of the Generative FAQ dashboard in the CCAI Insights console?

The Generative FAQ dashboard shows questions asked by customers and outlines how these questions are answered by the agent, helping to monitor trending questions and the popularity of each question.

What metrics are included in the Overall KPIs dashboard of the Call Center Operations?

The Overall KPIs dashboard includes metrics such as the number of calls, containment, call duration, cost savings, and average sentiment for the entirety of your data.

How can I review conversation topics by customer segmentation in the Customer and Topic Overview dashboard?

In the Customer and Topic Overview dashboard, you can review conversation topics by customer segmentation, topic volume over time, overall conversation volume per day, or percentage of repeat callers.

What insights can I gain from the Virtual Agent Performance dashboard?

The Virtual Agent Performance dashboard, applicable with Insights integration, allows you to monitor where your virtual agent performs well on its own or in comparison to live agents, providing insights into customer satisfaction and performance metrics.

How can the Agent Performance dashboard help in monitoring live agents' performance?

The Agent Performance dashboard helps in monitoring metrics such as waiting time to speak to a live agent by topic, identifying topics that require follow-ups, and reviewing a breakdown of live agents' performance, including average CSAT, number of conversations handled, and average waiting time.

What additional functionality does clicking on information in any dashboard provide?

Clicking on any piece of information in a dashboard or graph automatically applies a filter, updating the entire Call Center Operations page to show all metrics by the applied filter, allowing for more granular monitoring of metrics.

Outlines

00:00

📊 Monitoring Optimization Signals in CCAI Insights

In the Monitor phase of the Optimization process, the script discusses various signals available for monitoring in the CCAI Insights console. It highlights the Conversation hub with two dashboards: one showing daily conversation counts and the other offering high-level metrics such as total conversations, average duration, turns per conversation, and top conversation topics. Filters can be applied to analyze conversations by primary topic. The topic models dashboard displays topic distribution over time, allowing for the identification of spikes or trends in specific topics. The Conversation highlights dashboard tracks the popularity of keywords or phrases, with options to view Smart Highlights or Custom Highlights. The Generative FAQ dashboard monitors customer questions and their responses by the agent, providing insights into trending questions and the efficiency of answers. Additionally, the script touches on monitoring signals in Looker, detailing dashboards for Call Center Operations, including Overall KPIs, Customer and Topic Overview, Virtual Agent Performance, and Agent Performance, with the latter offering a granular view of live agent metrics and the ability to apply filters for detailed analysis.

05:01

👥 Agent Performance Analysis in CCAI Insights

The second paragraph delves into the specifics of the Agent Performance dashboard within the CCAI Insights console. It provides an example of monitoring individual agent performance, such as Tonya Koop, who did not handle any conversations in the last week. The dashboard allows for the review of customer satisfaction ratings, which for Tonya's customers were neutral or positive. This section underscores the capability of the Insights console to offer detailed insights into agent performance, including the number of handled conversations, average call duration, speaker distribution, average sentiment, topics handled over time, conversation duration by topic, and customer ratings, enabling a comprehensive evaluation of agent efficiency and customer satisfaction.

Keywords

💡Optimization process

The optimization process refers to a series of steps or stages aimed at improving the efficiency and effectiveness of a system or operation. In the context of the video, it is the method by which the CCAI Insights console is used to refine and enhance the performance of conversational AI. The script mentions the 'Monitor phase' as a part of this process, indicating that monitoring is a key component for identifying areas of improvement.

💡CCAI Insights console

The CCAI Insights console is the platform or interface through which users can access and analyze conversational data. It is central to the video's theme as it is the tool used for monitoring and optimizing conversational AI. The script describes various dashboards and features within the console that provide insights into conversation metrics and performance.

💡Conversation hub

The Conversation hub is a specific section within the CCAI Insights console that focuses on the analysis of conversations. It is a key concept in the video as it houses dashboards that display metrics such as the number of conversations created and conversation history. The script uses the Conversation hub to illustrate how users can monitor the volume and details of conversations.

💡Conversation history

Conversation history is a record of all the interactions that have taken place within the system. In the video, it is presented as a dashboard that shows high-level metrics such as total number of conversations and average duration. It helps users understand the scope and nature of the interactions that have occurred.

💡Topic models

Topic models are analytical tools used to identify and categorize the main subjects or themes discussed in a set of conversations. The script mentions a 'topic models dashboard' that shows the distribution of topics over time, helping users to detect trends or patterns in the conversation data.

💡Smart Highlights

Smart Highlights is a feature within the CCAI Insights console that allows users to track and visualize the frequency and popularity of specific keywords or phrases within the conversational data. The script describes how this feature can be used to monitor the usage of key terms over time, providing insights into the focus areas of the conversations.

💡Generative FAQ

Generative FAQ refers to a dashboard or section that displays frequently asked questions by customers and how they are addressed by the AI agent. In the video, it is used to monitor trending questions and assess the efficiency of the responses provided by the agent, which is crucial for improving customer service.

💡Call Center Operations

Call Center Operations is a broader term that encompasses the various metrics and operations related to managing a contact center. In the script, it is used in the context of a dashboard that provides an overview of key performance indicators, such as call volume and customer satisfaction, which are vital for optimizing call center performance.

💡Customer segmentation

Customer segmentation is the practice of dividing a customer base into groups based on specific characteristics, which can help in tailoring services or understanding customer behavior. The script mentions this concept in relation to the 'Customer and Topic Overview dashboard,' where it is used to analyze conversation topics by different customer segments.

💡Virtual Agent Performance

Virtual Agent Performance refers to the effectiveness and efficiency of an AI-driven virtual agent in handling customer inquiries. The script discusses a specific dashboard for monitoring this performance, which is crucial for assessing how well the virtual agent is serving customers and identifying areas for improvement.

💡Agent Performance

Agent Performance is a metric that measures the efficiency and effectiveness of live customer service agents. In the video, it is detailed through a dashboard that provides insights into various aspects such as waiting time, conversation handling, and customer satisfaction ratings, which are essential for optimizing human resource allocation and training.

Highlights

Monitoring different signals in the CCAI Insights console during the Monitor phase of the Optimization process.

Checking two dashboards in the Conversation hub: Conversations created and Conversation history.

Conversations created dashboard shows number of conversations per day to gauge traffic volume.

Conversation history dashboard displays high-level metrics like total conversations, average duration, turns per conversation, and top conversation topics.

Using filters to analyze conversations by primary topic, number of turns, and occurrence time.

Topic models dashboard shows topic distribution over time, helping identify spikes or less trafficked topics.

Conversation highlights dashboard to monitor the popularity of keywords or key phrases in conversational data.

Smart Highlights and Custom Highlights for tracking specific phrases over time with their counts.

Generative FAQ dashboard to monitor customer questions and how they are answered by the agent.

Monitoring call center operations metrics like calls, containment, duration, cost savings, and sentiment in Looker.

Overall KPIs dashboard provides an overview of key metrics for the last week.

Customer and Topic Overview dashboard for analyzing conversation topics by customer segmentation and volume over time.

Virtual Agent Performance dashboard for assessing virtual agent performance compared to live agents when integrated.

Agent Performance dashboard to monitor live agent metrics like waiting time, follow-ups, and CSAT.

Filtering and granular monitoring of metrics in dashboards by clicking on information or graphs.

Monitoring specific agent performance metrics like handled conversations, call duration, sentiment, and customer ratings.

Transcripts

Let’s double click in the Monitor phase of the Optimization process.

There are different signals you can monitor

in the CCAI Insights console: First, in the Conversation hub you can check

two dashboards: At the top, you’ll see conversations created.

This shows the number of conversations per day so you can quickly see how much traffic

is going through Insights.

Below it, you’ll see the Conversation history dashboard.

This shows high level metrics such as total number of conversations, their average duration,

average turns per conversation, as well as the top three conversation topics in your

data.

Additionally, you can use filters to get a list of conversations that share the same

primary topic and get a sense of how many turns they have or when they happened.

Then, the topic models dashboard shows topic distribution over time.

You can review this dashboard for spikes for some specific topics, or determine which topic

receives the most or the least amount of traffic.

Click on “View Legend” to open a sub-menu with a list of the identified topics.

By selecting one or more topics, you can highlight in the graph the specific topic you require

the distribution for.

The Conversation highlights dashboard lets you monitor how popular keywords or key phrases

are in your conversational data.

Check under Smart Highlights or Custom Highlights to view a graph that displays each tracked

phrase over time along with a list of all highlights and their count.

Additionally, in the Custom highlights dashboard, you can click on “View Legend” for a breakdown

of keywords and phrases that Insights is tracking.

Click on any of them to highlight their graph.

Lastly,let’s review the Generative FAQ dashboard, which is the last tab in the CCAI Insights

console.

This shows questions asked by customers and outlines how these questions are answered

by the agent.

This information can be used to monitor trending questions and give a general sense of how

popular each question is and how many turns are typically required to answer each of them.

There are different signals you can monitor in Looker.

Under the Call Center Operations dashboard, there are four areas to monitor different

metrics.

At the top is the Overall KPIs for the last week dashboard.

It shows overall metrics such as: The number of calls,

containment, call duration,

costs savings, and average sentiment for the entirety of

your data.

In our example here, the average call duration is 1.7 minutes and the general sentiment for

the contact center during the last week is neutral.

This example also shows a 73.5 percent call containment.

However this will only apply if Looker is connected to a virtual agent.

Next, is the Customer and Topic Overview dashboard.

Here, you can review conversation topics by: customer segmentation,

topic volume over time, overall conversation volume per day,

or percentage of repeat callers.

For example, the Calls by customer Segmentation view shows that opening a new account is more

popular among retirees in rural areas.

And the conversation Volume view shows that traffic volume is lower on Mondays.

The Virtual Agent Performance dashboard is only applicable if Insights is integrated

with a virtual agent.

If you perform this integration, you can use this dashboard to monitor where your virtual

agent performs well on its own, or in comparison to live agents.

For example, we can see that opening a new account drives more satisfaction if performed

with a live agent.

Lastly, the Agent Performance dashboard refers to your live agents.

In this last dashboard you can monitor waiting time to speak to a live agent by topic, identify

which topics typically require a follow up, and review a break down of your live agents

where you can see metrics like: their average CSAT,

The number of conversations handled, And the average waiting time.

For example, we can see here that online banking issues don’t require as many follow ups

as submitting a fraud alert.

Additionally, at any time, you can click on any piece of information in any dashboard

or graph.

This automatically applies a filter and the whole Call Center Operations page updates

to show all metrics by the filter applied.

This way, you can monitor all these metrics in a more granular way.

Under the Agent Performance dashboard, you can select a specific agent, exclude an agent,

or use other filters to get a view of their performance.

You can monitor metrics like: The number of conversations that were handled

in the last week, average call duration,

speaker distribution, average sentiment,

topics handled by the agent over time, conversation duration by topic over time,

and customer ratings.

For example, in this view we can see that Tonya Koop didn’t handle any conversations

last week and that customers that speak with her have a neutral or positive satisfaction

rating.