Dark Data – A Potential Gold Mine for Customer Support

All our activities, online and offline, leave an almost imperceptible electronic trail. Traffic cameras “see” you cross a junction around the same time each weekday, your phone lets your favorite coffee shop know when and how long you visit through their “free” Wi-Fi. The data collected by these devices or from our interactions with others tend to end up in a data archive somewhere, unused and unanalyzed. Gartner defines dark data as the information assets organizations collect, process, and store during regular business activities, but generally, fail to use for other purposes.

Now think about the treasure trove of interaction data that is collected and stored during the resolution of a customer support ticket. Who did what, how long did it take, which “experts” were consulted, or the insights hidden in attachments and ticket notes– all these are usually forgotten when the ticket is closed, and are reduced to a single-digit satisfaction rating! But with machine learning and unlimited computing power available at our fingertips, we should do better.

dark data

Where does Dark Data Come From and What is it Made Up of?

A lot of dark data is data that businesses don’t realize that they are collecting and storing. Most of this data remains unstructured and unanalyzed.

There are usually two kinds of dark data. Take an email or a chat message for example. The message content can easily turn dark if we don’t extract meaning from the text (natural language) in a way that a computer can analyze it and recall when needed. Secondly the metadata for the email or chat message –  the time of day it was sent, who sent it, who received it, the device or application used to send it and attachment details like content or location if any – go dark when messages are archived. While all of this data exists in siloed databases and repositories, not much is done to derive any insight from this data. It’s stored so it can be retrieved, if needed, in the future.

In today’s digital world where every interaction and transaction gets captured in a system, extracting insights and analyzing the data all the time feels like drinking from the firehose.

Service desk and support teams always operate with a sense of “organized chaos” while they work through a typical day of incidents and requests. There’s hardly any time to step back and figure out what’s really going on and why. The knowledge documentation, triage or troubleshooting decision trees are almost never updated or expanded. The data, however, is dark, and not in a state that can be useful to agents when a similar situation recurs.

Machine Learning and AI to the Rescue

What if there was a way for companies to illuminate these untapped sources to derive and deliver insights that lead to better experiences and smarter decisions across the board? Recent advances in machine learning, computer vision, pattern recognition, natural language processing, and generation are making this possible.

We can now apply AI in our day-to-day processes to tag data, add labels, classify information and accurately summarize it before it gets stored or filed away in an archive. This prevents the data from getting dark, and helps machine learning get better in the future — helping you predict which customer is likely to contact you on which channel and when, or who your best support agent is for a particular type of incident and maybe even easily find a needle in the proverbial data haystack when you need it.

For example, in a help desk system like Freshdesk, you could quite easily train a machine learning model to suggest what labels and categorization to apply to an incoming ticket, based on how similar tickets have been handled in the past. You could even go a step further to extract the sentiment of the author from the description in the ticket to determine the urgency of the issue and alert the support agent. We could even use the meta data in the tickets, the text in the descriptions and notes, information about the source of the interaction, and historical data about who solved such issues, to determine which agent can solve what kind of incidents in the future. You could even encourage collaboration amongst the agents by assigning credit to agents who help other agents resolve issues.

There are apps like FreshEngage for Freshdesk allows you to create company-specific machine learning models based on tickets that have been solved in the past, so we can predict tags and categories, and recommend issue resolutions for incoming tickets automatically. You can also automatically assign tickets to the most competent person for the issue, and highlight similar issues that have been resolved recently. The same machine learning model can power smarter, more resilient automation such as bots and most of all seamlessly augment the capabilities of your workforce.

Don’t be Afraid of the Dark

Dark data can reveal hidden linkages (weak & strong) that tie various data points together to reveal interesting causal relationships. Connecting these dots with machine learning and AI can help you create a unique organizational knowledge graph that can power all your interaction channels (voice, chat & video). In the case of customer support, having the relevant context, or knowing who the best agent for a particular type of issue is, can significantly reduce resolution times and improve service quality.

With advances in technology, businesses can not only churn large volumes of transactional data, but also apply cutting edge artificial intelligence technology to discover previously unknown relationships.

Dark Data is a potential gold mine of insights, causal relationships, and other hidden patterns, just waiting to be discovered!