The big data revolution has meant that there are nuggets of insight within customer data everywhere: CRM data, reviews, complaints, enquiries, surveys, social media etc. The ability to harvest and analyse these data in an automated way to provide predictive, actionable insight is a holy grail for marketers and customer experience professionals.

However, across all organisations, ever-expanding amounts of data remain unanalysed, primarily due to their growing size and complexity. And of course, the mass of these data is unstructured or in their raw form. Unstructured data such as text, image, audio, video, machine and sensor information all present major issues for organisations and the data scientists they employ. It is estimated that between over 90% of data in existence today is unstructured. However, organisations are just not able to access this insight. Some commentators estimate that less than 0.5% of data is currently analysed.

The current state of the art  

The existing state of the art when it comes to extracting insight from VoC data is using a combination of text analytics, predictive analytics packages and significant manual input from data scientists.

But firms are getting bogged down, spending too much time and resources cleansing and transforming data. CrowdFlower estimates “data preparation” at 80%. Further, assumptions and errors are an inevitable part of the process where human judgement and skill is required.

How technology is evolving to extract greater insight

Imagine if a system could analyse every single piece of verbatim from surveys, reviews and complaints with high degree of accuracy and automatically transform it, contextualise and classify it accurately not matter how complex? Even better, what if when the system’s predictive performance dropped, it would ‘ask’ the user to clarify in a highly efficient and minimalistic way? Sounds like science fiction. The good news is that it’s science fact as machine learning is now sophisticated enough to achieve this.

The secret behind the latest advancements is a new machine learning technology called Optimised Learning. Optimised Learning is a speedy way to automate the classification of complex textual data that crucially ‘asks’ users for specific input for where it needs it to optimise performance. As a result, it can automatically classify VoC data to a high degree of accuracy with minimal user input.

Optimised Learning eliminates this huge amount of human intervention and increases accuracy as it quickly starts to ‘ask’ a user, who doesn’t need to be a data scientist at all, for minimum input.

It’s this amalgamation of automation and promoted human intervention that is creating huge opportunities for organisations looking to gain more insight and utilise more of the data they accumulate.

How and where are new innovations being used?

The truly innovative enterprises are already piloting and implementing these latest innovations in machine learning. Having early warning allows innovative companies to innovate further, and thereby respond quickly to their new service or product launches.

For example, one of the world’s largest airlines sends out tens of thousands of surveys every month to customers of all of their services. They analysed the structured data such as Customer Satisfaction (CSat) scores per touchpoint on the customer journey (cabin cleanliness, check in, meals etc.). However, despite having the state of the art text mining technology they struggled to analyse the free text in a systematic way and really only used verbatims to follow up the structured findings in discrete projects which involved highly-skilled analysts spending a lot of time.

By generating automated predictive insight from survey data, the airline was able to not only pick up the structured data such as multiple choice answers and CSat scores, but also automatically generate the reasons why their customers were happy or unhappy and get some indication on what the predictive factors were such as routes, demographics or customer segments. All in near real time.

In another example, a leading manufacturer of home appliances is also using the latest in automated predictive analytics to automate the analysis of their customer interactions and ‘Voice of Customer’ (VoC) data. Since using Optimised Learning, they are able to launch and update products faster, enhance customer experience throughout the lifetime of the product, whilst also getting key insights from customers to support product improvements and new product development.

It has helped the company to move from a ‘replace and fix’ model to proactively alerting customers and sending parts when they can fix a product themselves.

They have improved business performance through better issue classification: enabling lower warranty costs, fewer repeat issues and the management of customer experience in a more systematic way - improving customer satisfaction and reducing costs simultaneously.


By Dan Somers, CEO of Warwick Analytics

GDPR Summit Series is a global series of GDPR events which will help marketers to prepare to meet the requirements of the GDPR ahead of May 2018 and beyond. Further information and conference details are available at

comments powered by Disqus