Dark data, Dirty data

Written by
Mark Bands

Partner

Published on

14 April 2022

Under the category

CLM Experts, Let's Talk

Does your organisation know the value hidden in dark data and, equally, the risks lurking in your dirty data?

It’s the 23rd March 2003 and the Academy Award for Best Animated Feature has just been won by an animated Japanese fantasy film, Spirited Away. In the imaginary story there are small, unfamiliar characters called Makkuro Kurosuke. In the English dub of the film, they are referred to as ‘Dust Bunnies’.  These elusive, sooty creatures are tennis ball-sized, pitch-black, and fuzzy-haired beings with two large eyes and long, thin limbs. They move by hovering around and can extend stick-like limbs from their bodies to carry out specific tasks or lift ultra-heavy objects. 

Hold that thought, park the fantastical imagery - and the 2003 nostalgia - and let's shift our focus for a moment. Looking into the domain of organisational data management, we note that in recent years two prevalent and important concepts have arisen relating to what are called ‘dark’ data and ‘dirty’ data. 

Dark data is defined as the data that organisations collect, process and store as an outcome of business-as-usual activities, but fail to identify, categorise, or even use for actionable business insights.  Conversely to this, dirty data is data that you already know you have, but that is clearly incomplete, incorrect, contains unreadable content or is duplicated in multiple data stores, in multiple forms. In short, the kind of data you don’t really want.

Let’s revert back to Spirited Away for a minuteAs the narrative events of the story unfold, the lead character Chihiro befriends several dust bunnies after learning that if they are not given a job to do, they turn back into soot. It can only be analogous then that like the dust bunnies, dark data should be given a job to do and dirty data should be fixed, and fixed fast. 

While it is logistically expensive to store large amounts of dark data, lost revenue opportunity is by far the larger issue.  Experience has proven that if this data is analysed, categorised, and mined, it contains hidden treasures in the form of insights that can be turned into actionable and revenue generating business initiatives.  

Dirty data is staggeringly risky for a financial services organisation to use in day-to-day operations because it inherently creates transactional, operational and compliance risks. Research has indicated that inaccuracies could be costing firms between 15% and 25% of annual revenue. With global banking revenues of over US$2.2 trillion, this means that dirty data may be costing the global banking sector over US$400 billion a year. Enough said, right?

In our Client Lifecycle Management (CLM) advisory work, Aurora SDE often encounters situations where the CLM journey can be markedly improved, both tactically and strategically, by understanding and then utilising dark data and/or fixing dirty data.  With this pursuit of happier banking customers in mind plus increased shareholder revenues, better utilised staff effort and a massive reduction in compliance and regulatory risk, we strongly recommend an immediate focus on your neglected data dust bunnies.     

We also really recommend watching Spirited Away if you haven’t already seen it.  

Keep Reading

Aurora Live »
Continue Reading »
Aurora Logo
Aurora SDE

18 November 2022

CLM 2040

Aurora’s Business Analysts Alexander Arezina and Cerys Stone explore what the future could have in store for the onboarding process - and if it's even viable to be making big predictions at all.

Continue Reading »
Alexander Arezina

Business Analyst

Cerys Stone

Business Analyst
ESG Ambassador

8 November 2022

Why your understanding of CLM is probably wrong.

Noticing a huge disparity between what CLM actually is and what most organisations think it is, our CEO Sean Vickers decided it was time to do some explaining.

Continue Reading »
Sean Vickers
Sean Vickers

Chief Executive Officer

8 November 2022

Still haven't
found what you're
looking for?

Get in Touch

Be the early bird and get in touch today if you have any questions, thoughts or ideas. We’d love to hear from you.

Book an appointment today

Go to Calendly