DQ-View: The Five Stages of Data Quality

Data Quality (DQ) View is an OCDQ regular segment. Each DQ-View is a brief video discussion of a data quality key concept.

In my experience, all organizations cycle through five stages while coming to terms with the daunting challenges of data quality, which are somewhat similar to The Five Stages of Grief.  So, in this short video, I explain The Five Stages of Data Quality:

1. Denial — Our organization is well-managed and highly profitable.  We consistently meet, or exceed, our business goals.  We obviously understand the importance of high-quality data.  Data quality issues can’t possibly be happening to us.
2. Anger — We’re now in the midst of a financial reporting scandal, and facing considerable fines in the wake of a regulatory compliance failure.  How can this be happening to us?  Why do we have data quality issues?  Who is to blame for this?
3. Bargaining — Okay, we may have just overreacted a little bit.  We’ll purchase a data quality tool, approve a data cleansing project, implement defect prevention, and initiate data governance.  That will fix all of our data quality issues — right?
4. Depression — Why, oh why, do we keep having data quality issues?  Why does this keep happening to us?  Maybe we should just give up, accept our doomed fate, and not bother doing anything at all about data quality and data governance.
5. Acceptance — We can’t fight the truth anymore.  We accept that we have to do the hard daily work of continuously improving our data quality and continuously implementing our data governance principles, policies, and procedures.

If you are having trouble viewing this video, then you can watch it on Vimeo by clicking on this link: DQ-View on Vimeo

You can also watch a regularly updated page of my videos by clicking on this link: OCDQ Videos

Related Posts

Posts related to the Denial Stage of Data Quality:

Data Quality and Chicken Little Syndrome

The Illusion-of-Quality Effect

Perception Filters and Data Quality

“Some is not a number and soon is not a time”

The Data Quality Wager

Posts related to the Anger Stage of Data Quality:

Jack Bauer and Enforcing Data Governance Policies

Beware the Data Governance Ides of March

Aristotle, Data Governance, and Lead Rulers

Why isn’t our data quality worse?

Don’t Do Less Bad; Do Better Good

Posts related to the Bargaining Stage of Data Quality:

Data Quality and Miracle Exceptions

Do you believe in Magic (Quadrants)?

Which came first, the Data Quality Tool or the Business Need?

The Technology Carousel

The Stakeholder’s Dilemma

Posts related to the Depression Stage of Data Quality:

There is No Such Thing as a Root Cause

The Dichotomy Paradox, Data Quality and Zero Defects

The Asymptote of Data Quality

To Our Data Perfectionists

Data Quality and the Bystander Effect

Posts related to the Acceptance Stage of Data Quality:

You only get a Return from something you actually Invest in

Data Governance Frameworks are like Jigsaw Puzzles

The HedgeFoxian Hypothesis

Finding Data Quality

Data Quality: Quo Vadimus?

Tweet

Comments

[Adrian]

Hi Jim,

I really enjoyed this. It rings true with my experience as a consultant as well. I had also wondered if there was a link between the Kübler-Ross model and data quality - you beat me to it!

Thanks again,

Adrian

[Jim Harris]

From the LinkedIn Group for the IAIDQ Professional Open Community, Andrew Dean commented:

“For how many years has Data Quality been a recognized business discipline? Quite a few right?

Yet we still have to battle hard to make ourselves heard. Why is this? Because it's very hard to get out of the Denial Stage.”

And I responded:

Thanks for your comment, Andrew.

Yes, whenever people ask me why it's so hard to get people to recognize the need for data quality and why, as you said, it should be seen as a business discipline, I respond by explaining that many organizations are in the Denial Stage, during which it can be nearly impossible to get your data quality message heard.

For all the (valid and necessary) talk about being proactive with data quality, the harsh reality is that many organizations don't start playing attention to data quality until the Anger Stage, which also has the unfortunate side effect of keeping organizations stuck in a reactive cycle of only data cleansing whenever a crisis happens (again), without ever implementing defect prevention and data governance.

Best Regards,

Jim

[Clarke Patterson]

Good points, Jim. Any thoughts on how may orgs actually make it to the "Acceptance" stage?

[Jim Harris]

@Clarke — Unfortunately, like many of life’s lessons, this can only be learned, not taught. Organizations have to experience for themselves what Data Quality Denial, Anger, Bargaining, and Depression feels like within their unique corporate culture, before they will make it to the Acceptance stage. And, of course, none of these stages are static, not even Acceptance because after awhile complacency can set in and the organization will find itself right back in the Denial stage all over again.