Are You Afraid Of Your Data Quality Solution?
Jim HarrisAs a data quality consultant, when I begin an engagement with a new client, I ask many questions. I seek an understanding of the current environment from both the business and technical perspectives. Some of the common topics I cover are what data quality solutions have been attempted previously, how successful were they and are they still in use today. To their credit, I find that many of my clients have successfully implemented data quality solutions that are still in use.
However, this revelation frequently leads to some form of the following dialogue:
OCDQ: "Am I here to help with the enhancements for the next iteration of the project?"
Client: "No, we don't want to enhance our existing solution, we want you to build us a brand new one."
OCDQ: "I thought you had successfully implemented a data quality solution. Is that not true?"
Client: "We believe the current solution is working as intended. It appears to handle many of our data quality issues."
OCDQ: "How long have you been using the current solution?"
Client: "Five years."
OCDQ: "You haven't made any changes in five years? Haven't there been requests for bug fixes and enhancements?"
Client: "Yes, of course. However, we didn't want to make any modifications because we were afraid we would break it."
OCDQ: "Who created the current solution? Didn't they provide documentation, training and knowledge transfer?"
Client: "A previous consultant created it. He provided some documentation and training, but only on how to run it."
A common data quality adage is:
"If you can't measure it, then you can't manage it."
A far more important data quality adage is:
"If you don't know how to maintain it, then you shouldn't implement it."
There are many important considerations when planning a data quality initiative. One of the most common mistakes is the unrealistic perspective that data quality problems can be permanently “fixed" by implementing a one-time "solution" that doesn't require ongoing improvements. This flawed perspective leads many organizations to invest in powerful software and expert consultants, believing that:
"If they build it, data quality will come."
However, data quality is not a field of dreams - and I know because I actually live in Iowa.
The reality is data quality initiatives can only be successful when they follow these very simple and time-tested instructions:
Measure, Improve, Repeat.
Reader Comments (4)
Jim, I like your OCDQ/Client dialogue.
“If it ain’t broke, don’t fix it” is of course not only limited to data quality implementations – but perhaps it is a common situation around with data quality solutions such as:
• A lot of solutions are “home made” scripts and other stuff.
• There are many tool vendors but each with only a limited installed base often concentrated on a single or few geographical markets – even the Quadrant Leaders DQ solutions are exotic in most markets.
I guess we will have a situation like this also in the years ahead.
The dialog you created sounds very familiar to many customer data integration projects. Data Quality/Customer Recognition is a journey not a destination. With the average rate of consumer data decay around 2%, within a year almost 25% records you have in your system are outdated. A process of ensuring fresh, clean data is utilized in your data quality solution is critical.
Were you that bird sitting on my window ledge 2 years ago? This gave me the chills! I had the same discussions with vendors while implementing a data quality practice in my last company.
Anyway, my 2 cents:
(1) Data quality and the business rules governing it always change.
(2) Data Quality is not a solution, it is a practice.
Michele,
Yes – I was using the beta version of twitter-on-the-ledge :-)
Thanks for your 2 cents – I wholeheartedly agree with both of your points!
Best Regards…
Jim