Microwavable Data Quality

Data quality is definitely not a one-time project, but instead requires a sustained program of enterprise-wide best practices that are best implemented within a data governance framework that “bakes in” defect prevention, data quality monitoring, and near real-time standardization and matching services—all ensuring high quality data is available to support daily business decisions.

However, implementing a data governance program is an evolutionary process requiring time and patience.

Baking and cooking also require time and patience.  Microwavable meals can be an occasional welcome convenience, and if you are anything like me (my condolences) and you can’t bake or cook, then microwavable meals can be an absolute necessity.

Data cleansing can also be an occasional (not necessarily welcome) convenience, or a relative necessity (i.e., a “necessary evil”).

Last year on Data Quality Pro, Dylan Jones hosted a great debate on the necessity of data cleansing, which is well worth reading, especially since the over 25 (and continuing) comments it received proves it is a polarizing topic for the data quality profession.

I reheated this debate (using the Data Quality Microwave, of course) earlier this year with my A Tale of Two Q’s blog post, which also received many commendable comments (but far less than Dylan’s blog post—not that I am counting or anything).

Similarly, a heated debate can be had over the health implications of the microwave.  Eating too many microwavable meals is certainly not healthy, but I have many friends and family who would argue quite strongly for either side of this “food fight.”

Both of these great debates can be as deeply polarizing as Pepsi vs. Coke and Soccer vs. Football.  Just for the official record, I am firmly for both Pepsi and Football—and by Football, I mean NFL Football—and firmly against both Coke and Soccer. 

Just as I advocate that everyone (myself included) should learn how to cook, but still accept the eternal reality of the microwave, I definitely advocate the implementation of a data governance program, but I also accept the eternal reality of data cleansing.   

However, my lawyers have advised me to report that beta testing for an actual Data Quality Microwave has not been promising.

Related Posts

A Tale of Two Q’s

Hyperactive Data Quality (Second Edition)

The General Theory of Data Quality

Follow OCDQ

If you enjoyed this blog post, then please subscribe to OCDQ via my RSS feed, my E-mail updates, or Google Reader.

You can also follow OCDQ on Twitter, fan the Facebook page for OCDQ, and connect with me on LinkedIn.

Comments

[Dylan Jones]

Thanks for posting a link to Data Quality Pro, Jim.

Totally agree with you that data cleansing has been by far the most polarising topic featured on our site since the launch. Like you, I agree that data governance is a marathon not a sprint but I do object to a lot of the data cleansing bashing that goes on.

I think that sometimes we should give people who purchase cleansing software far more credit than many of the detractors would be willing to offer. In the vast majority of cases data cleansing does provide a positive ROI and whilst some could argue it creates a cost base within the organisation it is still a step in the direction of data quality maturity.

I think this particular debate is going to run and run however so thanks for fanning the flames over on OCDQ Blog!

[Henrik Liliendahl Sørensen]

Well Jim, there are things we agree about and things we don’t agree about.

With football (called soccer in the US) versus American football (called football in the US) I am looking forward to the real Football World Cup in about two months in South Africa. The result may be found in my 2010 predictions.

Regarding Pepsi versus Coke, I couldn’t care less.

As for microwave cooking I agree. I love to prepare a good meal with the best raw materials, care and time needed. But some days our family is too busy with a lot of other exciting things and then the microwave oven is a fast and reliable work around.

Some of the exiting things that are keeping me busy is exactly the eternal reality of data cleansing – besides of course the best of bread future of data quality prevention.

[Peter Thomas]

Hi Jim,

The question is not [American] Football versus [everyone else] Football, but football versus a proper sport like rugby - or a religion like cricket.

On the last worth reading Playing Hard Ball by Ed Smith, an English cricketer and now a sports journalist who recounts the similarities and differences between baseball and cricket against the background of the subway series of 1998 (that's subway the transportation system, not the sandwich - probably had to point that out in the context of this article).

Also worth noting that if you are British and refer to Association Football as Soccer, it tends to suggest that you went to what is known as Public School in our rather curious lexicon (i.e., what would be called Private School in the colonies).

I agree that data cleansing can lead to as many entrenched opinions as sporting disagreements. For me anything done to "fix" data perpetuates the problem. Highlighting the issues provides motivation to address them. I do however fully understand that I am unlikely to "convert" anyone with the opposite perspective.

Peter

[Phil Simon]

Jim,

Good post.

The problem is that many people don't realize that overuse of the microwave is bad. What's more, the ingredients and food often spoil at time before the guests come over due to lack of refrigeration and proper storage.

Too metaphorical?

ps

[Jim Harris]

@Dylan I definitely agree with your many excellent points, especially that data cleansing can provide both a positive ROI and a step in the direction of data quality maturity.

@Henrik What good is a World Cup if it isn't filled with some form of caffeinated sugar water that is as bad for you as microwavable food? :-)

@Peter I should probably read Ed Smith's book because I have absolutely no idea what cricket is all about - at least I understand soccer - I don't like it, but I understand it. And I see that you are a "preventer" and not a "fixer" when it comes to data quality issues. At least that too is more understandable than cricket :-)

@Phil Way too metaphorical, you could've at least used a Rush metaphor :-)

[Kit Hamilton]

Microwavable data quality. Now that is an interesting concept. On the one hand, it implies cooking food for which you don’t have high expectations for in regard to taste. On the other hand, it also connotes ease, convenience, speed.

So let’s explore this.

Low quality food (or typical “microwave” fare)

Most great chefs cringe at the notion of a microwave. And no wonder. They take great pride in their work. Their food (the quality, the presentation) is their greatest asset. I would argue that in terms of an entity’s data, any leadership team who does not feel that their data is one of their greatest assets does so at their own peril. What could be more valuable? So for himself, a great chef might toss a bag of popcorn in the microwave for expediency sake. However, I suspect an expensive cut of meat or a signature dish intended for customers would never see the inside of a microwave oven. So why would an enterprise treat their most valuable asset any differently?

Ease. Convenience. Speed.

These are concepts we particularly hold dear in terms of data quality. I think that if an enterprise has the notion that data quality is hard or inconvenient or not something that can be done quickly, they could well be using the wrong cookbook. For example, data quality solutions can be built on SOA principles, utilizing web services that make them easy to install and use. Some solutions even enable a data steward to publish business rules in one click.

The bottom line is that you don’t have to skimp on taste to get a good meal quickly and easily.

Forget the microwave. Find a better recipe.