“Great pep talk Jim. We all need to be realistic when striving for excellence in data quality.

However, if we don't aim for perfection we will never know what is possible. I am always asked about measurement when it comes to data quality and constantly see some people expecting 100% accuracy while others being prepared to accept double digit percentages when it comes to error rates.

These sorts of beliefs or attitudes reflect the culture of an organisation as to what expectations exist and also what the level of commitment to quality is in general. Take a barometer of the quality of information in an organisation and you will get a good feel for the attitudes and standards in the organisation or lack thereof.

As data quality practitioners we need to constantly strive for excellence, accept that imperfection will exist and persist, and find remarkable ways to constantly deliver quality outcomes.”

“I know this has aged a bit since the last post and find this an interesting discussion. I would like to add another dimension to this discussion. I fully agree that the objective of managing data quality and data governance is to make more informed decisions. Speaking from experience in providing litigation support, SEC investigations, and other critical problem solving, the unfortunate thing is that we don't necessarily know the decisions that we will have to make in 6, 9, 12... months.

As such, it’s important to focus on quality issues that impact core processes and reporting, but also important to understand the overall health of the organization’s data. That way, if you need to make an unexpected critical decision, while you may not have perfect data, you will understand the general health of the data. Understanding this general health can help you better analyze it and understand its potential short comings.

Another driver that I have seen is the industry that the organization operates in. Highly regulated industries (healthcare, biotechnology, financial services...) may require you to do more management of data to manage data. This piggy backs to the above paragraph, because the only time you may need some of this data is when you could be facing regulatory inquiry.

All in all, I think a solid Enterprise Data/Information Management program should address many of these key points. As noted above and in other threads, the goal is not to manage data to manage it, leadership (could be a whole new thread on Chief Data Officers) has to find the appropriate balance of data governance, master data management, business intelligence, data quality, data security... that is cost effective and delivers the desired results. This is not an easy feat and there is no "silver bullet" approach. Understanding and managing stakeholder expectations and regulatory requirements should help set the course. Once the foundation is developed, augmenting it across the organization is also a considerable challenge.”

Yes, it does certainly seem that most organizations focus on building a fantastic concert hall, resplendent with the finest musical instruments money can buy, believing, perhaps, that if they simply build it, then beautiful music is sure to be heard.

It’s as if when reminded about the importance of the music itself (the data and decision sounding together in harmony), they respond like Radar O’Reilly from M*A*S*H:

“Ah, Bach!”

:-)

Best Regards,

Jim

Being a lover of both music and data, it struck all the right notes!

I think the analogy is a very good one - when I think about data as music, I think about a companies business intelligence architecture as being a bit like a very good concert hall, stage, and instruments. All very lovely to listen to music - but without the score itself (the data), there is nothing to play.

And while certainly a real live concert hall is fantastic for enjoying Bach, I'm enjoying some Bach right now on my laptop - and the MUSIC is really the key.

Companies very often focus on building fantastic concert halls (made with all the best biggest data warehouse appliances, ETL servers, web servers, visualization tools, portals, etc.) but forget that the point was to make that decision - and base it on data from the real world. Focusing on the quality of your data, and on the decision at hand, can often let you make wonderful music - and if your budget or schedule doesn't allow for a concert hall, you might be able to get there regardless.

“Good topic Jim, too many people have never actually measured the quality of their data and that forces them to use the mushy numbers. It seems that they would be happier, or at least more conscious, if they could begin with one solid DQ metric that they can begin to manage to and grow their organizations IQQ around.”

And I responded:

Yes, mushy numbers are as useless as data quality metrics that do not provide insight into how data is being used to make business decisions.

In my blog post The Winning Curve, which also referenced books by the Heath brothers and Seth Godin (hmmm, I am detecting a pattern), I wrote about how setting specific times when you will meet (or exceed) your goals is so difficult because most people view that delivery date as something that forebodingly looms on the calendar.

The delivery date is when your definition of success will be judged by others, which is why some people prefer the term Judgment Day since it seems far more appropriate.

“The only purpose of starting,” writes Godin, “is to finish, and while the projects we do are never really finished, they must ship.” Godin explains that the primary challenge to shipping (i.e., accomplishing your goal by or before your delivery date) is thrashing.

“Thrashing is the apparently productive brainstorming and tweaking we do for a project as it develops. Thrashing is essential. The question is: when to thrash? Professional creators thrash early. The closer the project gets to completion, the fewer people see it and the fewer changes are permitted.”

Thrashing is mostly about the pursuit of perfection.

We believe that if what we deliver isn’t perfect, then our efforts will be judged a failure. Of course, we know that perfection is impossible. However, our fear of failure is often based on our false belief that perfection was the actual expectation of others.

Therefore, our fear of failure offers this simple and comforting advice: if you don’t deliver, then you can’t fail.

Success or failure—or even worse, mediocrity—could be the judgment that you receive after you have delivered.

Success rocks and failure sucks—but only if you don’t learn from it.

Being remarkable means always delivering on the promises of your specified goals at your specified times.

It's something that drives me crazy!

And not just how this relates to data quality, but success measures and poor communication in general.

If you ask someone how the task is coming and their response is: "it's coming", or how much time do we have to complete this and you hear: "yesterday". If (ok, when) someone says that to me I respond with: "how many days does 'it's coming' mean"?, or "what is the drop dead date for 'yesterday' and what happens if it isn't met"?

I really bugs me that in instances such as these, and they happen all the time, that I have to waste my precious time trying to extract exact, honest and accurate information from people who don't even know what it means.

GRRRR...

Thanks once again for astute insight.

Jill

It's easy to imagine wanting to be perfect in healthcare since, after all, anything other than perfection (i.e., zero defects) would mean that patients are dying needless deaths — even one defect a year can be heartrendingly intolerable. But this is also why emergency medical situations use triage techniques — because attempting to save everyone means that you could end up saving no one.

It's far easier in healthcare to sleep at night by telling yourself that “our hospital doesn't have a defect rate.”

It's far easier to deny a problem exists, than to challenge yourself to battle against it, especially knowing that, despite your best intentions, sometimes you will lose.

But you must also be prepared to celebrate your successes.

Here are the closing words of Dr. Berwick's speech back on December 14, 2004, when he issued the challenge, and predicted how the world would look when they achieved their goal of saving 100,000 lives:

“And, we will celebrate. We will celebrate the importance of what we have undertaken to do, the courage of honesty, the joy of companionship, the cleverness of a field operation, and the results we will achieve.

We will celebrate ourselves, because the patients whose lives we save cannot join us, because their names can never be known. Our contribution will be what did not happen to them.

And, though they are unknown, we will know that mothers and fathers are at graduations and weddings they would have missed, and that grandchildren will know grandparents they might never have known, and holidays will be taken, and work completed, and books read, and symphonies heard, and gardens tended that, without our work, would have been only beds of weeds.”