Data Quality is not an Act, it is a Habit
Jim HarrisThe Second Law of Data Quality states that it is not a one-time project, but a sustained program. Or to paraphrase Aristotle:
“Data Quality is not an Act, it is a Habit.”
Habits are learned behaviors, which can become automatic after enough repetition. Habits can also be either good or bad.
Sometimes we can become so focused on developing new good habits that we forget about our current good habits. Other times we can become so focused on eliminating all of our bad habits that we lose ourselves in the quest for perfection.
This is why Aristotle was also an advocate of the Golden Mean, which is usually simplified into the sage advice:
“Moderation in all things.”
While helping our organization develop good habits for ensuring high quality data, we often use the term Best Practice.
Although data quality is a practice, it’s one we get better at as long as we continue practicing. Quite often I have observed the bad habit of establishing, but never revisiting, best practices.
However, as our organization, and the business uses for our data, continues to evolve, so must our data quality practice.
Therefore, data quality is not an act, but it’s also not a best practice. It’s a habit of continuous practice, continuous improvement, continuous learning, and continuous adaptation to continuous change—which is truly the best possible habit we can develop.
Data Quality is a Best Habit.
Reader Comments (7)
Jim,
This is a great post and immediately reminded me of the practice of Kaizen in the manufacturing industry. The idea being that continued small improvements yield large improvements in productivity when compounded.
For years now many of the thought leaders have preached that projects from BI to data quality to MDM to data governance, and so on, start small and that by starting small and focused they will yield larger benefits when all of the small projects are compounded. But the one thing that I have not seen it tied back to is the successes that were found in the leaders of the various industries that have adopted the Kaizen philosophy.
Data quality practitioners need to recognize that their success lies in the fundamentals of Kaizen: quality, effort, participation, willingness to change, and communication. The fundamentals put people and process before technology. In other words, technology may help eliminate the problem but it is the people and process that allow that elimination to occur.
Well written thoughts Jim.
Mentioning the “golden mean” makes me think about the terms “golden copy” and “golden record” which are often used terms in data quality improvement and master data management.
In using these terms I think we mostly are aiming on achieving extreme uniqueness. But we should rather go for symmetry, proportion, and harmony.
I didn't know about the Golden Mean. I learn a good amount reading this blog.
I have very little to add here. Great post and a solid comment by Rob that sums up my thoughts better than I could.
Subtle but immensely important because implementing a coordinated series of small, easily trained habits can add up to a comprehensive data quality program.
In my first data quality role we identified about 10 core habits that everyone in the team should adopt and the results were astounding. No need for big programs, expensive technology, change management and endless communication, just simple, achievable habits that importantly were focused on the workers.
To make habits work they need the "WIIFM" (What's In It For Me) factor.
Great post as ever Jim.
Hmmm . . . define "after enough repetition".
Best habits are those that show up on specials occasions, you know what I mean . . . On a daily basis, well, you know . . .
There is an ironic French research that discovered a huge positive effect in the number of washed hands in public bathrooms when someone else was present.
The world was simpler in Aristotle Era. We are now about to live in a "zero-privacy" kind of society where data is everything. Soon (very soon) each of us will be rated by our personal data quality indicator over the institutions which we interact with.
Data quality should be mandatory, but as with environmental questions, it will have to be imposed, reinforced, and permanently - public individually and independently - controlled. As with all other best habits and reputations and with similar results. A huge and long and expensive decision. Repetition will not be enough.
Thanks for your great comments, Rob, Henrik, Phil, Dylan, and Blog reader from Brazil.
Your feedback is always appreciated.
@Rob — Excellent points, especially about how technology helps eliminate problems, but it is the people and process that allow that elimination to occur.
I also agree that there is much to be learned from quality principles such as Kaizen, but I am always concerned with the attempt to apply the concept of Zero Defects, which although definitely applicable for Manufacturing Quality, it's an unrealistic goal for Data Quality, where such an absolute is impractical, distracts from the focus on delivering business value, and can be demoralizing to the team working on making improvements (i.e., it makes every defect seem like a failure undoing all the good work they have done and need to continue to do).
@Henrik — Yes, I was tempted to bring those other “golden concepts” into this post, and in fact, the original draft included the perceived best practice of achieving a Single Version of the Truth.
I eliminated it during my final round of edits only because I could not succinctly discuss it without going off on a tangent, but I think your remarks are succinct and powerful: symmetry, proportion, and harmony instead of extreme uniqueness.
@Phil — Yes, you can learn a good amount reading this blog, especially if you read the comments from my readers, which greatly improve the quality of my blog posts.
@Dylan — Yes, indeed. Implementing a coordinated series of small, easily trained, simple, achievable and repeatable habits is the foundation of a comprehensive and continuous data quality program.
@Blog reader from Brazil — Yes, repetition will not be enough, especially if we fall into the trap of the best practice, of believing in a static list of tasks to always perform without revisiting and refining those tasks over time in response to change being the only universal constant. Therefore, the habit we are repeating is to repeatedly improve our habits, which I know is a bit of circular semantics. Rapid evolution is perhaps a better metaphor.
From the LinkedIn Group for TDWI, Charles de Jager commented:
“Data Quality is one of the pillars of the Enterprise Information Management practice. The issue I have is that most people believe you can throw a tool at the problem and Presto!—All the problems are sorted out.
Data Governance is the missing component in any organization. There are many case studies that have been done with very large organizations that show the resulting financial losses due to poor data quality.
Many man hours and several large sums of money continue to be spent on implementing ERP solutions, content management solutions, business intelligence solutions (which turn out to be glorified data warehouses with some nice front end tools), strategy management and performance management applications.
As Gartner so eloquently put it: Start Managing Information, Not Just Technology.”
And I responded:
I definitely agree that technology alone will not resolve poor data quality, nor its negative impacts on the business performance of the organization.
From the LinkedIn Group for Data Governance & Stewardship, Mark Macas commented:
“Quite true.
It is a program not a project and it should revolve on business needs. IT should support business processes -- to streamline productivity and quality whether it may be a product-oriented, service-oriented business or a combination of both.
In one of my projects, it is an after-thought process. Reference data is always produced manually and it is very susceptible to redundancy and heuristic errors (mis-spelled, multiple entries having the same meaning, etc).
The client evangelizes an Agile approach, while it continues to see architecture, detailed design and analysis as a "waste of time" effort. It is frustrating to watch and be unable to do anything of considerable impact. Office politics at its inter-play is quite complex and so does the business world... it's a jungle out there!”