The Poor Data Quality Jar
Jim HarrisToday I am pondering whether or not the venerable tradition of The Swear Jar could be adapted to help organizations illustrate the costs of poor data quality.
As an example for those unfamiliar with the concept, my family used a swear jar when I was growing up. Anytime a family member swore (i.e., used profanity), they put an amount of money into the jar based on the severity of the swear.
Of course in my family, what exactly constituted “profanity” as well as what the severity of a particular swear should be would often cause considerable debate, which somewhat ironically lead to the increased use of unquestionable profanity.
Therefore, The Swear Jar was far from a perfect system (at least for my family).
But I am still imaging every organization instituting The Poor Data Quality Jar.
When an employee contributes to the organization's poor data quality, they put an amount of money into the jar based on the severity of the data quality issue, and perhaps with an increasing scale to be more punitive to repeat offenders.
Do you think The Poor Data Quality Jar can help your organization? If so, how much would you charge for different types of data quality issues? How would you determine the severity (i.e., financial impact) of each data quality issue?
Photo via Flickr (Creative Commons License) by: Karen Roe
Reader Comments (15)
Fun. I think this is a great idea, however I'm not sure ANYONE could afford the rates that would be realistic. Really honestly looking at what bad data quality costs in terms of lost opportunity will result in very large numbers.
Maybe the better way to do it is to deduct the costs of data quality from the budgets of the departments that tolerate the bad data generating practices...you might find that this motivates managers to encourage their resources to watch that data :-)
Another approach might be to have a large central team fixing data, and charge the time spent back to the originators of the bad data. The advantages of this approach would be that the data gets fixed, and rather than having to determine the cost of the bad data, you only have to determine the cost of the fixers - which is pretty clear. Of course, you still have to define what constitutes bad data - but that's probably a useful thing to do. Hopefully the central team would soon find itself out of work.
Very fun post Jim, and getting us thinking as always!
Jim,
I really like the idea...but I fear it's addressing the wrong problem in the wrong way.
The "public humiliation" works for a close-knit group and clearly visible transgressions...whereas the problem with Data Quality is more how to bring those transgressions to light and explain why it's so bad. Once you've done that, people agree that there has to be some improvement.
But you don't want to scare people off with a "tip jar" unless you've already built your reputation.
What's your take?
Thorsten
Thanks for your comments James and Thorsten, as always your feedback is greatly appreciated.
@James Good point. Nobody would be able to afford to contribute to the jar personally, and either the department budget deduction system, or the central team fixing data and charging the costs back to the originators would be a better approach.
@Thorsten Another good point. The Poor Data Quality Jar would probably only work best for a close-knit group, especially one comprised of data quality professionals who should know better (like us for example!).
Jim,
Maybe the swear jar is appropriate for those who intentionally screw things up. Honest mistakes are one thing but two types of people really irk me:
1. Those who just don't care about accuracy
2. Those who intentionally create data issues
Granted, I don't see too many of the latter group, but these people ought to circumvent the swear jar and move to electric shocks.
We actually tried to implement something like this in the phone company with regard to billing data quality issues that created compliance problems. Our aim was to have the cost of fixing the problem borne by the business area which created the issue, with the "swear jar" being the budget pool for remediation projects.
We ran into a few practical problems:
1) Many problems ultimately had multiple areas with responsibility (line-of-business workers bypassing processes, IT historically "right-sizing" scope on projects, business processes and business requirements not necessarily being defined properly resulting in inevitable errors)
2) Politics often prevented us from pushing the evidence we did have too hard as to the weighting of contributions towards any issue.
3) More often than not it was not possible to get hard metrics on which to base a weighting of contribution, and people tended to object to being blamed for a problem that was obviously complex with multiple inputs.
That said, the attempt to do it did help us to:
1) Justify our "claims" that these issues were often complex with multiple stakeholders involved.
2) Get stakeholders to think about the processes end-to-end, including the multiple IT systems that were involved in even the simplest process.
3) Ensure we had human resources assigned to projects because we had metrics to apply to a business case.
4) Start building a focus on prevention of defect rather than just error detection and fix.
We never did get around to using electric shocks on anyone. But I'd by lying if I said it was never a temptation.
Jim,
We are trying something similar at my organization. We have started compiling our time under a post-project category for any data quality cleansing activities resulting in a system release that ends up causing data quality issues. At this point, IT does not charge back to the business but this exercise will be a true eye opener. Results are yet to come.
You got me thinking on how we can also charge back to the business when the data quality cleansing activities are clearly related to business activities.
Thanks,
Nicole
Jim,
Another great, thought provoking post.
As data accuracy issues in some cases will be identified by front line staff, how likely are they going to be to report them? Whilst the electric chair would be a tempting solution for certain data quality transgressions, would it mean that more data quality problems are reported?
This presents a similar issue to that in large companies when they look at their accident reporting statistics and reports of near misses/near hits:
* Does a high number of reported accidents and near hits mean that the company is unsafe, or does it mean that there are high levels of reporting coupled with a supportive, learning culture?
* Does a low number of reported accidents and near hits mean that the company is safe, or does it mean that staff are too scared of repercussions to report anything?
If staff risk a large fine/electric shock for owning up to transgressions, they will not do it and will work hard to hide the evidence, if they can.
In organisational/industrial situations, there are often multiple contributing factors to accidents and data quality problems. To minimise the level of future problems, all contributory causes need to be identified and resolved. To achieve this, staff should not be victimised/blamed in any way and should be encouraged to report issues without fear.
Julian
Or ... develop a Virtual Jar where you put the amount correspondent to the cost of the scrap and rework processes executed due to poor data quality.
Great comments Phil, Daragh, Nicole, Julian, and Francisco! Thank you very much for your feedback.
@Phil Only two types of people really irk you? What about those who don't know Rush is the greatest band ever? The Poor Data Quality Jar 2.0 could be rigged with intermittent electrodes to deliver random electric shocks. I have R&D working on the prototype as I type.
@Daragh Raising awareness of data quality issues and getting the organization to understand the tangible (and negative) financial impact is often the difficult first step. As you (and Julian) point out, the next step is getting people past the natural reaction of not wanting to be blamed for causing (or failing to fix) data quality issues, which as you said, often have multiple complex root causes. Focusing on the business case and future defect prevention usually helps builds the necessary momentum to improvement.
@Nicole Great ideas! I am surprised when more organizations don't follow a similar approach. Too often a "data cleansing project" is created as a separate effort not tied back (even loosely) to the original project that spawned it. Tracking this effort under a post-project category should at least raise organizational awareness. After all, can a project finished "on-time and on-budget" truly be said to be either if it introduced data quality issues that have to be fixed by another project?
@Julian Excellent points. Similar to Thorsten's concern about the "public humiliation" of the The Poor Data Quality Jar, apparent compliance could actually be unreported issues for fear of the repercussion. Similar to how the "swear jar" my family used really only taught me and my siblings to reserve our profanity for when we were not at home :-)
@Francisco I like the Virtual Jar. Perhaps I could market it as a cloud computing based The Poor Data Quality Jar 3.0 :-)
It depends on what becomes of the money put into the jar. We actually had a team swear jar for awhile, but we used its proceeds to fund Friday afternoon beer runs. Let's just say it wasn't exactly a deterrent...
But Thorsten is right in that this type of "public humiliation" works for best a close-knit group and clearly visible transgressions - not for the larger organization as a whole.
Francisco's idea of a "virtual" swear jar - with some level of anonymity - might work, though. You could track by team or division rather than individual. Group dynamics work very well when competing with other groups. Being able to visually demonstrate how teams are doing in relation to other teams could be very powerful and motivating.
I tried a swear jar with my kids but I was always the worst offender. At least it made ME realize I was swearing too much.
Is the idea of a Data Quality swear jar like buying indulgences or offsetting carbon footprint?
Are we looking to identify the tradeoff in data quality with tactical decisions?
May be a good idea, but if we knew when we were causing data quality problems we would be a lot closer to knowing how to solve them.
Thanks for your awesome comments Crysta and April, your feedback is greatly appreciated.
@Crysta Now you have me thinking about The Poor Data Quality Jarboard, which would be a web-based dashboard reporting portal updated in near real-time :-)
@April It does sometimes seem like vendors are selling data quality indulgences and not data quality solutions :-)
But on a more serious note, you make an excellent point about knowing when we cause data quality problems bringing us a lot closer to knowing how to solve them.
Jim, the "poor data quality jar" is an interesting concept, but likely unfeasible and impractical in reality. But I suspect you know that. Poor data quality affects every part of the business and there are many moving parts that need to be addressed.
This can only be done successfully through a top-down perspective, with executive buy-in and leadership and involvement of all departments. It is not enough to assign blame or hold certain employees (but not others) solely responsible for the outcome of poor data quality.
If a company is serious about their data quality, they will recognize that a strong data governance program combined with an enterprise level data quality solution is required.
Thanks for your very insightful comment, Kit.
I completely agree with you that executive sponsorship and an enterprise-wide data governance program is essential for lasting data quality success.
Best Regards,
Jim
Jim,
In response to your post, I have resurrected an idea I had for an alternative approach to dealing with "Data Accidents" see our new blog post: The Data Accident Investigation Board
Compare and contrast....
Julian