Data Quality and the Bystander Effect

In his recent Harvard Business Review blog post Break the Bad Data Habit, Tom Redman cautioned against correcting data quality issues without providing feedback to where the data originated.  “At a minimum,” Redman explained, “others using the erred data may not spot the error.  There is no telling where it might turn up or who might be victimized.”  And correcting bad data without providing feedback to its source also denies the organization an opportunity to get to the bottom of the problem.

“And failure to provide feedback,” Redman continued, “is but the proximate cause.  The deeper root issue is misplaced accountability — or failure to recognize that accountability for data is needed at all.  People and departments must continue to seek out and correct errors.  They must also provide feedback and communicate requirements to their data sources.”

In his blog post The Secret to an Effective Data Quality Feedback Loop, Dylan Jones responded to Redman’s blog post with some excellent insights regarding data quality feedback loops and how they can help improve your data quality initiatives.

I definitely agree with Redman and Jones about the need for feedback loops, but I have found, more often than not, that no feedback at all is provided on data quality issues because of the assumption that data quality is someone else’s responsibility.

This general lack of accountability for data quality issues is similar to what is known in psychology as the Bystander Effect, which refers to people often not offering assistance to the victim in an emergency situation when other people are present.  Apparently, the mere presence of other bystanders greatly decreases intervention, and the greater the number of bystanders, the less likely it is that any one of them will help.  Psychologists believe that the reason this happens is that as the number of bystanders increases, any given bystander is less likely to interpret the incident as a problem, and less likely to assume responsibility for taking action.

In my experience, the most common reason that data quality issues are often neither reported nor corrected is that most people throughout the enterprise act like data quality bystanders, making them less likely to interpret bad data as a problem or, at the very least, not their responsibility.  But the enterprise’s data quality is perhaps most negatively affected by this bystander effect, which may make it the worst bad data habit that the enterprise needs to break.

Related Posts

DQ-Tip: “Don't pass bad data on to the next person...”

Hyperactive Data Quality (Second Edition)

A Farscape Analogy for Data Quality

There is No Such Thing as a Root Cause

Data Quality and the Q Test

The Data Quality Wager

The Third Law of Data Quality

The Data Governance Oratorio

Shared Responsibility

The Algebra of Collaboration

Collaboration isn’t Brain Surgery

The Three Most Important Letters in Data Governance

Related OCDQ Radio Episodes

Clicking on the link will take you to the episode’s blog post:

• Data Driven — Guest Tom Redman (aka the “Data Doc”) discusses concepts from one of my favorite data quality books, which is his most recent book: Data Driven: Profiting from Your Most Important Business Asset.

• Organizing for Data Quality — Guest Tom Redman (aka the “Data Doc”) discusses how your organization should approach data quality, including his call to action for your role in the data revolution.

• The Johari Window of Data Quality — Guest Martin Doyle discusses helping people better understand their data and assess its business impacts, not just the negative impacts of bad data quality, but also the positive impacts of good data quality.

• Redefining Data Quality — Guest Peter Perera discusses his proposed redefinition of data quality, as well as his perspective on the relationship of data quality to master data management and data governance.

• The Blue Box of Information Quality — Guest Daragh O Brien on why Information Quality is bigger on the inside, using stories as an analytical tool and change management technique, and why we must never forget that “people are cool.”

• Studying Data Quality — Guest Gordon Hamilton discusses the key concepts from recommended data quality books, including those which he has implemented in his career as a data quality practitioner.

Tweet

Comments

[Jim Harris]

From the LinkedIn Group for the Data Quality Network, Jane Sanders commented:

“However, there can be many reasons for that, right? Being a bystander might have many reasons, it's like a symptom that needs further analysis in order to identify the cause.

For example, the bystander might not even care, might not be aware of the consequence due to lack of information, might be having a workload that does not allow to spend time on such observations, the company might be organized in such a way that there is no process for handling observed DQ issues, etc.

Maybe 'the bystander' can be grouped in two: those who choose to be bystanders, and those who are not aware that they are acting like a bystander?”

And I responded:

Thanks for your comment, Jane.

You make an excellent point about the need to differentiate data quality bystanders into two groups.

Using a Seinfeld analogy, we could call them Innocent Bystanders and Guilty Bystanders. Due to, as you noted, within most organizations, the lack of established processes for at least reporting data quality issues, data quality probably has slightly more Innocent Bystanders.

Best Regards,

Jim

[Jill Wanless]

Excellent post Jim. This is so true!

Our organization is starting to develop data governance processes and one of the processes we have deliberately designed is to get to the root cause of data quality issues. We've designed it so that the errors that are reported also include the userid and the system where the data was generated.

Errors are then filtered by function and the business steward responsible for that function is the one who is responsible for determining and addressing the root cause (which of course may require escalation to solve). The business steward for the functional area has the most at stake in the data and is typically the most knowledgeable as to the process or system that may be triggering the error.

We have yet to test this as we are currently in the process of deploying a pilot stewardship program. However, we are very confident that it will help us uncover many of the causes of the data quality problems and with lots of PLAN, DO, CHECK and ACT our goal is to continuously improve so that our need for stewardship eventually (many years away no doubt) is reduced.

Thanks always for your excellent insights :)

Jill

[Shivanand]

Bystanders are mostly Users (Consumers of Data), so often the most they could do is provide feedback.

The time involved in going from feedback to action is generally so high that users do not feel motivated to give anymore feedback, and hence end up being bystanders.

If organizations create an atmosphere or environment where action is taken on feedback more quickly, and the whistle blower is thanked or rewarded, then the number of bystanders will significantly reduce.