Expectation and Data Quality
/One of my favorite recently read books is You Are Not So Smart by David McRaney. Earlier this week, the book’s chapter about expectation was excerpted as an online article on Why We Can’t Tell Good Wine From Bad, which also provided additional examples about how we can be fooled by altering our expectations.
“In one Dutch study,” McRaney explained, “participants were put in a room with posters proclaiming the awesomeness of high-definition, and were told they would be watching a new high-definition program. Afterward, the subjects said they found the sharper, more colorful television to be a superior experience to standard programming.”
No surprise there, right? After all, a high-definition television is expected to produce a high-quality image.
“What they didn’t know,” McRaney continued, “was they were actually watching a standard-definition image. The expectation of seeing a better quality image led them to believe they had. Recent research shows about 18 percent of people who own high-definition televisions are still watching standard-definition programming on the set, but think they are getting a better picture.”
I couldn’t help but wonder if establishing an expectation of delivering high-quality data could lead business users to believe that, for example, the data quality of the data warehouse met or exceeded their expectations. Could business users actually be fooled by altering their expectations about data quality? Wouldn’t their experience of using the data eventually reveal the truth?
Retailers expertly manipulate us with presentation, price, good marketing, and great service in order to create an expectation of quality in the things we buy. “The actual experience is less important,” McRaney explained. “As long as it isn’t total crap, your experience will match up with your expectations. The build up to an experience can completely change how you interpret the information reaching your brain from your otherwise objective senses. In psychology, true objectivity is pretty much considered to be impossible. Memories, emotions, conditioning, and all sorts of other mental flotsam taint every new experience you gain. In addition to all this, your expectations powerfully influence the final vote in your head over what you believe to be reality.”
“Your expectations are the horse,” McRaney concluded, “and your experience is the cart.” You might think it should be the other way around, but when your expectations determine your direction, you shouldn’t be surprised by the journey you experience.
If you find it difficult to imagine a positive expectation causing people to overlook poor quality in their experience with data, how about the opposite? I have seen the first impression of a data warehouse initially affected by poor data quality create a negative expectation causing people to overlook the improved data quality in their subsequent experiences with the data warehouse. Once people expect to experience poor data quality when using it, people stop trusting, and stop using, the data warehouse.
Data warehousing is only one example of how expectation can affect the data quality experience. How are your organization’s expectations affecting its experiences with data quality?
