Data Qualia
Jim Harris in Data Quality tagged Data Profiling, Data Quality Assessment, Philosophy Tuesday, March 1, 2011 at 5:00PM In philosophy (according to Wikipedia), the term qualia is used to describe the subjective quality of conscious experience.
Examples of qualia are the pain of a headache, the taste of wine, or the redness of an evening sky. As Daniel Dennett explains:
“Qualia is an unfamiliar term for something that could not be more familiar to each of us:
The ways things seem to us.”
Like truth, beauty, and singing ability, data quality is in the eyes of the beholder, or since data quality is most commonly defined as fitness for the purpose of use, we could say that data quality is in the eyes of the user.
However, most data has both multiple uses and multiple users. Data of sufficient quality for one use or one user may not be of sufficient quality for other uses and other users. Quite often these diverse data needs and divergent data quality perspectives make it a daunting challenge to provide meaningful data quality metrics to the organization.
Recently on the Data Roundtable, Dylan Jones of Data Quality Pro discussed the need to create data quality reports that matter, explaining that if you’re relying on canned data profiling reports (i.e., column statistics and data quality metrics at an attribute, table, and system level), then you are measuring data quality in isolation of how the business is performing.
Instead, data quality metrics must measure data qualia—the subjective quality of the user’s business experience with data:
“Data Qualia is an unfamiliar term for something that must become more familiar to the organization:
The ways data quality impact business performance.”
Related Posts
DQ-BE: Single Version of the Time
Beyond a “Single Version of the Truth”
DQ-Tip: “There is no such thing as data accuracy...”
Data Quality and the Cupertino Effect
DQ-Tip: “Data quality is primarily about context not accuracy...”
Reader Comments (1)
Hi Jim,
When I first read the title it struck me that "qualia" is about as close to "quality" as most organizations get. Not quite there, but only a few letters missing. To insist on anything else would be pedantic, would it not?
One major problem is, as you say, that people's perception of Data Quality is entirely subjective. They are in fact chasing "qualia", which they can achieve. But, as it is different for everybody, it never meets the overall needs of the enterprise.
Few, if any, individuals in an organization can define what "quality" is with regard to data - even in their part of the enterprise. None can define it for the whole enterprise. For this reason, all efforts in data quality are, ultimately, doomed to failure. They will take the enterprise from where it is to somewhere else, but is that any closer to where it ought to be?
The answer is, probably not!
Why? Because nobody has defined where that is. No Function Model was ever created, so no (quality) logical data model could be produced. Because of this there is no point to which data quality efforts can converge - so true, objective quality (which actually does exist) cannot be achieved.
Without the essential architectural elements being in place, data qualia is all that can ever be achieved. But hey, it gives people a warm fuzzy feeling, just like a pretty sunset!
Regards,
John