Data Quality Industry: Problem Solvers or Enablers?

This morning I had the following Twitter conversation with Andy Bitterer of Gartner Research and ANALYSTerical, sparked by my previous post about Data Quality Magic, the one and only source of which I posited comes from the people involved:

 

What Say You?

Although Andy and I were just joking around, there is some truth beneath these tweets.  After all, according to Gartner research, “the market for data quality tools was worth approximately $727 million in software-related revenue as of the end of 2009, and is forecast to experience a compound annual growth rate (CAGR) of 12% during the next five years.” 

So I thought I would open this up to a good-natured debate. 

Do you think the data quality industry (software vendors, consultants, analysts, and conferences) is working harder to solve the problem of poor data quality or perpetuate the profitability of its continued existence?

All perspectives on this debate are welcome without bias.  Therefore, please post a comment below.

(Please Note: Comments advertising your products and services (or bashing your competitors) will NOT be approved.)

 

Related Posts

Which came first, the Data Quality Tool or the Business Need?

Do you believe in Magic (Quadrants)?

Can Enterprise-Class Solutions Ever Deliver ROI?

Promoting Poor Data Quality

The Once and Future Data Quality Expert

Imagining the Future of Data Quality

Data Quality Magic

In previous posts I explained that, at least in regards to data quality, there are no magic beans, tooth fairies, or magic tricks.

However, and before I am branded a Muggle, I want to assure you that magic does indeed exist in the world of data quality.

The common mistake is looking for data quality magic in the wrong places.  Historically, the quest begins with technology, and perhaps because of Clarke’s Third Law: “Any sufficiently advanced technology is indistinguishable from magic.”

Data quality tools are often believed to be magic, and especially by their salespeople.

But data quality tools are not magic.

The quest continues with methodology, and perhaps because of the Hedgehogian dream of a single, all-encompassing theory, which provides the certainty and control that comes from “just following the framework.”

Data quality methodologies are also often believed to be magic, and especially by our data perfectionists.

But data quality methodologies are not magic.

This is where the quest typically ends, after believing in magic technology and/or magic methodology both fail, but usually not from a lack of repeatedly trying—and repeatedly failing.

So if data quality magic doesn’t come from either technology or methodology, where does it come from?

In the 1988 movie Willow, the title character fails the test to become an apprentice of the village wizard.  The test was to choose which of the wizard’s fingers was the source of his magic—the correct answer was for Willow to choose his own finger.

Data quality magic comes from data quality magicians—from the People working on data quality initiatives, people who are united by trust and collaboration, guided by an adaptive methodology, and of course, enabled by advanced technology.

However, without question, the one and only source of Data Quality Magic comes from Data Quality People.

 

Related Posts

DQ-Tip: “Data quality tools do not solve data quality problems...”

There are no Magic Beans for Data Quality

The Tooth Fairy of Data Quality

Data Quality is not a Magic Trick

Do you believe in Magic (Quadrants)?

Video: Oh, the Data You’ll Show!

The Tell-Tale Data

Data Quality is People!

Data Quality is not an Act, it is a Habit

The Second Law of Data Quality states that it is not a one-time project, but a sustained program.  Or to paraphrase Aristotle:

“Data Quality is not an Act, it is a Habit.”

Habits are learned behaviors, which can become automatic after enough repetition.  Habits can also be either good or bad.

Sometimes we can become so focused on developing new good habits that we forget about our current good habits.  Other times we can become so focused on eliminating all of our bad habits that we lose ourselves in the quest for perfection.

This is why Aristotle was also an advocate of the Golden Mean, which is usually simplified into the sage advice:

“Moderation in all things.”

While helping our organization develop good habits for ensuring high quality data, we often use the term Best Practice.

Although data quality is a practice, it’s one we get better at as long as we continue practicing.  Quite often I have observed the bad habit of establishing, but never revisiting, best practices.

However, as our organization, and the business uses for our data, continues to evolve, so must our data quality practice.

Therefore, data quality is not an act, but it’s also not a best practice.  It’s a habit of continuous practice, continuous improvement, continuous learning, and continuous adaptation to continuous change—which is truly the best possible habit we can develop.

Data Quality is a Best Habit.

Trust is not a checklist

This is my seventh blog post tagged Karma since I promised to discuss it directly and indirectly on my blog throughout the year after declaring KARMA my theme word for 2010 back on the first day of January, which is now almost ten months ago.

 

Trust and Collaboration

I was reminded of the topic of this post—trust—by this tweet by Jill Wanless sent from the recent Collaborative Culture Camp, which was a one day conference on enabling collaboration in a government context, held on October 15 in Ottawa, Ontario.

I followed the conference Twitter stream remotely and found many of the tweets interesting, especially ones about the role that trust plays in collaboration, which is one of my favorite topics in general, and one that plays well with my karma theme word.

 

Trust is not a checklist

The title of this blog post comes from the chapter on The Emergence of Trust in the book Start with Why by Simon Sinek, where he explained that trust is an organizational performance category that is nearly impossible to measure.

“Trust does not emerge simply because a seller makes a rational case why the customer should buy a product or service, or because an executive promises change.  Trust is not a checklist.  Fulfilling all your responsibilities does not create trust.  Trust is a feeling, not a rational experience.  We trust some people and companies even when things go wrong, and we don’t trust others even though everything might have gone exactly as it should have.  A completed checklist does not guarantee trust.  Trust begins to emerge when we have a sense that another person or organization is driven by things other than their own self-gain.”

 

Trust is not transparency

This past August, Scott Berkun blogged about how “trust is always more important than authenticity and transparency.”

“The more I trust you,” Berkun explained, “the less I need to know the details of your plans or operations.  Honesty, diligence, fairness, and clarity are the hallmarks of good relationships of all kinds and lead to the magic of trust.  And it’s trust that’s hardest to earn and easiest to destroy, making it the most precious attribute of all.  Becoming more transparent is something you can do by yourself, but trust is something only someone else can give to you.  If transparency leads to trust, that’s great, but if it doesn’t you have bigger problems to solve.”

 

Organizational Karma

Trust and collaboration create strong cultural ties, both personally and professionally.

“A company is a culture,” Sinek explained.  “A group of people brought together around a common set of values and beliefs.  It’s not the products or services that bind a company together.  It’s not size and might that make a company strong, it’s the culture, the strong sense of beliefs and values that everyone, from the CEO to the receptionist, all share.”

Organizations looking for ways to survive and thrive in today’s highly competitive and rapidly evolving marketplace, should embrace the fact that trust and collaboration are the organizational karma of corporate culture.

Trust me on this one—good karma is good business.

 

Related Posts

New Time Human Business

The Great Rift

Social Karma (Part 6)

The Challenging Gift of Social Media

The Importance of Envelopes

True Service

The Game of Darts – An Allegory

“I can make glass tubes”

My #ThemeWord for 2010: KARMA

The Business versus IT—Tear down this wall!

The Road of Collaboration

Video: Declaration of Data Governance

Create a Slippery Slope

Enterprise information initiatives, such as data governance, master data management, data quality, and business intelligence all face a common challenge—they require your organization to take on a significant and sustained change management effort.

Organizational change requires behavioral change.

Behavioral change requires more than just an executive management decree and a rational argument.  You need to unite the organization around a shared purpose, encourage collaboration, and elevate the change to a cause. 

Although some people within the organization will answer this call to action and become champions for the cause, many others will need more convincing.  As Guy Kawasaki advises, overcome this challenge by intentionally creating a slippery slope:

“Provide a safe first step.  Don’t put up any big hurdles in the beginning of the process.
The path to adopting a cause needs a slippery slope.”

Therefore, to get your enterprise information initiative off to a good start, make it easy for people to adopt the cause.

Create a slippery slope.

 

Related Posts

Common Change

“Some is not a number and soon is not a time”

Video: Declaration of Data Governance

Don’t Do Less Bad; Do Better Good

Delivering Data Happiness

The Business versus IT—Tear down this wall!

The Road of Collaboration

Darth Data

Darth Tater

While I was grocery shopping today, I couldn’t resist taking this picture of Darth Tater.

As the Amazon product review explains: “Be it a long time ago, in a galaxy far, far away or right here at home in the 21st century, Mr. Potato Head never fails to reinvent himself.”

I couldn’t help but think of how although data’s quality is determined by evaluating its fitness for the purpose of business use, most data has multiple business uses, and data of sufficient quality for one use may not be for other, perhaps unintended, business uses.

It is this “Reinventing data for mix and match business fun!” that often provides the context for what, in hindsight, appear to be obvious data quality issues.

It makes me wonder if it’s possible to turn high quality data to the dark side of the Force by misusing it for a business purpose for which it has no applicability, resulting in bad, albeit data-driven, business decisions.

Please post a comment and let me know if you think it is possible to turn Data-kin Quality-walker into Darth Data.

May the Data Quality be with you, always.

Commendable Comments (Part 7)

Blogging has made the digital version of my world much smaller and allowed my writing to reach a much larger audience than would otherwise be possible.  Although I am truly grateful to all of my readers, I am most grateful to my commenting readers. 

Since its inception over a year ago, this has been an ongoing series for expressing my gratitude to my readers for their truly commendable comments, which greatly improve the quality of my blog posts.

 

Commendable Comments

On Do you enjoy writing?, Corinna Martinez commented:

“To be literate, a person of letters, means one must occasionally write letters by hand.

The connection between brain and hand cannot be overlooked as a key component to learning.  It is by the very fact that it is labor intensive and requires thought that we are able to learn concepts and care thought into action.

One key feels the same as another and if the keyboard is changed then even the positioning of fingers while typing will have no significance.  My bread and butter is computers but all in the name of communications, understanding and resolution of problems plaguing people/organizations.

And yet, I will never be too far into a computer to neglect to write a note or letter to a loved one.  While I don’t journal, and some say that writing a blog is like journaling online, I love mixing and matching even searching for the perfect word or turn of phrase.

Although a certain number of simians may recreate something legible on machines, Shakespeare or literature of the level to inspire and move it will not be.

The pen is mightier than the sword—from as earthshaking as the downfall of nations to as simple as my having gotten jobs after handwriting simple thank you notes.

Unfortunately, it may go the way of the sword and be kept in glass cases instead of employed in its noblest and most dangerous task—wielded by masters of mind and purpose.”

On The Prince of Data Governance, Jarrett Goldfedder commented:

“Politics and self-interest are rarely addressed factors in principles of data governance, yet are such a strong component during some high-profile implementations, that data governance truly does need to be treated as an art rather than a science.

Data teams should have principles and policies to follow, but these can be easily overshadowed by decisions made from a few executives promoting their own agendas.  Somehow, built into the existing theories of data governance, we should consider how to handle these political influences using some measure of accountability that all team members—stakeholders included—need to have.”

On Jack Bauer and Enforcing Data Governance Policies, Jill Wanless commented:

“Data Governance enforcement is a combination of straightforward and logical activities that when implemented correctly will help you achieve compliance, and ensure the success of your program.  I would emphasize that they ALL (Documentation, Communication, Metrics, Remediation, Refinement) need to be part of your overall program, as doing one or a few without the others will lead to increased risk of failure.

My favorite?  Tough to choose.  The metrics are key, as are the documentation, remediation and refinement.  But to me they all depend upon good communications.  If you don’t communicate your policies, metrics, risks, issues, challenges, work underway, etc., you will fail!  I have seen instances where policies have been established, yet they weren’t followed for the simple fact that people were unaware they existed.”

On Is your data complete and accurate, but useless to your business?, Dylan Jones commented:

“This sparks an episode I had a few years ago with an engineering services company in the UK.

I ran a management workshop showing a lot of the issues we had uncovered.  As we were walking through a dashboard of all the findings one of the directors shouted out that the 20% completeness stats for a piece of engineering installation data was wrong, she had received no reports of missing data.

I drilled into the raw data and sure enough we found that 80% of the data was incomplete.

She was furious and demanded that site visits be carried out and engineers should be incentivized (i.e., punished!) in order to maintain this information.

What was interesting is that the data went back many years so I posed the question:

‘Has your decision-making ability been impeded by this lack of information?’

What followed was a lengthy debate, but the outcome was NO, it had little effect on operations or strategic decision making.

The company could have invested considerable amounts of time and money in maintaining this information but the benefits would have been marginal.

One of the most important dimensions to add to any data quality assessment is USEFULNESS, I use that as a weight to reduce the impact of other dimensions.  To extend your debate further, data may be hopelessly inaccurate and incomplete, but if it’s of no use, then let’s take it out of the equation.”

On Is your data complete and accurate, but useless to your business?, Gordon Hamilton commented:

“Data Quality dimensions that track a data set’s significance to the Business such as Relevance or Impact could help keep the care and feeding efforts for each data set in ratio to their importance to the Business.

I think you are suggesting that the Business’s strategic/tactical objectives should be used to self-assess and even prune data quality management efforts, in order to keep them aligned with the Business rather than letting them have an independent life of their own.

I wonder if all business activities could use a self-assessment metric built in to their processing so that they can realign to reality.  In the low levels of biology this is sometimes referred to as a ‘suicide gene’ that lets a cell decide when it is no longer needed.  Suicide is such a strong term though, maybe it could be called an: annual review to realign efforts to organizational goals gene.”

On Is your data complete and accurate, but useless to your business?, Winston Chen commented:

“A particularly nasty problem in data management is that data created for one purpose gets used for another.  Often, the people who use the data don't have a choice.  It’s the only data available!

And when the same piece of data is used for multiple purposes, it gets even tougher.  As you said, completeness and accuracy has a context: the same piece of data could be good for one purpose and useless for another.

A major goal of data governance is to define and enforce policies that aligns how data is created with how data is used.  And if conflicts arise—they surely will—there’s a mechanism for resolving them.”

On Data Quality and the Cupertino Effect, Marty Moseley commented:

“I usually separate those out by saying that validity is a binary measurement of whether or not a value is correct or incorrect within a certain context, whereas accuracy is a measurement of the valid value’s ‘correctness’ within the context of the other data surrounding it and/or the processes operating upon it.

So, validity answers the question: ‘Is ZW a valid country code?’ and the answer would (currently) be ‘Yes, on the African continent, or perhaps on planet Earth.’

Accuracy answers the question: ‘Is it 2.5 degrees Celsius today in Redding, California?’

To which the answer would measure several things: is 2.5 degrees Celsius a valid temperature for Redding, CA? (yes it is), is it probable this time of year? (no, it has never been nearly that cold on this date), and are there any weather anomalies noted that might recommend that 2.5C is valid for Redding today? (no, there are not). So even though 2.5C is a valid air temperature, Redding, CA is a valid city and state combination, and 2.5C is valid for Redding in some parts of the year, that temperature has never been seen in Redding on July 15th and therefore it is probably not accurate.

Another ‘accuracy’ use case is one I’ve run into before: Is it accurate that Customer A purchased $15,049.00 in <product> on order 123 on <this date>?

To answer this, you may look at the average order size for this product (in quantity and overall price), the average order sizes from Customer A (in quantity ordered and monetary value), any promotions that offer such pricing deals, etc.

Given that the normal credit card charges for this customer are in the $50.00 to $150.00 range, and that the products ordered are on average $10.00 to $30.00, and that even the best customers normally do not order more than $200, and that there has never been a single order from this type of customer for this amount, then it is highly unlikely that a purchase of this size is accurate.”

On Do you believe in Magic (Quadrants)?, Len Dubois commented:

“I believe Magic Quadrants (MQ) are a tool that clients of Gartner, and any one else that can get their hands on them, use as one data point in their decision making process.

Analytic reports, like any other data point, are as useful or dangerous as the user wants/needs it to be.  From a buyer’s perspective, a MQ can be used for lots of things:

1. To validate a market
2. To identify vendors in the marketplace
3. To identify minimum qualifications in terms of features and functionality
4. To identify trends
5. To determine a company’s viability
6. To justify one’s choice of a vendor
7. To justify value of a purchase
8. Worse case scenario: defends one choice of a failed selection
9. To demonstrate business value of a technology

I also believe they use the analysts, Ted and Andy in this instance, as a sounding board to validate what they believe or learned from other data points, i.e. references, white papers, demos, friends, colleagues, etc.

In the final analysis though, I know that clients usually make their selection based on many things, the MQ included.  One of the most important decision points is the relationship they have with a vendor or the one they believe they are going to be able to develop with a new vendor—and no MQ is going to tell you that.”

Thank You

Thank you all for your comments.  Your feedback is greatly appreciated—and truly is the best part of my blogging experience.

This entry in the series highlighted commendable comments on OCDQ Blog posts published in May, June, and July of 2010. 

Since there have been so many commendable comments, please don’t be offended if one of your comments wasn’t featured.

Please keep on commenting and stay tuned for future entries in the series.

 

Related Posts

Commendable Comments (Part 6)

Commendable Comments (Part 5)

Commendable Comments (Part 4)

Commendable Comments (Part 3)

Commendable Comments (Part 2)

Commendable Comments (Part 1)

DQ-Tip: “Data quality tools do not solve data quality problems...”

Data Quality (DQ) Tips is an OCDQ regular segment.  Each DQ-Tip is a clear and concise data quality pearl of wisdom.

“Data quality tools do not solve data quality problems—People solve data quality problems.”

This DQ-Tip came from the DataFlux IDEAS 2010 Assessing Data Quality Maturity workshop conducted by David Loshin, whose new book The Practitioner's Guide to Data Quality Improvement will be released next month.

Just like all technology, data quality tools are enablers.  Data quality tools provide people with the capability for solving data quality problems, for which there are no fast and easy solutions.  Although incredible advancements in technology continue, there are no Magic Beans for data quality.

And there never will be.

An organization’s data quality initiative can only be successful when people take on the challenge united by collaboration, guided by an effective methodology, and of course, enabled by powerful technology.

By far the most important variable in implementing successful and sustainable data quality improvements is acknowledging David’s sage advice:  people—not tools—solve data quality problems.

 

Related Posts

DQ-Tip: “There is no such thing as data accuracy...”

DQ-Tip: “Data quality is primarily about context not accuracy...”

DQ-Tip: “There is no point in monitoring data quality...”

DQ-Tip: “Don't pass bad data on to the next person...”

DQ-Tip: “...Go talk with the people using the data”

DQ-Tip: “Data quality is about more than just improving your data...”

DQ-Tip: “Start where you are...”

Once Upon a Time in the Data

“Once upon a time, and a very good time it was, there was a moocow coming down along the road, and this moocow that was coming down along the road, met a nicens little boy named baby tuckoo . . .”

That is the opening line from the novel A Portrait of the Artist as a Young Man by James Joyce.

This novel’s unstructured data can be quite challenging, especially the opening chapter since it is written from the perspective of a young child discovering both the world and the words used to describe it.

Harry Levin, editor of a collection of Joyce’s work, commented that “the novelist and the poet, through their command of words, are mediators between the world of ideas and the world of reality.”

All data professionals, through their command of data, are also mediators between the world of ideas—whether recorded in the structured data of relational databases and spreadsheets, or the unstructured data of documents and social media content—and the world of reality, which is what all of that structured and unstructured data are discovering and attempting to describe.

 

Data is not Literal

As I have written about in previous posts, whether it’s an abstract description of real-world entities (i.e., “master data”) or an abstract description of real-world interactions (i.e., “transaction data”) among entities, data is an abstract description of reality.

The inconvenient truth is that the real world is not the same thing as these abstract descriptions of it—not even when we believe that data perfection is possible (or have managed to convince ourselves that our data is perfect).

Although real-world alignment is a good definition for data quality, there is always a digital distance between data and reality.  Data is not literal, which means that it can never literally represent reality—data can only describe reality.

 

Data is Literary

There is a structure in the unstructured data of novels and poetry, but it’s eerily reminiscent of the structure we impose on reality by describing it with data.  A novel is a narrative creating a static and comforting—but fictional—semblance of reality.

To make sense of a novel or a poem—or of any data—we must enter its reality, we must believe that its fiction is fact.

Samuel Taylor Coleridge explained the necessity of believing in this “semblance of truth sufficient to procure for these shadows of imagination that willing suspension of disbelief for the moment, which constitutes poetic faith.”

“The final belief,” Wallace Stevens once wrote, “is to believe in a fiction, which you know to be a fiction.”  Stevens believed that reality is created by our imagination, which we use to understand the constantly changing world around us.  “Reality is the product of the most august imagination.”

Data is a fiction we believe in, which we know to be a fiction.  Data is not literal, but literary—data tells us a story.

 

Data is a Storyteller

Our data tells us stories about the real world.  Data quality is concerned with how well these stories describe who was involved and what happened.  Master data are the story’s characters and subjects, while transaction data are the events and interactions comprising the narrative of the story.  Let’s use a simple (and fictional) example:

Michelle Davis-Donovan purchases a life insurance policy for her husband Michael Donovan from Vitality Insurance.

The characters are Michelle Davis-Donovan, Michael Donovan, and Vitality Insurance.  The event bringing them together is the purchase of what becomes the subject of the story that connects them, a life insurance policy, around which a narrative forms.

One of the recurring interactions in the narrative are the premium payments that Michelle sends to Vitality.  Another event, occurring later in the story, is Michael’s unexpected death, which triggers both the end of the premium payments and the beginning of the processing of the insurance claim, eventually resulting in a payment made to Michelle by Vitality.

In data management terms, Michelle Davis-Donovan and Michael Donovan (Customers), the life insurance policy (Product), and Vitality Insurance (Vendor) are all master data, and the life insurance premium and claim payments are transaction data.

It may be tempting to think of the similar stories in our databases as non-fiction, as a historical account describing real-world people and events.  After all, it’s probably safe to assume that Vitality Insurance had verified that Michelle had, in fact, paid the premiums on the life insurance policy, as well as verified that Michael was, in fact, dead, before cutting a check for the claim.

But even history is a convenient fiction, which is open to revision based on the presentation of newly discovered “facts.”

Let’s imagine that Michelle starts a new chapter in her life’s story by changing her given name to Clarissa and then marrying Richard Dalloway.  Mrs. Dalloway then purchases a life insurance policy for her husband from Vitality Insurance.

After a few years of bank verified premium payments made by Clarissa to Vitality, Richard unexpectedly dies.

How is this reality described by the data managed by Vitality Insurance?  Is Clarissa Dalloway the same real-world person as Michelle Davis-Donovan?  Is Michelle, if that’s even her real name, killing her husbands to collect on their life insurance policies?

No doubt there are characters, subjects, events, and interactions like these to be found in the stories your data is telling you.

Is your data fact or fiction?  More specifically, is your data a fiction that you feel you have to believe in?

 

Once Upon a Time in the Data

Stephen Dedalus, the protagonist of A Portrait of the Artist as a Young Man, was James Joyce’s literary alter ego, some aspects of which accurately described him and his actual real-life experiences.  Does this make author and character literally equivalent?

Would data matching routines identify Stephen Dedalus and James Joyce as duplicate customers?

What about your data?  I do not mean the data you work with as a data professional, I mean your personal data.  How many companies view you as a customer?  How many companies have master and transaction data that is telling stories about you?

All of that data is your literary alter ego.  Is that data fact or fiction?  Are all of those stories about you true?

I am pretty sure that the companies believe so, but does every aspect of that data accurately describe you?  Do these stories tell the truth about your current postal addresses, e-mail addresses, and telephone numbers?  Do these stories tell the truth about your age, the number of times you have been married, or how many children you currently have?

I often wonder about my personal data that is roaming countless databases in countless companies, telling stories about how:

“Once upon a time in the data, and a very good time it was, there was some customer data entered, and this customer data that was entered, told the story of a nicens real-world person named Jimmy . . .”

The Future of Our Data’s Story

Data privacy and protection are increasingly prevalent topics of discussion, especially in relation to data moving into the cloud.  Earlier this year, I wrote a blog post that examined some of the impacts of the semantic web on the future of data management.

Lately I’ve been thinking about how these two trends could provide customers with greater control over their literary alter egos, giving them more control over their personal data—and the stories that it could tell.

Perhaps when this finally happens, our data’s story will become more fact than fiction.

 

Related Posts

DQ-BE: Data Quality Airlines

The Data-Decision Symphony

The Road of Collaboration

The Idea of Order in Data

Hell is other people’s data

To Our Data Perfectionists

Had our organization but money enough, and time,
This demand for Data Perfection would be no crime.

We would sit down and think deep thoughts about all the wonderful ways,
To best model our data and processes, as slowly passes our endless days.
Freed from the Herculean Labors of Data Cleansing, we would sing the rhyme:
“The data will always be entered right, the first time, every time.”

We being exclusively Defect Prevention inclined,
Would only rubies within our perfected data find.
Executive Management would patiently wait for data that’s accurate and complete,
Since with infinite wealth and time, they would never fear the balance sheet.

Our vegetable enterprise data architecture would grow,
Vaster than empires, and more slow.

One hundred years would be spent lavishing deserved praise,
On our brilliant data model, upon which, with wonder, all would gaze.
Two hundred years to adore each and every defect prevention test,
But thirty thousand years to praise Juran, Deming, English, Kaizen, Six Sigma, and all the rest.
An age at least to praise every part of our flawless data quality methodology,
And the last age we would use to write our self-aggrandizing autobiography.

For our Corporate Data Asset deserves this Perfect State,
And we would never dare to love our data at any lower rate.

But at my back I always hear,
Time’s winged chariot hurrying near.

And if we do not address the immediate business needs,
Ignored by us while we were lost down in the data weeds.
Our beautiful enterprise data architecture shall no more be found,
After our Data Perfectionists’ long delay has run our company into the ground.

Because building a better tomorrow at the expense of ignoring today,
Has even with our very best of intentions, caused us to lose our way.
And all our quaint best practices will have turned to dust,
As burnt into ashes will be all of our business users’ trust.

Now, it is true that Zero Defects is a fine and noble goal,
For Manufacturing Quality—YES, but for Data Quality—NO.

We must aspire to a more practical approach, providing a critical business problem solving service,
Improving data quality, not for the sake of our data, but for the fitness of its business purpose.
Instead of focusing on only the bad we have done, forcing us to wear The Scarlet DQ Letter,
Let us focus on the good we are already doing, so from it we can learn how to do even better.

And especially now, while our enterprise-wide collaboration conspires,
To help us grow our Data Governance Maturity beyond just fighting fires.
Therefore, let us implement Defect Prevention wherever and whenever we can,
But also accept that Data Cleansing will always be an essential part of our plan.

Before our organization’s limited money and time are devoured,
Let us make sure that our critical business decisions are empowered.

Let us also realize that since change is the only universal constant,
Real best practices are not cast in stone, but written on parchment.
Because the business uses for our data, as well as our business itself, continues to evolve,
Our data strategy must be adaptation, allowing our dynamic business problems to be solved.

Thus, although it is true that we can never achieve Data Perfection,
We can deliver Business Insight, which always is our true direction.

___________________________________________________________________________________________________________________

This blog post was inspired by the poem To His Coy Mistress by Andrew Marvell.

#FollowFriday and The Three Tweets

Today is Friday, which for Twitter users like me, can mean only one thing . . .

It is FollowFriday—the day when Twitter users recommend other users that you should follow.  In other words, it’s the Twitter version of peer pressure: “I recommended you, why didn't you recommend me?”

So why does anyone follow anyone on Twitter?  There are many theories, mine is called . . .

 

The Three Tweets

From my perspective, there are only three kinds of tweets:

  1. Informative Tweets — Providing some form of information, or a link to it, these tweets deliver the practical knowledge or thought-provoking theories, allowing you to almost convince your boss that Twitter is a required work activity.
  2. Entertaining Tweets — Providing some form of entertainment, or a link to it, these tweets are often the funny respites thankfully disrupting the otherwise serious (or mind-numbingly boring) routine of your typical business day.
  3. Infotaining Tweets — Providing a combination of information and entertainment, or a link to it, these tweets make you think a little, laugh a little, and go on and sway (just a little) along with the music that often only you can hear.

Let’s take a look at a few examples of each one of The Three Tweets.

 

Informative Tweets

 

Entertaining Tweets

 

Infotaining Tweets

 

#FollowFriday Recommendations

By no means a comprehensive list, and listed in no particular order whatsoever, here are some great tweeps, and especially for mostly informative tweets about Data Quality, Data Governance, Master Data Management, and Business Intelligence:

 

PLEASE NOTE: No offense is intended to any of my tweeps not listed above.  However, if you feel that I have made a glaring omission of an obviously Twitterific Tweep, then please feel free to post a comment below and add them to the list.  Thanks!

I hope that everyone has a great FollowFriday and an even greater weekend.  See you all around the Twittersphere.

 

Related Posts

Dilbert, Data Quality, Rabbits, and #FollowFriday

Twitter, Meaningful Conversations, and #FollowFriday

The Fellowship of #FollowFriday

Video: Twitter #FollowFriday – January 15, 2010

Social Karma (Part 7)

If you tweet away, I will follow

Video: Twitter Search Tutorial

DQ-BE: Data Quality Airlines

Data Quality By Example (DQ-BE) is a new OCDQ segment that will provide examples of data quality key concepts.

“Good morning sir!” said the smiling gentleman behind the counter—and a little too cheerily for 5 o’clock in the morning.  “Welcome to the check-in counter for Data Quality Airlines.  My name is Edward.  How may I help you today?”

“Good morning Edward,” I replied.  “My name is John Smith.  I am traveling to Boston today on flight number 221.”

“Thank you for choosing Data Quality Airlines!” responded Edward.  “May I please see your driver’s license, passport, or other government issued photo identification so that I can verify your data accuracy.”

As I handed Edward my driver’s license, I explained “it’s an old photograph in which I was clean-shaven, wearing contact lenses, and ten pounds lighter” since I now had a full beard, was wearing glasses, and, to be honest, was actually thirty pounds heavier.

“Oh,” said Edward, his plastic smile morphing into a more believable and stern frown.  “I am afraid you are on the No Fly List.”

“Oh, that’s right—because of my name being so common!” I replied while fumbling through my backpack, frantically searching for the piece of paper, which I then handed to Edward.  “I’m supposed to give you my Redress Control Number.”

“Actually, you’re supposed to use your Redress Control Number when making your reservation,” Edward retorted.

“In other words,” I replied, while sporting my best plastic smile, “although you couldn’t verify the accuracy of my customer data when I made my reservation on-line last month, you were able to verify the authorization to immediately charge my credit card for the full price of purchasing a non-refundable plane ticket to fly on Data Quality Airlines.”

“I don’t appreciate your sense of humor,” replied Edward.  “Everyone at Data Quality Airlines takes accuracy very seriously.”

Edward printed my boarding pass, wrote BCS on it in big letters, handed it to me, and with an even more plastic smile cheerily returning to his face, said: “Please proceed to the security checkpoint.  Thank you again for choosing Data Quality Airlines!”

“Boarding pass?” asked the not-at-all smiling woman at the security checkpoint.  After I handed her my boarding pass, she said, “And your driver’s license, passport, or other government issued photo identification so that I can verify your data accuracy.”

“I guess my verified data accuracy at the Data Quality Airlines check-in counter must have already expired,” I joked as I handed her my driver’s license.  “It’s an old photograph in which I was clean-shaven, wearing contact lenses, and ten pounds lighter.”

The woman silently examined my boarding pass and driver’s license, circled BCS with a magic marker, and then shouted over her shoulder to a group of not-at-all smiling security personnel standing behind her: “Randomly selected security screening!”

One of them, a very large man, stepped toward me as the sound from the snap of the fresh latex glove he had just placed on his very large hand echoed down the long hallway that he was now pointing me toward.  “Right this way sir,” he said with a smile.

Ten minutes later, as I slowly walked to the gate for Data Quality Airlines Flight Number 221 to Boston, the thought echoing through my mind was that there is no such thing as data accuracy—there are only verifiable assertions of data accuracy . . .

Related Posts

DQ-Tip: “There is no such thing as data accuracy...”

Why isn’t our data quality worse?

The Real Data Value is Business Insight

Is your data complete and accurate, but useless to your business?

Data Quality and the Cupertino Effect

DQ-Tip: “Data quality is primarily about context not accuracy...”

DQ-Tip: “There is no such thing as data accuracy...”

Data Quality (DQ) Tips is an OCDQ regular segment.  Each DQ-Tip is a clear and concise data quality pearl of wisdom.

“There is no such thing as data accuracy — There are only assertions of data accuracy.”

This DQ-Tip came from the Data Quality Pro webinar ISO 8000 Master Data Quality featuring Peter Benson of ECCMA.

You can download (.pdf file) quotes from this webinar by clicking on this link: Data Quality Pro Webinar Quotes - Peter Benson

ISO 8000 is the international standards for data quality.  You can get more information by clicking on this link: ISO 8000

Data Accuracy

Accuracy, which, thanks to substantial assistance from my readers, was defined in a previous post as both the correctness of a data value within a limited context such as verification by an authoritative reference (i.e., validity) combined with the correctness of a valid data value within an extensive context including other data as well as business processes (i.e., accuracy).

“The definition of data quality,” according to Peter and the ISO 8000 standards, “is the ability of the data to meet requirements.”

Although accuracy is only one of many dimensions of data quality, whenever we refer to data as accurate, we are referring to the ability of the data to meet specific requirements, and quite often it’s the ability to support making a critical business decision.

I agree with Peter and the ISO 8000 standards because we can’t simply take an accuracy metric on a data quality dashboard (or however else the assertion is presented to us) at face value without understanding how the metric is both defined and measured.

However, even when well defined and properly measured, data accuracy is still only an assertion.  Oftentimes, the only way to verify the assertion is by putting the data to its intended use.

If by using it you discover that the data is inaccurate, then by having established what the assertion of accuracy was based on, you have a head start on performing root cause analysis, enabling faster resolution of the issues—not only with the data, but also with the business and technical processes used to define and measure data accuracy.

The Business versus IT—Tear down this wall!

Business Information Technology

This diagram was published in the July 2009 blog post Business Information Technology by Steve Tuck of Datanomic, and was based on a conference conversation with Gwen Thomas of the Data Governance Institute, about the figurative wall, prevalent in most organizations, which literally separates the Business, who usually own its data and understand its use in making critical daily business decisions, from Information Technology (IT), who usually own and maintain the hardware and software infrastructure of its enterprise data architecture.

The success of all enterprise information initiatives requires that this wall be torn down, ending the conflict between the Business and IT, and forging a new collaborative union that Steve and Gwen called Business Information Technology.

 

Isn’t IT a part of the Business?

In his recent blog post Isn’t IT a Part of “the Business”?, Winston Chen of Kalido examined this common challenge, remarking how “IT is often a cost center playing a supporting role for the frontline functions.  But Finance is a cost center, too.  Is Finance really the Business?  How about Human Resources?  We don’t hear HR people talk about the Business versus HR, do we?”

“Key words are important in setting the tone for communication,” Winston explained.  “When our language suggests IT is not a part of the Business, it cements a damaging us-versus-them mentality.”

“It leads to isolation.  What we need today, more than ever, is close collaboration.”

 

Purple People

Earlier this year in his blog post “Purple People”: The Key to BI Success, Wayne Eckerson of TDWI used a colorful analogy to discuss this common challenge within the context of business intelligence (BI) programs.

Wayne explained that the color purple is formed by mixing two primary colors: red and blue.  These colors symbolize strong, distinct, and independent perspectives.  Wayne used red to represent IT and blue to represent the Business.

Purple People, according to Wayne, “are key intermediaries who can reconcile the Business and IT and forge a strong and lasting partnership that delivers real value to the organization.”

“Pure technologists or pure business people can’t harness BI successfully.  BI needs Purple People to forge tight partnerships between business people and technologists and harness information for business gain.”

I agree with Wayne, but I believe all enterprise information initiatives, and not just BI, need Purple People for success.

 

Tearing down the Business-IT Wall

My overly dramatic blog post title is obviously a reference to the famous speech by United States President Ronald Reagan at the Berlin Wall on June 12, 1987.  For more than 25 years, the Berlin Wall had stood as a symbol of not only a divided Germany and divided political ideologies, but more importantly, it was both a figurative and literal symbol of a deeper human divide.

Although Reagan’s speech was merely symbolic of the numerous and complex factors that eventually lead to the dismantling of the Berlin Wall and the end of the Cold War, symbolism is a powerful aspect of human culture—including corporate culture.

The Business-IT Wall is only a figurative wall, but it literally separates the Business and IT in most organizations today.

So much has been written about the need for Business-IT Collaboration on successful enterprise information initiatives that the message is often ignored because people are sick and tired of hearing about it.

However, although there are other barriers to success, and people, process, and technology are all important, by far the most important factor for true and lasting success to be possible is—peoplecollaborating.

Organizations must remove all symbolic obstacles, both figurative and literal, which contribute to the human divide preventing enterprise-wide collaboration within their unique corporate culture.

As for the Business-IT Wall, and all other similar barriers to our collaboration and success, the time is long overdue for us to:

Tear down this wall!

Related Posts

The Road of Collaboration

Finding Data Quality

Data Transcendentalism

Declaration of Data Governance

Podcast: Business Technology and Human-Speak

Not So Strange Case of Dr. Technology and Mr. Business

Data Quality is People!

You're So Vain, You Probably Think Data Quality Is About You

DQ View: Achieving Data Quality Happiness

Data Quality (DQ) View is an OCDQ regular segment.  Each DQ-View is a brief video discussion of a data quality key concept.

Continuing the happiness meme making its way around the data quality blogosphere, which I contributed to with my previous blog posts Delivering Data Happiness and Why isn’t our data quality worse?, in this new DQ-View segment I want to discuss achieving data quality happiness.

 

DQ View: Achieving Data Quality Happiness

 

If you are having trouble viewing this video, then you can watch it on Vimeo by clicking on this link: DQ-View on Vimeo

 

Related Posts

Delivering Data Happiness

Why isn’t our data quality worse?

Video: Oh, the Data You’ll Show!

Data Quality is not a Magic Trick

DQ-View: The Cassandra Effect

DQ-View: Is Data Quality the Sun?

DQ-View: Designated Asker of Stupid Questions