OCDQ Blog

This recent tweet (expanded using TwitLonger) by Ted Friedman of Gartner Research conspired with the swashbuckling movie Pirates of the Caribbean: The Curse of the Black Pearl, leading, really quite inevitably, to the writing of this Data Quality Tale.

Pirates of the Computer: The Curse of the Poor Data Quality

Jack Sparrow was once the Captain of Information Technology (IT) at the world famous Es el Pueblo Estúpido Corporation. 

However, when Jack revealed his plans for recommending to executive management the production implementation of the new Dystopian Automated Transactional Analysis (DATA) system and its seamlessly integrated Magic Beans software, his First Mate Barbossa mutinied by stealing the plans and successfully pitching the idea to the CIO—thereby getting Captain Sparrow fired.

As the new officially appointed Captain of IT, Barbossa implemented DATA and Magic Beans, which migrated and consolidated all of the organization’s information assets, clairvoyantly detected and corrected existing data quality problems, and once fully implemented into production, was preventing any future data quality problems from happening.

As soon as a source was absorbed into DATA, Magic Beans automatically freed up disk space by deleting all traces of the source, including all backups—somehow even the off-site archives.

DATA was then the only system of record, truly becoming the organization’s Single Version of the Truth.

DATA and Magic Beans seemed almost too good to be true.

And that’s because they were.

A few weeks after the last of the organization’s information assets had been fully integrated into DATA, it was discovered that Magic Beans was apparently infected with a nasty computer virus known as The Curse of the Poor Data Quality.

Mysterious “computer glitches” began causing bizarre data quality issues.  At first, the glitches seemed rather innocuous, such as resetting all user names to “TED FRIEDMAN” and all passwords to “GARTNER RESEARCH.”

But that’s hardly worth mentioning, especially when compared with what happened next.

All of the business-critical information stored in DATA—and all new information added—suddenly became completely inaccurate and totally useless as the basis for making any business decisions.

DATA and Magic Beans were cursed!  It was believed that the only way The Curse of the Poor Data Quality could be lifted was by re-installing the organization’s original systems and software.

William “Backup Bill” Turner, Jack’s only supporter, believing the organization deserved to remain cursed for betraying Jack, sent a USB drive to his young son, Will, which contained the only surviving backup copy of the original systems and software.

Many years later, Will Turner, still wearing his father’s old USB drive around his neck, but not knowing its alleged value, is told by Jack Sparrow that Captain Barbossa killed Will’s father and kidnapped Will’s ex-girlfriend, Elizabeth Swann.

Jack and Will infiltrate the DATA center disguised as PIRATEs (Professional Information Retrieval and Technology Experts). 

Jack tells Will that he needs the USB drive to determine where Elizabeth is being held.  Will gives Jack the USB drive and he uses it to begin restoring the original systems and software.  Moments later, Barbossa and Elizabeth walk into the DATA center.

“Elizabeth!  Don’t worry, I’m here to save you!” Will proudly declares.

“Will?” Elizabeth responds, confused.  “What are you talking about?  You’re here to save me from what?  My new job?”

Embarrassed, and turning toward Jack, Will shouts, “You told me Barbossa killed my father and kidnapped Elizabeth!”

“I’m terribly sorry, but I lied,” replies Jack.  “I’m a PIRATE, that’s what we do.”

“Killed your father?” Barbossa interjects.  “No, not literally.  Years ago, I killed a UNIX process he was running in production, and he threw a temper tantrum then quit.  I just hired Elizabeth last week in order to help us overcome our DATA problems.”

“You are Jack Sparrow?” asks Elizabeth.  “You are, without doubt, the worst PIRATE I’ve ever heard of.”

“But you have heard of me,” replies Jack, proudly smiling.

“Security!” yells Barbossa.  “Please escort Mr. Sparrow out of the building—immediately!”

“That’s Captain Sparrow,” Jack retorts.  “And it’s too late, Barbossa!  I just restored the original systems and software.  Ha ha!  DATA and Magic Beans are no more!  Without doubt, this will earn my rightful reinstatement as the Captain of IT!”

“Oh no it won’t,” Barbossa responds slowly, while staring at his monitor in disbelief.  “DATA and Magic Beans are gone alright, but The Curse of the Poor Data Quality remains!”

“The what?” asks Elizabeth.

“The Curse of the Poor Data Quality,” Barbossa angrily replies.  “All of our information assets are still completely inaccurate and totally useless as the basis for making any business decisions.  Therefore, we are still cursed with unresolved data quality issues!”

“What did you expect to happen?” remarks Will.  “Technology is never the solution to any problem.  Technology is the problem.  And unabated advancements in technology will eventually lead to computers becoming self-aware and taking over the world.”

Laughing, Barbossa asks, “You do realize that only happens in really bad movies, right?”

“No, curses only happen in really bad movies,” replies Will.  “Sentient computers taking over the world is really going to happen.  After all, it was very clearly explained in that excellent documentary series produced by the governor of California.”

“Oh, shut up Will!” shouts Elizabeth.  “I don’t won’t to hear another one of your anti-technology rants!  That’s why I broke up with you in the first place.  Although technology didn’t cause the data quality problems, Luddite Will is right about one thing, technology is not the solution.”

“What in blazes are you talking about?” Jack and Barbossa retort in unison.

“Seriously, I actually have to explain this?” replies Elizabeth.  “After all, the name of this corporation is Es el Pueblo Estúpido!”

Jack, Barbossa, and Will just stare at Elizabeth with puzzled looks on their faces.

“It’s Spanish for,” explains Elizabeth, “It’s the People, Stupid!”

“Well, we don’t speak Spanish,” Barbossa and Jack reply.  “The only languages we speak are Machine Language, FORTRAN, LISP, COBOL, PL/I, BASIC, Pascal, C, C++, C#, Java, JavaScript, Perl, SQL, HTML, XML, PHP, Python, SPARQL . . .”

“Enough!” Elizabeth finally screams. 

“The point that I am trying to make is that although people, business processes, and yes, of course, technology, are all important for successful data quality management, by far the most important of all is . . . Do I really have to say it one more time?”

“It’s the People, Stupid!”

“This corporation should really be renamed to Todos los hombres son idiotas!” Elizabeth concludes, while shaking her head and looking at the clock.  “We can discuss all of this in more detail next week after I return from my Labor Day Weekend vacation.”

“You’re going away for Labor Day Weekend?” asks Will cheerily.  “Perhaps you would be so kind as to invite me to join you?”

“It’s a good thing you’re cute,” replies Elizabeth.  “Yes, you’re invited to join me, but you’ll have to carry my purse—all weekend.”

“Can we pretend,” Will says, grimacing as he reluctantly accepts her purse, “that I am carrying your laptop computer bag?”

“Oh sure, why not?” replies Elizabeth sarcastically with a sly smile.  “And while we’re at it, let’s all just continue pretending that the key to ongoing data quality improvement isn’t focusing more on people, their work processes, and their behaviors . . .”

Related Posts

Data Quality is People!

The Tell-Tale Data

There are no Magic Beans for Data Quality

Do you believe in Magic (Quadrants)?

Data Quality is not a Magic Trick

The Tooth Fairy of Data Quality

Which came first, the Data Quality Tool or the Business Need?

Predictably Poor Data Quality

The Scarlet DQ

The Poor Data Quality Jar

Wednesday Word is an OCDQ regular segment intended to provide an occasional alternative to my Wordless Wednesday posts.  Wednesday Word provides a word (or words) of the day, including both my definition and an example of recommended usage.

Quality-ish

Definition – Similar to truthiness, which my mentor Sir Dr. Stephen T. Colbert, D.F.A. defines as “truth that a person claims to know intuitively from the gut without regard to evidence, logic, intellectual examination, or facts,” quality-ish is defined as the quality of the data that an organization is using as the basis to make its critical business decisions without regard to performing data analysis, measuring completeness and accuracy, or even establishing if the data has any relevance at all to the critical business decisions being based upon it.

Example – “At today’s press conference, the CIO of Acme Marketplace Analytics heralded data-driven decision-making as the company’s key competitive differentiator.  In related news, the stock price of Acme Marketplace Analytics fell to a record low after their new quality-ish report declared the obsolesce of iTunes based on the latest Betamax videocassette sales projections.”

Is your organization basing its critical business decisions upon high quality data or highly quality-ish data?

Related Posts

The Circle of Quality

Is your data complete and accurate, but useless to your business?

Finding Data Quality

The Dumb and Dumber Guide to Data Quality

Wednesday Word: June 23, 2010 – Referential Narcissisity

Wednesday Word: June 9, 2010 – C.O.E.R.C.E.

Wednesday Word: April 28, 2010 – Antidisillusionmentarianism

Wednesday Word: April 21, 2010 – Enterpricification

Wednesday Word: April 7, 2010 – Vendor Asskisstic

Although The Rolling Forecasts recently got the band back together for the Data Rock Star World Tour, the tour scheduling (as well as its funding and corporate sponsorship) has encountered some unexpected delays. 

For now, please enjoy the following lyrics from another one of our greatest hits—this one reflects our country music influences.

A Record Named Duplicate *

My data quality consultant left our project after month number three,
And he didn’t leave much to my project team and me,
Except this old laptop computer and a bunch of empty bottles of beer.
Now, I don’t blame him ‘cause he run and hid,
But the meanest thing that he ever did,
Was before he left, he went and created a record named “Duplicate.”

Well, he must of thought that it was quite a joke,
But it didn’t get a lot of laughs from any executive management folk,
And it seems I had to fight that duplicate record my whole career through.
Some Business gal would giggle and I’d get red,
And some IT guy would laugh and I’d bust his head,
I tell ya, life ain’t easy with a record named “Duplicate.”

Well, I became a data quality expert pretty damn quick,
My defect prevention skills become pretty damn slick,
And I worked hard everyday to keep my organization’s data nice and clean.
I came to be known for my mean Data Cleansing skills and my keen Data Gazing eye,
And realizing that business insight was where the real data value lies,
As I roamed our data, source to source, I became the Champion of our Data Quality Cause.

But as I collected my fair share of accolades and battle scars, I made a vow to the moon and stars,
That I’d search all the industry conferences, the honky tonks, and the airport bars,
Until I found that data quality consultant who created a record named “Duplicate.”

Well, it was the MIT Information Quality Industry Symposium in mid-July,
And I just hit town and my throat was dry,
So I thought I’d stop by Cheers and have myself a brew.
At that old saloon on Beacon Street,
There at a table, escaping from the Boston summer heat,
Sat the dirty, mangy dog that created a record named “Duplicate.”

Well, I knew that snake was my old data quality consultant,
From the worn-out picture next to his latest Twitter tweet,
And I knew those battle scars on his cheek and his Data Gazing eye.
He was sitting smugly in his chair, looking mighty big and bold,
And as I looked at him sitting there, I could feel my blood running cold.

And I walked right up to him and then I said: “Hi, do you remember me?
On this USB drive in my hand, is some of the dirtiest data you’re ever gonna see,
You think the dirty, mangy likes of you could challenge me at Data Quality?”

Well, he smiled and he took the drive,
And we set up our laptops on the table, side by side.
We data profiled, re-checked the business requirements, and then we data analyzed,
We data cleansed, we standardized, we data matched, and then we re-analyzed.

I tell ya, I’ve fought tougher data cleansing men,
But I really can’t say that I remember when.
I heard him laugh and then I heard him cuss,
And I saw him conquer data defects, then reveal business insight, all without a fuss.

He went to signal that he was done, but then he noticed that I had already won,
And he just sat there looking at me, and then I saw him smile.

Then he said: “This world of Data Quality sure is rough,
And if you’re gonna make it, you gotta be tough,
And I knew I wouldn’t be there to help you along.
So I created that duplicate record and I said goodbye,
I knew you’d have to get tough or watch your data die,
But it’s that duplicate record that helped to make you strong.”

He said: “Now you just fought one hell of a fight,
And I know you hate me, and you got the right,
To tell me off, and I wouldn’t blame you if you do.
But you ought to thank me before you say goodbye,
For your mean Data Cleansing skills and your keen Data Gazing eye,
‘Cause I’m the son-of-a-bitch that helped you realize you have a passion for Data Quality.”

I got all choked up and I realized I should really thank him for what he'd done,
And then he said he could use a beer and I said I’d buy him one,
So we walked over to the Bull & Finch and we had our selves a brew.
And I walked away from the bar that day with a totally different point of view.

I still think about him, every now and then,
I wonder what data he’s cleansing, and wonder what data he’s already cleansed.
But if I ever create a record of my own, I think I’m gonna name it . . .
“Golden” or “Best” or “Survivor”—anything but “Duplicate”—I still hate that damn record!

___________________________________________________________________________________________________________________

* In 1969, Johnny Cash released a very similar song called A Boy Named Sue.

Related Posts

Data Rock Stars: The Rolling Forecasts

Data Quality is such a Rush

Data Quality is Sexy

Imagining the Future of Data Quality

The Very Model of a Modern DQ General

Wednesday Word is an OCDQ regular segment intended to provide an occasional alternative to my Wordless Wednesday posts.  Wednesday Word provides a word (or words) of the day, including both my definition and an example of recommended usage.

Referential Narcissisity

Definition – When referential integrity is enforced, a relational database table’s foreign key columns must only contain data values from their parent table’s primary key column, but referential narcissisity occurs when a table’s foreign key columns refuse to acknowledge data values from their alleged parent table—especially when the parent table was created by another DBA.

Example – The following scene is set on the eighth floor of the Nemesis Corporation, where within the vast cubicle farm of the data architecture group, Bob, a Business Analyst struggling with an ad hoc report, seeks the assistance of Doug, a Senior DBA.

Bob: “Excuse me, Doug.  I don’t mean to bother you, I know you are a very busy and important man, but I am trying to join the Sales Transaction table to the Customer Master table using Customer Key, and my queries always return zero rows.”

Doug: “That is because although Doug created the Sales Transaction table, the Customer Master table was created by Craig.  Doug’s tables do not acknowledge any foreign key relationships with Craig’s tables.  Doug is superior to Craig in every way.  Doug’s Kung Fu is the best—and until Craig publicly acknowledges this, your joins will not return any rows.”

Bob: “Uh, why do you keep referring to yourself in the third person?”

Doug: “Doug is bored with this conversation now.  Be gone from my sight, lowly business analyst.  You should be happy that Doug even acknowledged your presence at all.” 

Related Posts

Wednesday Word: June 9, 2010 – C.O.E.R.C.E.

Wednesday Word: April 28, 2010 – Antidisillusionmentarianism

Wednesday Word: April 21, 2010 – Enterpricification

Wednesday Word: April 7, 2010 – Vendor Asskisstic

Although you don’t have to actually wait until you are riding in an elevator with a member of executive management to use it, every data quality professional needs to have a well-rehearsed and highly effective elevator pitch ready to go for convincing your organization’s business stakeholders and financial decision makers of the importance of data quality initiatives.

In this blog post, I wanted to provide a few examples of what definitely won’t work as an effective elevator pitch for data quality.

The Five Worst Elevator Pitches for Data Quality

1. “I’m ramping up my job search because I hate working here so much, and my headhunter really thinks a data quality project would look great on my résumé, so how about you be a good sport and approve one?  Additionally, could you make me the leader of the project, and give me some awesome sounding title that would look great in the sans-serif font.”

    

2. “About 23% of the columns in our operational databases are NULL 42% of the time, and 18% of the fields in our analytical reports contain inconsistent formats 35% of the time, and duplicate rates for customer names and postal addresses vary from 8% to 16% depending on who you ask.  I don’t know what any of that means in business terms, but it can’t be good.”

    

3. “Like everybody, on, like, Twitter, Facebook, YouTube, and, like, most of the blogosphere, keeps saying data quality is, like, kinda really important, like some kinda best practice or something.  So I was wondering if, like, you could give us, like, oh I don’t know, like, a couple million dollars, so we could like, do a data quality project or something.  Yeah, like, that would be like, really cool of you, and I would, like totally, like say so on Twitter and Facebook and my blog, like, for real, totally.”

    

4. “I just came back from a major industry conference, and every one of the conference speakers, industry thought leaders, international experts, hardware, software, and consulting vendors were in unanimous and unambiguous agreement—that everything we’re currently doing is totally wrong.  We need to invest in a new master data management, data governance, and business intelligence center of excellence—all built upon a solid data quality foundation.  And it should only cost us about one billion dollars—not counting the annual maintenance fees, of course.”

    

5. “All of us down in IT are so bored maintaining the existing systems you use for those reports that contain made up data more than half the time anyway, so we’d like you to buy a bunch of cool new technology for us to play with.  Pick us up a few new data profiling and data cleansing tools, one of those master data management things everybody’s talking about, and throw in one of those data warehouse appliances too.  Oh, by the way, the enterprise data warehouse just went down and we’re pretty sure that thing’s never coming back up again.  Well anyway, have a great weekend, executive dude.”

Let’s hear your elevator pitch for data quality

Surely, you could do better—or even better, maybe you could do worse—than these five silly examples. 

Please share your (seriously effective or seriously funny) elevator pitch for data quality by posting a comment below.

Related Posts

Data, data everywhere, but where is data quality?

The Circle of Quality

Why is data quality important?

DQ-Tip: “Data quality is about more than just improving your data...”

Poor Data Quality is a Virus

All I Really Need To Know About Data Quality I Learned In Kindergarten