DQ-Tip: “There is no point in monitoring data quality…”

Data Quality (DQ) Tips is an OCDQ regular segment.  Each DQ-Tip is a clear and concise data quality pearl of wisdom.

“There is no point in monitoring data quality if no one within the business feels responsible for it.” 

This DQ-Tip came from the Enterprise Data World 2010 conference presentation Monitor the Quality of your Master Data by Thomas Ravn, MDM Practice Director at Platon.

It reminds me of a similar quote from Thomas Redman: “It is a waste of effort to improve the quality of data no one ever uses.” 

A common mistake made by those advocating that data needs to be viewed as a corporate asset is discussing data independent of both its use and its business relevance.  Additionally, Marty Moseley recently blogged about how data quality metrics often do a poor job in relaying the business value of data quality, strategic or otherwise.

Data profiling can help you begin to understand your data characteristics and usage.  A full data quality assessment can help you create the metrics that establish an initial baseline measurement, as well as continue monitoring your data quality over time.

However, without data quality metrics that meaningfully represent tangible business relevance, you should neither expect anyone within your organization to feel responsible for data quality, nor expect anyone to view data as a corporate asset.

 

Related Posts

DQ-Tip: “Start where you are...”

DQ-Tip: “Data quality is about more than just improving your data...”

DQ-Tip: “...Go talk with the people using the data”

DQ-Tip: “Data quality is primarily about context not accuracy...”

DQ-Tip: “Don't pass bad data on to the next person...”

Comic Relief: Dilbert on Project Management

For truly comic relief, there is perhaps no better resource than Scott Adams and the Dilbert comic strip.

Since I don’t read newspapers very often (does anyone anymore?), nowadays I get my Dilbert fix online.  However, I don’t always find the time to read the comic strip on a regular basis.  Therefore, I catch up by reading several weeks of it all at once. 

I like to find one or more recurring themes (which is very easy to do with Scott Adams) and then share some of my favorites.

Today’s blog post provides some recent Dilbert Views on the wonderful world of project management.  Enjoy!

Dilbert on Project Management

The first step in project management is proper planning, which starts by selecting a good acronym:

The next step is to properly establish realistic estimates for the primary tasks in the project plan:

Of course, the most important resource allocation is the project leader, so you must choose wisely:

Finalizing the delivery date can involve some tricky math, it’s much easier to just add more resources:

However if simply adding additional resources won’t really help, there is always an alternative approach:

 

Once it becomes impossible to meet the project’s carefully determined deadline, you know what must be done:

But fear not, your project can be brought to a graceful conclusion following this standard best practice:

Related Posts

Comic Relief: Dilbert to the Rescue

A Superb Lyrebird is a Superb Liar

The Superb Lyrebird is a small ground-dwelling Australian bird that is most notable for its superb ability to mimic almost any sound.  During an excellent special that I watched recently on the Discovery Channel, a Superb Lyrebird demonstrated this extraordinary ability by mimicking not only the sounds of many animals, which also included the human voice, but also various musical instruments, power tools such as drills and chainsaws, electronic devices such as car and fire alarms, and even some incredibly realistic sounding gunshots and explosions.

Male lyrebirds use this ability mainly during their song and dance courtship rituals.

As fascinating (well, I find it fascinating) as this information is, you are probably wondering why I am blogging about it. 

No, despite the rumors circulating the Twitterverse, I am not auditioning for my own primetime show on Animal Planet. 

However, I have recently been participating in the song and dance courtship ritual otherwise known as job interviews.

Résumés

I have always found the very concept of a résumé (or a curriculum vitae or far more often nowadays, a LinkedIn profile) to be truly fascinating.  The idea that a well-written document (printed, electronic, or online) that provides a mixture of summarized and detailed information about your professional experience, career goals, job history, academic qualifications, and references, can somehow encapsulate what kind of employee you would make is highly specious—at least in my humble opinion.

I think that the Superb Lyrebird is an excellent metaphor for a résumé because the job seeker is essentially attempting to mimic the sounds that the employer wants to hear.  Do you have an academic degree in a discipline relevant to the job opening?  If not, did you at least graduate from a prestigious college or university?  Does your job history include professional experience relevant to the job opening?  If not, did you at least have some past jobs with either impressive descriptions or titles?  Are your career goals ambitious enough—but not so ambitious that they could be considered potentially threatening to your new direct manager?

I am not suggesting that these questions are completely irrelevant, nor am I suggesting that some level of screening can’t be effectively performed using them.  However, is it really difficult to make sure that your résumé at least sounds good?

Gaming the System

Although you can’t embellish your education, you can easily get quite creative with the rest, such as using the right keywords in your job descriptions.  A cursory review (either manual or automated) of keywords is still a very common practice performed by human resources (HR) during the preliminary screening to determine what résumés will reach the desks of hiring managers.

So it would seem that “gaming the system” is what you have to do if you want to secure gainful employment.  In fact, it could be easily argued that the system is purposely designed to be gamed.

This is akin to my university literature professor not really caring what I actually thought about Don Quixote.  If I wanted to pass the final exam, then I had to mimic the professor’s belief that Miguel de Cervantes intended his wonderful novel to be an allegory for the critical but sometimes dangerous role that an active imagination can play in the human experience.

Telling my literature professor what he wanted to hear doesn’t mean that I truly appreciated or even understood the brilliance of the novel.  Although I gained the experience of reading it, passed a course that contributed to my graduation, and can sound good at a dinner party where guests have an interest in discussing the novel with me, does that really qualify me as an expert?

I can play buzzword bingo with the best of the best.  I can quote from the books and blogs of industry thought leaders.  I can customize my résumé so its loaded with all the right keywords.  I can use my Internet prowess to wow you during a telephone interview by using Google and Wikipedia to sound like the smartest man on the planet.  I can cram for the in-person interview like I crammed for my literature final exam because if I do my research well, I will know every question you are going to ask, and I will know exactly how you want me to answer them.

A Superb Lyrebird is a Superb Liar

Just like a Superb Lyrebird convincingly mimicking a lion only makes it sound like a lion, and convincingly mimicking what my professor wants to hear only makes me sound like a great student, convincingly mimicking what you are looking for only makes me sound like a potentially great employee.  But how many “lions on paper” or “lions during the interview” have you or your organization hired only to end up with a mostly flightless bird incapable of doing anything other than sounding impressive?

The reason that this happens is incredibly simple—a Superb Lyrebird is a Superb Liar.

However, my point is not to suggest that either job seekers are deceptive or that employers are easily deceived. 

My point is I believe that the system is fundamentally broken because it actually encourages job seekers to act like lyrebirds and actually encourages employers to hire lyrebirds.

In my career, I have been on both sides of the interview desk.  I have made hiring recommendations that resulted in terrible employees, as well as disagreed with hiring decisions that resulted in excellent employees.  I have performed poorly during interviews that resulted in getting hired anyway, as well as performed brilliantly during interviews that resulted in no offer. 

I acknowledge that some truly qualified people, who would make great employees, simply do not interview well.  Some people (including so-called “professional students”) excel at interviews (and in the classroom), but at absolutely nothing else.  Also, some interviewers simply do not know how to conduct a truly effective interview (or in some cases, how to conduct a legal interview).

Therefore, I completely accept that there is no way to perfect the process (and that I am also making sweeping generalizations).

Tilting at Windmills

Recently I have been very disappointed with both the questions that I have and have not been asked during an interview. 

I have also been very disappointed to observe interviewers getting frustrated with me for telling them the truth as opposed to telling them what they wanted to hear. 

Perhaps I should just play along like a good little Superb Lyrebird?  It certainly sounds like that is what is expected of me.

After all, The Ingenious Hidalgo Don Quixote of La Mancha is really an allegory about deception, both self-deception and the deception imposed on us by others—and about acknowledging not only the negative, but also the positive aspects of deception.

My good friend Sancho has just arrived, meaning it’s time once again to do battle with the hulking giants and try to slay them. 

Even though I know that I am really only tilting at windmills, for whatever reason, it still always makes me feel better anyway.

Wednesday Word: April 28, 2010

Wednesday Word is an OCDQ regular segment intended to provide an occasional alternative to my Wordless Wednesday posts.  Wednesday Word provides a word (or words) of the day, including both my definition and an example of recommended usage.

Antidisillusionmentarianism

Definition – A corporate philosophy opposed to freeing executive management from any of their own illusions or false beliefs.

Example – “I explained that we have serious data quality problems and most of them have business process or people issues as their root causes, and that according to every industry data governance maturity model, our organization was very undisciplined, and that buying more technology wasn’t the solution—and then the CEO fired me for violating antidisillusionmentarianism.”

Related Posts

Wednesday Word: April 21, 2010 – Enterpricification

Wednesday Word: April 7, 2010 – Vendor Asskisstic

Commendable Comments (Part 6)

Last September, and on the exact day of the sixth mensiversary (yes, that’s a real word, look it up) of my blog, I started this series as an ongoing celebration of the truly commendable comments that I regularly receive from my heroes—my readers.

Commendable Comments

On The Circle of Quality, Kelly Lautt commented:

“One of the offerings I provide as a consultant is around data readiness specifically for BI.  Sometimes, you have to sneak an initial data quality project into a company tightly connected to a project or initiative with a clear, already accepted (and budgeted) ROI.  Once the client sees the value of data quality vis a vis the BI requirements, it is easier to then discuss overall data quality (from multiple perspectives).

And, I have to add, I do feel that massive, cumbersome enterprise DQ programs sometimes lose the plot by blindly ‘improving’ data without any value in sight.  I think there has to be a balance between ignoring generalized DQ versus going overboard when there will be a diminishing return at some point.

Always drive effort and investment in any area (including DQ) from expected business value!”

On The Poor Data Quality Jar, Daragh O Brien commented:

“We actually tried to implement something like this with regard to billing data quality issues that created compliance problems.  Our aim was to have the cost of fixing the problem borne by the business area which created the issue, with the ‘swear jar’ being the budget pool for remediation projects.

We ran into a few practical problems:

1) Many problems ultimately had multiple areas with responsibility (line-of-business workers bypassing processes, IT historically ‘right-sizing’ scope on projects, business processes and business requirements not necessarily being defined properly resulting in inevitable errors)

2) Politics often prevented us from pushing the evidence we did have too hard as to the weighting of contributions towards any issue.

3) More often than not it was not possible to get hard metrics on which to base a weighting of contribution, and people tended to object to being blamed for a problem that was obviously complex with multiple inputs.

That said, the attempt to do it did help us to:

1) Justify our ‘claims’ that these issues were often complex with multiple stakeholders involved.

2) Get stakeholders to think about the processes end-to-end, including the multiple IT systems that were involved in even the simplest process.

3) Ensure we had human resources assigned to projects because we had metrics to apply to a business case.

4) Start building a focus on prevention of defect rather than just error detection and fix.

We never got around to using electric shocks on anyone.  But I’d be lying if I said it wasn’t a temptation.”

On The Poor Data Quality Jar, Julian Schwarzenbach commented:

“As data accuracy issues in some cases will be identified by front line staff, how likely are they going to be to report them?  Whilst the electric chair would be a tempting solution for certain data quality transgressions, would it mean that more data quality problems are reported?

This presents a similar issue to that in large companies when they look at their accident reporting statistics and reports of near misses/near hits:

* Does a high number of reported accidents and near hits mean that the company is unsafe, or does it mean that there are high levels of reporting coupled with a supportive, learning culture?

* Does a low number of reported accidents and near hits mean that the company is safe, or does it mean that staff are too scared of repercussions to report anything?

If staff risk a large fine/electric shock for owning up to transgressions, they will not do it and will work hard to hide the evidence, if they can.

In organizational/industrial situations, there are often multiple contributing factors to accidents and data quality problems.  To minimize the level of future problems, all contributory causes need to be identified and resolved.  To achieve this, staff should not be victimized/blamed in any way and should be encouraged to report issues without fear.”

On The Scarlet DQ, Henrik Liliendahl Sørensen commented:

“When I think about the root causes of many of the data quality issues I have witnessed, the original data entry was actually made in good faith by people trying to make data fit for the immediate purpose of use.  Honest, loyal, and hardworking employees striving to get the work done.

Who are the bad guys then?  Either it is no one or everyone or probably both.

When I have witnessed data quality problems solved it is most often done by a superhero taking the lead in finding solutions.  That superhero has been different kinds of people.  Sometimes it is a CEO, sometimes a CFO, sometimes a CRM-manager, sometimes it is anyone else.”

On The Scarlet DQ, Jacqueline Roberts commented:

“I work with engineering data and I find that the users of the data are not the creators of data, so by the time that data quality is questioned the engineering project has been completed, the engineering teams have been disbanded and moved on to other projects for other facilities. 

I am sure that if the engineers had to put the spare part components on purchasing contracts for plant maintenance, the engineers would start to understand some of the data quality issues such as incomplete part numbers or descriptions, missing information, etc.”

On The Scarlet DQ, Thorsten Radde commented:

“Is the question of ‘who is to blame’ really that important?

For me, it is more important to ask ‘what needs to be done to improve the situation.’

I don’t think that assigning blame helps much in improving the situation.  It is very rare that people cooperate to ‘cover up their mistakes.’  I found it more helpful to point out why the current situation is ‘wrong’ and then brainstorm with people on what can be done about it - which additional conventions are required, what can be checked automatically, if new functionality is needed, etc.

Of course, to be able to do that, you’ve got to have the right people on board that trust each other - and the blame game doesn’t help at all.  Maybe you need a ‘blame doll’ that everyone can beat in order to vent their frustrations and then move on to more constructive behavior?”

On Can Enterprise-Class Solutions Ever Deliver ROI?, James Standen commented:

“Fantastic question.  I think the short answer of course as always is ‘it depends’.

However, what’s important is exactly WHAT does it depend on.  And I think while the vendors of these solutions would like you to believe that it depends on the features and functionality of their various applications, that what it all depends on far more is the way they are installed, and to what degree the business actually uses them.

(Insert buzz words here like: ‘business process alignment’, ‘project ownership’, ‘Business/IT collaboration’)

But if you spend Gazillions on a new ERP, then customize it like crazy to ensure that none of your business processes have to change and none of your siloed departments have to talk to each other (which will cost another gazillion in development and consulting by the way), which will then ensure that ongoing maintenance and configuration is more expensive as well, and will eliminate any ability to use pre-built business intelligence solutions etc., etc.  Your ROI is going to be a big, negative number.

Unfortunately, this is often how it’s done.  So my first comment in this debate is - If enterprise systems enable real change and optimization in business processes, then they CAN have ROI.  But it’s hard. And doesn't happen often enough.”

On Microwavable Data Quality, Dylan Jones commented:

“Totally agree with you that data cleansing has been by far the most polarizing topic featured on our site since the launch.  Like you, I agree that data governance is a marathon not a sprint but I do object to a lot of the data cleansing bashing that goes on.

I think that sometimes we should give people who purchase cleansing software far more credit than many of the detractors would be willing to offer.  In the vast majority of cases data cleansing does provide a positive ROI and whilst some could argue it creates a cost base within the organization it is still a step in the direction of data quality maturity.

I think this particular debate is going to run and run however so thanks for fanning the flames.”

On The Challenging Gift of Social Media, Crysta Anderson commented:

“This is the biggest mindshift for a lot of people.  When we started Social Media, many wanted to build our program based only on the second circle - existing customers.  We had to fight hard to prove that the third circle not only existed (we had a hunch it did), but that it was worth our time to pursue.  Sure, we can't point to a direct sales ROI, but the value of building a ‘tribe’ that raises the conversation about data quality, MDM, data governance and other topics has been incredible and continues to grow.”

Thank You

Thank you all for your comments.  Your feedback is greatly appreciated—and truly is the best part of my blogging experience.

Since there have been so many commendable comments, please don’t be offended if one of your comments wasn’t featured. 

Please keep on commenting and stay tuned for future entries in the series.

Related Posts

Commendable Comments (Part 5)

Commendable Comments (Part 4)

Commendable Comments (Part 3)

Commendable Comments (Part 2)

Commendable Comments (Part 1)

Follow OCDQ

For more blog posts and commendable comments, subscribe to OCDQ via my RSS feed, my E-mail updates, or Google Reader.

You can also follow OCDQ on Twitter, fan the Facebook page for OCDQ, and connect with me on LinkedIn.

The Challenging Gift of Social Media

I recently finished reading (and also highly recommend) the excellent book Linchpin: Are You Indispensable? by Seth Godin. 

Although it’s not the subject of the book, in this blog post I’ll focus on one of its concepts that is very applicable to social media. 

The Circles of the Gift System

Godin uses the term “Gift Culture” to describe an emerging ethos facilitated by (but not limited to) the Internet and social media, which involves what he calls “The Circles of the Gift System” that I have attempted to represent in the above diagram.

In the first circle are your true real-world friends and family, the people that you would never interact with on the basis of trying to make money (i.e., the people you freely give “true gifts” while expecting nothing in return).

In the second circle are your customers and clients, the people that you conduct commerce with and who must pay you for your time, products, and services (i.e., the people and organizations you don’t give gifts because you need them to help pay your bills).

In the third circle is the social media and extended (nowadays mostly online) community, where following the freemium model, you give freely so that you can reach as many people as possible.  It is in the third circle that you assemble your tribe comprised of blog readers, Twitter followers, Facebook fans, and other “friendlies” — the term Godin uses for our social media connections.

It is the third circle that many (if not most) people struggle with and often either resist or ignore.  However, as Godin explains:

“This circle is new.  It’s huge and it’s important, because it enables you to enlarge the second circle and make more money, and because it enables you to affect more people and improve more lives.” 

However, dedicating the necessary time and effort to enlarge the third circle doesn’t guarantee you will enlarge the second circle, which risks turning freemium into simply free.  It is on this particular aspect that I will focus the remainder of my blog post.

The Intriguing Opportunity of Social Media

It is difficult to imagine a business topic generating more widespread discussion these days than social media.  That’s not to say that it is (or that it even should be) considered the most important topic.  However, almost every organization as well as most individual professionals have at the very least considered getting involved with social media in a business context.

The intriguing opportunity of social media is difficult to ignore—even after you ignore most of the hype (which is no easy task).

But as I wrote in the Social Karma series, if we are truly honest, then we all have to admit that we have the same question:

“What’s in this for me?”

Using social media effectively can definitely help promote you, your expertise, your company, and its products and services.  The primary reason I started blogging was to demonstrate my expertise and establish my authority with regards to data quality and its related disciplines.  As an independent consultant, I am trying to help sell my consulting, speaking, and writing services.

The Sobering Reality of Social Media

A social media strategy focused entirely on your own self-promotion will be easily detected by the online community, and could therefore easily result in doing far more harm than good.  Effectively using social media for business requires true participation, sustained engagement, and making meaningful contributions to the community’s goals—and not just your own.

The sobering reality of social media is that it’s not something you can simply do whenever it’s convenient for you.

Using social media effectively, more than anything else, requires a commitment that is mostly measured in time.  It requires a long-term investment in the community, and the truth is you must be patient because any returns on this investment will take a long time to materialize. 

If you are planning on a quick get in, get out, short-term marketing campaign requiring little effort, then don’t waste your time, but much more importantly, don’t waste the community’s time.

The Challenging Gift of Social Media

Godin opens his chapter on “The Powerful Culture of Gifts” by joking that he must have been absent the day they taught the power of unreciprocated gifts at Stanford business school. 

In fact, it’s probably a safe bet that the curriculum at most business schools conveniently ignores the fifty thousand year tradition of human tribal economies based on mutual support and generosity, when power used to be about giving, not getting.

Although we maintain some semblance of this tribal spirit in our personal lives with respect to the first circle, when it comes to our professional lives in the second circle, we want money for our time, product, or service—and we usually don’t come cheap.

Therefore, by far the most common question that I get asked (and that I often ask myself) about social media is:

“Is it really worth all that time and effort, especially when you aren’t getting paid for it?”

Although I honestly believe that it is, truthfully there have been many times when I have doubted it.  But those were usually times when I allowed myself to give in to the natural tendency we all have to become hyper-focused on our own goals. 

The paradox is that the best way to accomplish our selfish goals is—first and foremost—to focus on helping others. 

Of course, helping others doesn’t guarantee they’ll reciprocate, especially with financial returns on our social media investment.  Returning to Godin’s analogy, enlarging (or even just maintaining) the third circle doesn’t guarantee enlarging the second circle.

However, true service to the social media community requires giving true gifts to the third circle. 

Godin explains that these gifts—which do not demand reciprocation—turn the third circle into your tribe.  Giving gifts fulfills your tribal obligation.  Recipients pay it forward by also giving gifts—but perhaps to another tribal member—and not back to you.

And this is the challenging gift of social media—it is a gift that you may keep on giving without ever getting anything in return.

Related Posts

Freemium is the future – and the future is now

Social Karma

True Service

Wednesday Word: April 21, 2010

Wednesday Word is an OCDQ regular segment intended to provide an occasional alternative to my Wordless Wednesday posts.  Wednesday Word provides a word (or words) of the day, including both my definition and an example of recommended usage.

Enterpricification

Definition – whereas “enterprisification” is a slang term used to describe scaling or otherwise evolving a technology or service to the point of being able to handle enterprise-level needs, enterpricification is simply increasing the price of a non-scalable or otherwise limited technology or service to the cost usually associated with an enterprise-class solution.

Example – “In a rare moment of honesty, the CTO of Acme Software admitted today that the only distinguishing characteristic of the recently released enterprise edition of their product is enterpricification.”

Related Posts

Wednesday Word: April 7, 2010

Can Enterprise-Class Solutions Ever Deliver ROI?

Data, data everywhere, but where is data quality?

“Two young fish are swimming along when they happen to meet an older fish swimming the other way, who nods at them and says: ‘Morning, boys.  How’s the water?’  And the two young fish swim on for a bit, then eventually one of them looks over at the other and goes: ‘What the hell is water?’”

The acclaimed novelist David Foster Wallace told that story during a speech he delivered at Kenyon College in 2005.  Although he certainly wasn’t speaking on the topic of data management, I believe that his story can easily be adapted as a data metaphor:

“Two young kids are walking along, tweeting and uploading new pictures to Facebook on their iPhones, when they happen to meet an older man walking the other way checking his work e-mail on his BlackBerry, who nods at them and says: ‘Morning, boys.  How’s the data?’  And the two young kids walk on for a bit, then eventually one of them looks over at the other and goes: ‘What the hell is data?’”

My point is that what once was a seemingly esoteric word (“data”) used mostly by computer geeks such as myself, has now so thoroughly pervaded mainstream culture that we hardly seem to notice we are quite literally swimming in data on a daily basis.

Why Data Matters

In his recent blog post, Rich Murnane was hit with the realization that data isn’t for data geeks anymore.  The post included an excellent IBM video (and commercial) about “Why Data Matters” that states every day we are creating fifteen petabytes of data, which is eight times as much data as there is in all of the libraries in the United States combined. 

Data matters because everything—and not just the rows in our relational databases and spreadsheets, but also our status updates from Facebook and Twitter, our blog posts, and even most of our daily conversations—is data. 

The growing challenge is can we extract meaningful insights from these vast and veritable oceans of unrelenting data volumes, and use those insights to make better decisions in near real-time in order to positively impact the various aspects of our lives.

Paradoxical Business Situation

Even in the business world, where data management used to be viewed solely as a concern for those computer geeks down in IT, more and more people all throughout more and more organizations are coming to view data as a strategic corporate asset.

In his recent Network World article Data Everywhere, But Not Enough Smart Management, Thomas Wailgum described the “data, data everywhere” phenomenon as “an awe-inspiring and unprecedented push and pull of data and information needs.”  Wailgum described the push as a growing surge of terabytes of data flooding enterprise systems and applications, and the pull as the growing demand from users for sweeping, individualized access to analytics and business information.

However, just because data is flowing everywhere doesn’t automatically mean that data quality is sure to follow.

Wailgum cites research from a recent Forbes survey where executives reported that the “bad data problem” is currently estimated to be costing their organizations between five and twenty million dollars annually, which leads him to ask the question:

“If everyone agrees on the strategic importance of data and information management, and everyone knows what the negative consequences are, then why are there still so many problems?” 

Wailgum calls this the “paradoxical business situation” and cited survey results indicating “fragmented data ownership” is the single biggest roadblock to successful enterprise information management.  Nearly 80% of IT managers said data quality was their responsibility, whereas nearly 75% of business (finance, sales, and marketing) managers said it was their responsibility.

“While IT managers largely concede that information is the users’ not theirs, they take the position that data and information management systems are under IT’s purview,” concludes the survey.  “This differing perspective puts IT and business executives in conflicting camps, particularly when it comes to data quality.”

This debate over data ownership reminded me of the great discussion sparked by a recent Henrik Liliendahl Sørensen blog post questioning whether “data owner” was a bad word.  Many commenters agreed that “data stewardship” was more relevant and that although data quality is a shared responsibility for the entire enterprise, corporate culture is far more challenging than what can amount to a largely semantic argument over the proper use of terminology such as “data ownership” or “ data stewardship.”

Why Data Quality Matters

As I posited in The Circle of Quality, an organization’s success is measured by the quality of its results, which are dependent on the quality of its business decisions, which rely on the quality of its information, which is based on the quality of its data. 

Therefore, data quality matters because high quality data serves as a solid foundation for business success.

Organizations are not only facing the challenging realities that data is everywhere and its burgeoning volumes continue to rise, but also that data is no longer limited to the traditional structured forms stored in relational databases.  Unstructured data from social media, the Internet, and mobile devices are contributing an abundant new source to the enterprise’s information ocean.

In The Rime of the Ancient Mariner, Samuel Taylor Coleridge wrote:

“Day after day, day after day,
We stuck, nor breath nor motion;
As idle as a painted ship
Upon a painted ocean.

Water, water, everywhere,
And all the boards did shrink;
Water, water, everywhere,
Nor any drop to drink.”

When data is abundant, but data quality remains scarce, then the thirst to acquire knowledge and insight remains unquenched, and data hangs like a heavy albatross around the enterprise’s neck.

Related Posts

The Circle of Quality

Beyond a “Single Version of the Truth”

Poor Data Quality is a Virus

DQ-Tip: “Don't pass bad data on to the next person...”

The Only Thing Necessary for Poor Data Quality

Hyperactive Data Quality (Second Edition)

The General Theory of Data Quality

Data Governance and Data Quality

The Data-Information Continuum

Follow OCDQ

If you enjoyed this blog post, then please subscribe to OCDQ via my RSS feed, my E-mail updates, or Google Reader.

You can also follow OCDQ on Twitter, fan the Facebook page for OCDQ, and connect with me on LinkedIn.

The Fellowship of #FollowFriday

During the dawn of the Second Age of Digital-Earth, in the land of Twitter there was formed a group of like-minded tweeps who were well known for their wisdom about Data Quality, Data Governance, Master Data Management, and Business Intelligence.

They battled against the dark forces of poor data quality, undisciplined organizations, multiple conflicting versions of the truth, flawed business decisions, vast boiling oceans of unmanaged data assets, uncontrolled costs, and unmitigated compliance risks.

Collectively, these valiant heroes were known as: The Fellowship of FollowFriday.

Okay, so clearly I am a total dork—geek, nerd, and dweeb are also completely acceptable alternatives.

J. R. R. Tolkien's The Lord of the Rings three-volume book and Peter Jackson’s adapted movie trilogy were awe inspiring epics, and also the theme of this blog post about FollowFriday, the weekly tradition of recommending great folks to follow on Twitter.

Please note that simply for the purposes of organizing the following lists, I have made the United States the kingdom of Gondor, Canada the kingdom of Rohan, and all of Europe collectively The Shire.  No offense intended to my tweeps from other lands.

I hope that everyone has a great FollowFriday and an even greater weekend.  See you all around the Twittersphere.

Tweeps of Gondor

• Phil Simon – @PhilSimon
• Rob Paller – @RobPaller
• Rich Murnane – @murnane
• Steve Sarsfield – @SteveSarsfield
• Baseline Consulting – @BaselineConsult
• Jill Dyché – @JillDyche
• Initiate, an IBM Company – @IBMInitiate
• Marty Moseley – @wmmarty
• Jacqueline Roberts – @JackieMRoberts
• Terri Rylander – @BIMarcom
• Dalton Cervo – @dcervo
• William Sharp – @dqchronicle
• April Reeve – @Datagrrl
• Corinna Martinez – @Futureratti
• Ted Louie – @TedLouie
• Garnie Bolling – @GarnieBolling
• Paul Boal – @PaulBoal
• Shawn Rogers – @shawnrog
• Dan Power – @dan_power
• David Loshin – @DavidLoshin
• Merv Adrian – @merv
• Robert Karel – @rbkarel
• Ted Friedman – @ted_friedman
• Loraine Lawson – @LoraineLawson
• DataFlux – @DataFlux
• Talend – @Talend

Tweeps of Rohan

• Jill Wanless – @sheezaredhead
• Datamartist – @Datamartist
• James Standen – @JamesStanden
• Karen Lopez – @datachick
• Rob Drysdale – @projmgr
• Indicee – @Indicee
• Geoff Devereux – @GeoffDevereux
• Alec Sharp – @AlecSharp
• Kelly Lautt– @KellyLautt

Tweeps of The Shire

• Dylan Jones – @DataQualityPro
• Henrik Liliendahl Sørensen – @hlsdk
• Charles Blyth – @CharlesBlyth
• Julian Schwarzenbach – @jschwa1
• Phil Wright – @faropress
• Daragh O Brien – @daraghobrien
• Ken O'Connor – @KenOConnorData
• Peter Thomas – @PeterJThomas
• Sarah Burnett – @SarahBurnett
• Thorsten Radde – @ThRadde
• Graham Rhind – @GrahamRhind
• Alexej Freund – @alexej_freund

Related Posts

Social Karma (Part 7)

The Wisdom of the Social Media Crowd

The Twitter Clockwork is NOT Orange

Video: Twitter #FollowFriday – January 15, 2010

Video: Twitter Search Tutorial

Live-Tweeting: Data Governance

Brevity is the Soul of Social Media

If you tweet away, I will follow

Tweet 2001: A Social Media Odyssey

The Spam Tax

Since 1955, April 15 has been “Tax Day” in the United States—the deadline for filing your state and federal income tax returns.

Therefore, it’s common for alternative taxation models to be discussed today.  For example, one such alternative is the FairTax. 

I would like to propose another alternative—The Spam Tax.

I Don’t Like Spam!

Although never a big fan of the “food” version of Spam, I am proposing a tax on the electronic version, as defined by Wikipedia:

“Spam is the abuse of electronic messaging systems to send unsolicited bulk messages indiscriminately.  While the most widely recognized form of spam is e-mail spam, the term is applied to similar abuses in other media: instant messaging spam, Usenet newsgroup spam, Web search engine spam, spam in blogs, wiki spam, online classified ads spam, mobile phone messaging spam, Internet forum spam, junk fax transmissions, social networking spam, television advertising and file sharing network spam.”

Can you even imagine how much money could be raised if we could find a viable way to tax spam?

Even conservative estimates indicate almost 80% of all e-mail sent world-wide is spam.  A similar percentage of blog comments are spam, and spam generating bots are quite prevalent on Twitter and other microblogging and social networking services.

Of course, I have absolutely no idea how we would actually implement The Spam Tax. 

Even if I did, Gelatinous Glaze (aka “The Spam Lobby” in Washington, D.C.) would demand a pound of chopped shoulder meat from every member of the United States Congress known to be under their influence (aka “in the tiny tin can of Big Spam”).

If only there was a way to start a grassroots movement that could convince our political leaders that now is the time for change.

Wait a minute!  I’ve got it!  Every one of us could send our Representatives and Senators an e-mail message!

Perhaps something like the following:

I Like Spam! (the Monty Python sketch)

No respectable discussion of spam can be said to be truly complete without the obligatory inclusion of the Monty Python sketch.

If you are having trouble viewing this video, then you can watch it on YouTube by clicking on this link: Spam (Monty Python)

Wordless Wednesday: April 14, 2010

Photos via Flickr (Creative Commons License) by: Thomas Hawk and freedryk

Microwavable Data Quality

Data quality is definitely not a one-time project, but instead requires a sustained program of enterprise-wide best practices that are best implemented within a data governance framework that “bakes in” defect prevention, data quality monitoring, and near real-time standardization and matching services—all ensuring high quality data is available to support daily business decisions.

However, implementing a data governance program is an evolutionary process requiring time and patience.

Baking and cooking also require time and patience.  Microwavable meals can be an occasional welcome convenience, and if you are anything like me (my condolences) and you can’t bake or cook, then microwavable meals can be an absolute necessity.

Data cleansing can also be an occasional (not necessarily welcome) convenience, or a relative necessity (i.e., a “necessary evil”).

Last year on Data Quality Pro, Dylan Jones hosted a great debate on the necessity of data cleansing, which is well worth reading, especially since the over 25 (and continuing) comments it received proves it is a polarizing topic for the data quality profession.

I reheated this debate (using the Data Quality Microwave, of course) earlier this year with my A Tale of Two Q’s blog post, which also received many commendable comments (but far less than Dylan’s blog post—not that I am counting or anything).

Similarly, a heated debate can be had over the health implications of the microwave.  Eating too many microwavable meals is certainly not healthy, but I have many friends and family who would argue quite strongly for either side of this “food fight.”

Both of these great debates can be as deeply polarizing as Pepsi vs. Coke and Soccer vs. Football.  Just for the official record, I am firmly for both Pepsi and Football—and by Football, I mean NFL Football—and firmly against both Coke and Soccer. 

Just as I advocate that everyone (myself included) should learn how to cook, but still accept the eternal reality of the microwave, I definitely advocate the implementation of a data governance program, but I also accept the eternal reality of data cleansing.   

However, my lawyers have advised me to report that beta testing for an actual Data Quality Microwave has not been promising.

Related Posts

A Tale of Two Q’s

Hyperactive Data Quality (Second Edition)

The General Theory of Data Quality

Follow OCDQ

If you enjoyed this blog post, then please subscribe to OCDQ via my RSS feed, my E-mail updates, or Google Reader.

You can also follow OCDQ on Twitter, fan the Facebook page for OCDQ, and connect with me on LinkedIn.

True Service

“It is not the urge to surpass all others at whatever cost, but the urge to serve others at whatever cost.”

— Arthur Ashe

Service Providers

As I wrote at the beginning of the year in my blog post declaring karma as my theme word for 2010, we all have some way of expressing the concept of what we expect to happen when we help other people—when we provide a service for them.

In one way or another, in both our professional and our personal lives, we are all service providers. 

The most commonly used differentiation between professional and personal service is the exchange of money.  Your employer pays you to do your job, and not because you’re a wonderful human being—even though you are.  Your friends and family will help you whenever necessary, and not because you’re paying them—nice dinners, birthday presents, and other gifts don’t count.

Service Contracts

Whether it is a formal written document or an informal social agreement, all service is based upon some type of contract.

Once again because the exchange of money is typically involved, a professional service most commonly uses a written document, whereas a personal service most commonly uses a social agreement, which is often unwritten and frequently also unspoken.

A written document details the terms of service, which is the service level agreement that contractually binds the service provider to whomever they are providing service.  This service contract allows the parties involved to discuss any dissatisfaction or dispute in a relatively straightforward and civilized manner—or if lawyers get involved, in a needlessly complex and expensive manner.

Professional service contracts tend to focus on the minimum requirement necessary to fulfill the contractual commitment and therefore normally do not engender either party to go beyond the specific terms since nothing would be explicitly gained.

However, the party that is paying (i.e., “the party of the second part” for my lawyer readers) will normally attempt to exert subtle pressure on “the party of the first part” (i.e., the service provider for my non-lawyer readers) to deliver above and beyond the minimum requirement dictated by the service contract.  This is one aspect of what I like to refer to as “service psychology.”

Service Psychology

First of all, everyone prefers to get as much as possible without paying anything.  And when you do have to pay for something, everyone wants at the very least to “get what you paid for” while getting more than what you paid for is considered even better. 

These truths are universal and they do not automatically turn all of us into bad people (or all companies into evil corporations).

We also usually want to provide good service whether or not we are being paid, but when we are, there is a general tendency to be concerned about providing value worthy of our compensation.  This is the aspect of service psychology that can cause us to be receptive to the subtle pressure to exceed the minimum requirement dictated by the service contract.

Employers use it on employees.  Customers use it on companies (or more precisely, on their customer service representatives).  Business partners use it on each other.  And of course, this aspect of service psychology can be reversed to exert subtle pressure for encouraging acceptance that the minimum requirement dictated by the service contract has already been met.

We can also condition ourselves to resist these subtle pressures and even claim that we are simply defending ourselves from being taken advantage of by the other party—and regardless of which “side” of the service contract we currently find ourselves.

Such “service psychological warfare” will sometimes escalate until the lawyers eventually come crashing through the skylight, rappelling down ropes with one hand, while holding the original signed copy of the service contract in their other hand, and quoting aloud the terms, conditions, warranties, and indemnification from page 13, section 8, sub-section 3, paragraph 5.

Social Agreements

Since money is typically not involved and barring a few exceptions (e.g., a divorce or a contested will) no lawyers come into play, we tend not to view our (often unwritten, unspoken) social agreements with friends and family as “personal service contracts.” 

However, the underlying principles of service psychology apply just as much to social agreements where perhaps paradoxically, we have both a much higher expectation for those that serve us and a much greater sense of obligation to those we serve.

Therefore, our social agreements truly are personal service contracts.  There are terms and conditions, minimum requirements, and constant measurement of our costs, risks, and returns.  We all have a natural tendency to “keep score” one way or another.

All Service is a Stage

All service is a stage, and all of us are merely players, each having our exits and entrances, and in our time playing many parts, some professional and some personal, in many “service dramas” seemingly fraught with equal potential for tragedy and comedy. 

Forgive the Shakespearean flourish, but all of our narratives tend towards the dramatic and service is certainly no exception. 

Although we can easily turn our professional services into a drama (even without rappelling lawyers crashing through skylights), our social agreements generally involve more “emotional service” and are therefore far more inclined to become dramatic.

However, all service dramas are often simply a crisis of perspective—specifically our preference for our own above all others.

True Service

I began this blog post using only the second sentence from a famous Arthur Ashe quote, which in its entirety reads:

“True heroism is remarkably sober, very undramatic.

It is not the urge to surpass all others at whatever cost, but the urge to serve others at whatever cost.”

Although I am very fond of the original wording, I will end this blog post by paraphrasing the full quote:

“True service is remarkably rare, very undramatic, totally unconcerned with personal benefit, and completely content to serve others at whatever cost.”

Related Posts

The Game of Darts – An Allegory

“I can make glass tubes”

My #ThemeWord for 2010: KARMA

The Scarlet DQ

The Scarlet DQ is the superhero name of Jill Wanless (aka sheezaredhead).

Just kidding—I would never reveal a superhero’s secret identity.

Although I was never a big fan of the book, the title of this blog post is inspired by The Scarlet Letter by Nathaniel Hawthorne, where the novel’s protagonist Hester Prynne is forced to wear The Scarlet Letter A as a badge of shame for committing the act of adultery, which lead to the birth of her daughter Pearl.

The book came to mind while I was reading the commendable comments received last week on The Poor Data Quality Jar, where a recurring theme was the valid criticism of the “public humiliation” aspect of having employees put money into the jar when they contribute to the organization's poor data quality.

Using such an approach to help organizations illustrate the costs of poor data quality is equivalent to making the offenders wear The Scarlet DQ as a badge of shame, which will only make it far more likely that data quality issues will not be reported at all.

But even without my “swear jar” inspired idea, I think that the fear of public humiliation is what prevents poor data quality from being acknowledged by many organizations, which often leads to a major data quality related crisis that “no one saw coming.” 

For example, if you are in need of some quiet time alone for taking a good power nap in a conference room, then try scheduling a meeting to discuss known data quality issues and their root causes.  If your organization is like most, then you could probably book one of those really nice conference rooms with the big comfy reclining chairs—because nobody will attend your meeting.

Data quality can be somewhat of a taboo topic.  Many organization assume that their data quality must be “good enough” otherwise “how could we possibly still be in business?”  Nobody likes to talk about data quality for one simple reason:

If data quality issues exist (and they do), then no one wants to be blamed for causing or failing to fix them.

It’s as if everyone is afraid that they will be forced to wear The Scarlet DQ.

This is one of the many human dynamics that can render even the best technology and proven methodology completely useless. 

What Say You?

Please post a comment and share your recommendations about how to foster an environment in which poor data quality can be reported freely without fear of blame or reprisal.  All viewpoints are welcome.  Nathaniel Hawthorne references are not required.

Related Posts

The Poor Data Quality Jar

The Third Law of Data Quality

The Dumb and Dumber Guide to Data Quality

Not So Strange Case of Dr. Technology and Mr. Business

You're So Vain, You Probably Think Data Quality Is About You

Follow OCDQ

If you enjoyed this blog post, then please subscribe to OCDQ via my RSS feed, my E-mail updates, or Google Reader.

You can also follow OCDQ on Twitter, fan the Facebook page for OCDQ, and connect with me on LinkedIn.

Wednesday Word: April 7, 2010

Wednesday Word is an OCDQ regular segment intended to provide an occasional alternative to my Wordless Wednesday posts.  Wednesday Word provides a word (or words) of the day, including both my definition and an example of recommended usage.

Vendor Asskisstic

Definition – whereas “vendor agnostic” describes a general methodology or solution that does not require the technology or services provided by a specific vendor, vendor asskisstic is the complete opposite.

Example – “Although we requested a vendor agnostic proposal from Acme Consulting, their recommendation was so blatantly vendor asskisstic that it might as well have been printed in Big Blue letters.”