This Sunday is February 14—Valentine's Day—the annual celebration of enduring romance, where true love is publicly judged according to your willingness to purchase chocolate, roses, and extremely expensive jewelry, and privately judged in ways that nobody (and please, trust me when I say nobody) wants to see you post on Twitter, Facebook, Flickr, YouTube, or your blog.
Valentine's Day is for people in love to celebrate their love privately in whatever way works best for them.
Valentine's Day is not for data.
However, when was the last time you showed your data how much you care?
Data needs love too.
Tainted Data
Sometimes, I am sure that you feel you've got to run away, you've got to get away from the pain that poor data quality has driven into the heart of your organization.
The data you share throughout the enterprise seems to have lost its light, for you toss and turn, you can't sleep at night.
Once you ran to data—you ran—now you run from data—this tainted data you've been given.
You feel you've given data all a person could give. It's taken your tears and that's not nearly all.
Oh, tainted data—tainted data.
You really want IT (if you're with the Business) or the Business (if you're with IT) to make things right.
And you think data quality just needs a one-time cleansing project for someone else to play.
But I'm sorry, data quality doesn't play that way.
Don't ignore data, please. It cannot stand the way you tease. Data loves you though you hurt it so.
Data doesn't want to pack its things and go.
It's not your data, it's you
The majority of data quality initiatives are reactive projects launched in the aftermath of an event when poor data quality negatively impacted decision-critical information.
Many of these projects end in failure. Some fail because of lofty expectations or unmanaged scope creep. Most fail because they are based on the flawed perspective that data quality problems can be permanently “fixed” by a one-time project as opposed to needing a sustained program.
Tactical initiatives will often have a necessarily narrow focus. Reactive data quality projects are sometimes driven by a business triage for the most critical data problems requiring near-term prioritization that simply can't wait for the effects that would be caused by implementing a proactive strategic initiative (i.e., one that may have prevented the problems from happening).
Even when a reactive data quality project is successful, it's success will only be short-lived.
Another project will be necessary when the organization is forced into triage once again during the next inevitable crisis where poor data quality negatively impacts decision-critical information.
One reactive project at a time will never do data quality right—because one is the loneliness number that you'll ever do.
Maybe you're just not that into your data?
Across the vast digital landscape of the Internet, I see you rolling your eyes because you know what's coming next—the talk.
That's right—it's time to talk about your relationship with data, about your need to take responsibility for data quality.
I see you hesitate. After all, nobody has a data governance ring on their finger, do they?
Data governance establishes policies and procedures to align people throughout the organization. Successful data quality initiatives require the Business and IT to forge an ongoing and iterative collaboration.
Neither the Business nor IT alone has all of the necessary knowledge and resources required to achieve data quality success.
The Business usually owns the data and understands its meaning and use in the day-to-day operation of the enterprise and IT usually owns the hardware and software infrastructure of the enterprise's technical architecture.
The Business and IT must partner together to define the necessary data quality standards and processes.
But maybe you previously attempted a data governance program or other initiative requiring Business-IT collaboration.
Perhaps harsh words were spoken, promises were broken, old wounds were opened, and collaboration walked out that door.
Are you too proud to make up? Are you ready to break up?
Or maybe you're just not that into your data?
I don't know much
Look at your data, I know its poor quality is showing. Look at your organization, you don't know where it's going.
So many questions still left unanswered, so much that's never broken through.
But the Business and IT were made for each other. Just like Data Governance and Data Quality were made for each other.
Just like you and your data were made for each other.
I don't know much, but I know data needs love too. And that may be all I need to know.
I had to say I Love Data Quality in a Blog Post
Well, I know it was kind of strange. I hope it made some sense to you. But what I had to say couldn't wait.
I know you will understand. Every time I tried to tell you, the words just came out wrong.
So, I had to say I love data quality in a blog post.
Maybe every time the time was right for you to start your data governance program, all your words just came out wrong.
Maybe you'll have to say you love data quality and you need a data governance program using this blog post?
Happy Valentine's Day to you and yours.
Happy Data Governance and Data Quality to you and your data.
However, in this blog post I want to channel the seldom tapped wisdom of Lloyd Christmas and Harry Dunne, from the Academy Award Eligible and American Cinema Classic – Dumb and Dumber.
The Dumb and Dumber Guide to Data Quality
“What the one doesn't have, the other is missing.”
Data and Quality—do they really need each other?
The Business and IT—do they really need to work together?
Isn't data quality an IT issue? After all, the data is stored in databases and applications that they manage. Therefore, if there are problems with the data, then IT is responsible for cleaning up their own mess. Aren't they?
Isn't data quality a Business issue? After all, the data is created by business processes and users that they manage. Therefore, if there are problems with the data, then the Business is responsible for cleaning up their own mess. Aren't they?
Listening to the Business and IT argue like this reminds me of Lloyd and Harry playing the game of Tag:
Lloyd: “You're it.”
Harry: “You're it.”
Lloyd: “You're it, quitsies!”
Harry: “Anti-quitsies, you're it, quitsies, no anti-quitsies, no startsies!”
Lloyd: “You can't do that!”
Harry: “Can too!”
Lloyd: “Cannot, stamp it!”
Harry: “Can too, double stamp it, no erasies!”
Lloyd: “Cannot, triple stamp, no erasies, touch blue make it true.”
Harry: “No, you can't do that . . . You can't triple stamp a double stamp! Lloyd!”
Lloyd [with hands over his ears]: “LA-LA LA-LA LA-LA!”
Harry: “LLOYD! LLOYD! LLOYD!”
Yes, the Business usually owns the data and understands its meaning and use in the day-to-day operation of the enterprise. And yes, IT usually owns the hardware and software infrastructure of the enterprise's technical architecture.
However, neither the Business nor IT alone has all of the necessary knowledge and resources required to truly be successful. Data quality requires that the Business and IT forge an ongoing and iterative collaboration.
Tag—you're both it! And executive management says: No quitsies!
Not every theory looks good in a tuxedo
“Hey, look, The Monkees. They were a huge influence on The Beatles.”
Without question, there are many theories available about how to properly execute a data quality initiative.
You read about them in critically acclaimed books. You hear about them in expert presentations at major industry conferences. You even sometimes see them published in blog posts underneath pictures of two weird looking dudes wearing wacky tuxedos.
Most theories include models describing an organization's evolution through a series of stages intended to measure its capability and maturity, tendency toward being reactive or proactive, and inclination to be project-oriented or program-oriented.
I am certainly an advocate of searching for sound theory and working with proven methodology.
But the harsh reality is there is no “one theory to rule them all” or one-size-fits-all methodology—and anyone who tells you otherwise should be treated with the same disdain as those who truly believe The Monkees were a huge influence on The Beatles.
You need to find something that will adapt to your organization's unique culture. Most important, you need to find something that will meet your organization wherever it happens to currently be within the capability and maturity model.
Just because some expert says you should be wearing a black Armani tuxedo with a crimson cummerbund and monogrammed cufflinks, doesn't mean you should. Maybe the bright orange or powder blue tuxedo with the frilly shirt, top hat, and cane is more your style. Or maybe it is the only thing currently in your size—or the only thing you can currently afford.
Rock whatever tuxedo (theory) fits you best today. Just remember—it's a rental. As your organization and the individual change agents leading the way mature and evolve, your wardrobe (culture) will become ready to evolve right along with it.
Only you can decide what theory works best for your organization—as well as when you're ready to take it to the next level.
Not every practice can be considered best
“Well, it's not gonna do us any good sitting here whining about it. We're in a hole. We're just going to have to dig ourselves out.”
So you have selected a theory and now you're ready to get to work. Every theory includes some recommended best practices.
However, if you are looking to follow a step-by-step, paint-by-numbers, only color inside the lines, fool-proof plan, then you are going to fail before you even begin.
Best practices simply provide a reference of recommended options of what proved successful for other data quality initiatives.
Best practices should be reviewed in order to determine what can be learned from them, as well as to select what you think will work in your environment and what simply won't. However, it often won't be easy to tell the difference.
The key word in “best practice” is practice—and not best, as in the perfectly stupid phrase: “practice makes perfect.”
Real practice doesn't make perfect. Real practice is messy. Real practice colors with the red crayon much more often than with the green crayon. Real practice doesn't color inside the lines—it draws on the walls.
In other words, you're going to make mistakes—lots and lots and lots—of mistakes.
And not because you are dumb—or dumber than others who successfully followed the same recommendations.
Not even best practices make perfect because nobody works at a company called Perfect, Incorporated. Through trial and error you will figure out what works bestfor you and your organization, and those practices will become your best practices.
Couldn't we get by just fine without data quality?
“So you're telling me there's a chance...”
You may be thinking that a data quality initiative sounds like a lot of work. You may be wondering if it is really worth the investment of all that time, effort, and money. You may be asking if you really need a data quality initiative.
Couldn't we get by just fine without data quality?
The following dialogue between Lloyd and Mary Swanson provides a better answer than anything else I can imagine:
Lloyd: “What do you think the chances are of us getting by just fine without data quality?”
Mary: “Well, Lloyd, that's difficult to say. I mean, we don't really...”
Lloyd: “Hit me with it! Just give it to me straight! What are the chances?”
Mary: “Not good.”
Lloyd: “You mean not good like one out of a hundred?”
A “blog-bout” is a good-natured debate between two bloggers. This blog-bout is between Jim Harris and Phil Simon, where they debate which board game is the better metaphor for an Information Technology (IT) project: “Risk” or “Monopoly.”
IT projects and “Risk” have a great deal in common. I thought long and hard about this while screaming obscenities and watching professional sports on television, the source of all of my great thinking. I came up with five world dominating reasons.
1. Both things start with the players marking their territory. In Risk, the game begins with the players placing their “armies” on the territories they will initially occupy. On IT projects, the different groups within the organization will initially claim their turf.
Please note that the term “Information Technology” is being used in a general sense to describe a project (e.g. Data Quality, Master Data Management, etc.) and should not be confused with the IT group within an organization. At a very high level, the Business and IT are the internal groups representing the business and technical stakeholders on a project.
The Business usually owns the data and understands its meaning and use in the day-to-day operation of the enterprise. IT usually owns the hardware and software infrastructure of the enterprise's technical architecture.
Both groups can claim they are only responsible for what they own, resist collaborating with the “other side” and therefore create organizational barriers as fiercely defended as the continental borders of Europe and Asia in Risk.
2. In both, there are many competing strategies. In Risk, the official rules of the game include some basic strategies and over the years many players have developed their own fool-proof plans to guarantee victory. Some strategies advocate focusing on controlling entire continents, while others advise fortifying your borders by invading and occupying neighboring territories. And my blog-bout competitor Phil Simon half-jokingly claims that the key to winning Risk is securing the island nation of Madagascar.
On IT projects, you often hear a lot of buzzwords and strategies bandied about, such as Lean, Agile, Six Sigma, and Kaizen, to name but a few. Please understand – I am an advocate for methodology and best practices, and there are certainly many excellent frameworks out there, including the paradigms I just mentioned.
However, a general problem that I have with most frameworks is their tendency to adopt a one-size-fits-all strategy, which I believe is an approach that is doomed to fail. Any implemented framework must be customized to adapt to an organization’s unique culture.
In part, this is necessary because implementing changes of any kind will be met with initial resistance, but an attempt at forcing a one-size-fits-all approach almost sends a message to the organization that everything they are currently doing is wrong, which will of course only increase the resistance to change.
Starting with a framework simply provides a reference of best practices and recommended options of what has worked on successful IT projects. The framework should be reviewed in order to determine what can be learned from it and to select what will work in the current environment and what simply won't.
3. Pyrrhic victories are common during both endeavors. In Risk, sacrificing everything to win a single battle or to defend your favorite territory can ultimately lead you to lose the war. Political fiefdoms can undermine what could otherwise have been a successful IT project. Do not underestimate the unique challenges of your corporate culture.
Obviously, business, technical and data issues will all come up from time to time, and there will likely be disagreements regarding how these issues should be prioritized. Some issues will likely affect certain stakeholders more than others.
Keeping data and technology aligned with business processes requires getting people aligned and free to communicate their concerns. Coordinating discussions with all of the stakeholders and maintaining open communication can prevent a Pyrrhic victory for one stakeholder causing the overall project to fail.
4. Alliances are the key to true victory. In Risk, it is common for players to form alliances by combining their resources and coordinating their efforts in order to defend their shared borders or to eliminate a common enemy.
On IT projects, knowledge about data, business processes and supporting technology are spread throughout the organization. Neither the Business nor IT alone has all of the necessary information required to achieve success.
Successful projects are driven by an executive management mandate for the Business and IT to forge an alliance of ongoing and iterative collaboration throughout the entire project.
5. The outcomes of both are too often left to chance. IT projects are complex, time-consuming, and expensive enterprise initiatives. Success requires people taking on the challenge united by collaboration, guided by an effective methodology, and implementing a solution using powerful technology.
But the complexity of an IT project can sometimes work against your best intentions. It is easy to get pulled into the mechanics of documenting the business requirements and functional specifications, drafting the project plan and then charging ahead on the common mantra: “We planned the work, now we work the plan.”
Once an IT project achieves some momentum, it can take on a life of its own and the focus becomes more and more about making progress against the tasks in the project plan, and less and less on the project's actual business goals. Typically, this leads to another all too common mantra: “Code it, test it, implement it into production, and then declare victory.”
In Risk, the outcomes are literally determined by a roll of the dice. If you allow your IT project to lose sight of its business goals, then you treat it like a game of chance. And to paraphrase Albert Einstein:
“Do not play dice with IT Projects.”
Why “Monopoly” is a better metaphor for an IT Project
IT projects and “Monopoly” have a great deal in common. I thought long and hard about this at the gym, the source of all of my great thinking. I came up with six really smashing reasons.
1. Both things take much longer than originally expected. IT projects typically take much longer than expected for a wide variety of reasons. Rare is the project that finishes on time (with expected functionality delivered).
The same holds true for Monopoly. Remember when you were a kid and you wanted to play a quick game? Now, I consider the term “a quick game of Monopoly” to be the very definition of an oxymoron. You’d better block off about four to six hours for a proper game. Unforeseen complexities will doubtlessly delay even the best intentions.
2. During both endeavors, screaming matches typically erupt. Many projects become tense. I remember one in which two participants nearly came to blows. Most projects have key players engage in very heated debates over strategic vision and execution.
With Monopoly, especially after the properties are divvied up, players scream and yell over what constitutes a “fair” deal. “What do you mean Boardwalk for Ventnor Avenue and Pennsylvania Railroad isn’t reasonable? IT’S COMPLETELY FAIR!” Debates like this are the rule, not the exception.
3. While the basic rules may be the same, different people play by different rules. The vast majority of projects on which I have worked have had the usual suspects: steering committees, executive sponsors, PMOs, different stages of testing, and ultimately system activation. However, different organizations often try to do things in vastly different ways. For example, on two similar projects in different organizations, you are likely to find differences with respect to:
the number of internal and external folks assigned to a project
the project’s timeline and budget
project objectives
By the same token, people play Monopoly in somewhat different ways. Many don’t know about the auction rule. Others replenish Free Parking with a new $500 bill after someone lands on it. Also, many people disregard altogether the property assessment card while sticklers like me assess penalties when that vaunted red card appears.
4. Personal relationships can largely determine the outcome in both. Negotiation is key on IT projects. Clients negotiate rates, prices, and responsibilities with consulting vendors and/or software vendors.
In Monopoly, personal rivalries play a big part in who makes a deal with whom. Often players chime in (uninvited, of course) with their opinions on potential deals, without a doubt to affect the outcome.
5. Little things really matter, especially at the end. Towards the end of an IT project, snakes in the woodwork often come out to bite people when they least expect it. A tightly staffed or planned project may not be able to withstand a relatively minor problem, especially if the go-live date is non-negotiable.
In Monopoly, the same holds true. Laugh all you want when your opponent builds hotels on Mediterranean Avenue and Baltic Avenue, but at the end of the game those $250 and $450 charges can really hurt, especially when you’re low on cash.
6. Many times, each does not end; it is merely abandoned. A good percentage of projects have their plugs pulled prior to completion. A CIO may become tired with an interminable project and decide to simply end it before costs skyrocket even further.
I’d say that about half of the Monopoly games that I’ve played in the last fifteen years have also been called by “executive decision.” The writing is on the board, as 1 a.m. rolls around and only two players remain. Often player X simply cedes the game to player Y.
You are the Referee
All bouts require a referee. Blog-bouts are refereed by the readers. Therefore, please cast your vote in the poll and also weigh in on this debate by sharing your thoughts by posting a comment below. Since a blog-bout is co-posted, your comments will be copied (with full attribution) into the comments section of both of the blogs co-hosting this blog-bout.
About Jim Harris
Jim Harris is the Blogger-in-Chief at Obsessive-Compulsive Data Quality (OCDQ), which is an independent blog offering a vendor-neutral perspective on data quality. Jim is also an independent consultant, speaker, writer and blogger with over 15 years of professional services and application development experience in data quality (DQ), data integration, data warehousing (DW), business intelligence (BI), customer data integration (CDI), and master data management (MDM). Jim is also a contributing writer to Data Quality Pro, the leading online magazine and community resource dedicated to data quality professionals.
“Your opinion is irrelevant. We wish to improve ourselves.
We will add your business and technological distinctiveness to our own.
Your culture will adapt to service us.
You will be assimilated. Resistance is futile.”
Continuing my Star Trek theme, which began with my previous post Hailing Frequencies Open, imagine that you have been called into the ready room to be told your enterprise has decided to implement the proven data quality framework known as Business Operations and Reporting Governance – the BORG.
Frameworks are NOT Futile
Please understand – I am an advocate for methodology and best practices, and there are certainly many excellent frameworks that are far from futile. I have worked on many data quality initiatives that were following a framework and have seen varying degrees of success in their implementation.
However, the fictional BORG framework that I am satirizing exemplifies a general problem that I have with any framework that advocates a one-size-fits-all strategy, which I believe is an approach that is doomed to fail.
Any implemented framework must be customized to adapt to an organization's unique culture. In part, this is necessary because implementing changes of any kind will be met with initial resistance. An attempt at forcing a one-size-fits-all approach almost sends a message to the organization that everything they are currently doing is wrong, which will of course only increase the resistance to change.
Resistance is NOT Futile
Everyone has opinions – and opinions are never irrelevant. Fundamentally, all change starts with changing people's minds.
The starting point has to be improving communication and encouraging open dialogue. This means listening to what people throughout the organization have to say and not just telling them what to do. Keeping data aligned with business processes and free from poor quality requires getting people aligned and free to communicate their concerns.
Obviously, there will be dissension. However, you must seek a mutual understanding by practicing empathic listening. The goal is to foster an environment in which a diversity of viewpoints is freely shared without bias.
“One of the real dangers is emphasizing consensus over dissent,” explains James Surowiecki in his excellent book The Wisdom of Crowds. “The best collective decisions are the product of disagreement and contest, not consensus or compromise. Group deliberations are more successful when they have a clear agenda and when leaders take an active role in making sure that everyone gets a chance to speak.”
Avoid Assimilation
In order to be successful in your attempt to implement any framework, you must have realistic expectations.
Starting with a framework simply provides a reference of best practices and recommended options of what has worked on successful data quality initiatives. But the framework must still be reviewed in order to determine what can be learned from it and to select what will work in the current environment and what simply won't.
This doesn't mean that the customized components of the framework will be implemented simultaneously. All change will be gradual and implemented in phases – without the use of BORG nanoprobes. You will NOT be assimilated.
Your organization's collective consciousness will be best served by adapting the framework to your corporate culture.
Your data quality initiative will facilitate the collaboration of business and technical stakeholders, as well as align data usage with business metrics, and enable people to be responsible for data ownership and data quality.
Best practices will be disseminated throughout your collective – while also maintaining your individual distinctiveness.
“This is Captain James E. Harris of the Data Quality Starship Collaboration...”
Clearly, I am a Star Trek nerd – but I am also a people person. Although people, process, and technology are all important for successful data quality initiatives, without people, process and technology are useless.
Collaboration is essential. More than anything else, it requires effective communication – which begins with effective listening.
Seek First to Understand...Then to Be Understood
This is Habit 5 from Stephen Covey's excellent book The 7 Habits of Highly Effective People. “We typically seek first to be understood,” explains Covey. “Most people do not listen with the intent to understand; they listen with the intent to reply.”
We are all proud of our education, knowledge, understanding, and experience. Since it is commonly believed that experience is the path that separates knowledge from wisdom, we can't wait to share our wisdom with the world. However, as Covey cautions, our desire to be understood can make “our conversations become collective monologues.”
Covey explains that listening is an activity that can be practiced at one of the following five levels:
Ignoring – we are not really listening at all.
Pretending – we are only waiting for our turn to speak, constantly nodding and saying: “Yeah. Uh-huh. Right.”
Selective Listening – we are only hearing certain parts of the conversation, such as when we're listening to the constant chatter of a preschool child.
Attentive Listening – we are paying attention and focusing energy on the words that are being said.
Empathic Listening – we are actually listening with the intent to really try to understand the other person's frame of reference. You look out through it, you see the world the way they see the world, you understand their paradigm, you understand how they feel.
“Empathy is not sympathy,” explains Covey. “Sympathy is a form of agreement, a form of judgment. And it is sometimes the more appropriate response. But people often feed on sympathy. It makes them dependent. The essence of empathic listening is not that you agree with someone; it's that you fully, deeply, understand that person, emotionally as well as intellectually.”
Vulcans
Some people balk at discussing the use of emotion in a professional setting, where typically it is believed that rational analysis must protect us from irrational emotions. To return to a Star Trek metaphor, these people model their professional behavior after the Vulcans.
Vulcans live according to the philosopher Surak's code of emotional self-control. Starting at a very young age, they are taught meditation and other techniques in order to suppress their emotions and live a life guided by reason and logic alone.
Be Truly Extraordinary
In all professions, it is fairly common to encounter rational and logically intelligent people.
Truly extraordinary people masterfully blend both kinds of intelligence – intellectual and emotional. A well-grounded sense of self-confidence, an empathetic personality, and excellent communication skills, exert a more powerfully positive influence than simply remarkable knowledge and expertise alone.
Your Away Mission
As a data quality consultant, when I begin an engagement with a new client, I often joke that I shouldn't be allowed to speak for the first two weeks. This is my way of explaining that I will be asking more questions than providing answers.
I am seeking first to understand the current environment from both the business and technical perspectives. Only after I have achieved this understanding, will I then seek to be understood regarding my extensive experience of the best practices that I have seen work on successful data quality initiatives.
As fellow Star Trek nerds know, the captain doesn't go on away missions. Therefore, your away mission is to try your best to practice empathic listening at your next data quality discussion – “Make It So!”
Data quality initiatives require a holistic approach involving people, process, and technology. You must consider the people factor first and foremost, because it will be the people involved, and not the process or the technology, that will truly allow your data quality initiative to “Live Long and Prosper.”
As always, hailing frequencies remain open to your comments. And yes, I am trying my best to practice empathic listening.
Regular readers know that I often blog about the common mistakes I have observed (and made) in my professional services and application development experience in data quality (for example, see my post: The Nine Circles of Data Quality Hell).
According to Wikipedia: “Data governance is an emerging discipline with an evolving definition. The discipline embodies a convergence of data quality, data management, business process management, and risk management surrounding the handling of data in an organization.”
Since I have never formally used the term “data governance” with my clients, I have been researching what data governance is and how it specifically relates to data quality.
Thankfully, I found a great resource in Steve Sarsfield's excellent book The Data Governance Imperative, where he explains:
“Data governance is about changing the hearts and minds of your company to see the value of information quality...data governance is a set of processes that ensures that important data assets are formally managed throughout the enterprise...at the root of the problems with managing your data are data quality problems...data governance guarantees that data can be trusted...putting people in charge of fixing and preventing issues with data...to have fewer negative events as a result of poor data.”
Although the book covers data governance more comprehensively, I focused on three of my favorite data quality themes:
Business-IT Collaboration
Data Quality Assessments
People Power
Business-IT Collaboration
Data governance establishes policies and procedures to align people throughout the organization. Successful data quality initiatives require the Business and IT to forge an ongoing and iterative collaboration. Neither the Business nor IT alone has all of the necessary knowledge and resources required to achieve data quality success. The Business usually owns the data and understands its meaning and use in the day-to-day operation of the enterprise and must partner with IT in defining the necessary data quality standards and processes.
Steve Sarsfield explains:
“Business users need to understand that data quality is everyone's job and not just an issue with technology...the mantra of data governance is that technologists and business users must work together to define what good data is...constantly leverage both business users, who know the value of the data, and technologists, who can apply what the business users know to the data.”
Data Quality Assessments
Data quality assessments provide a much needed reality check for the perceptions and assumptions that the enterprise has about the quality of its data. Data quality assessments help with many tasks including verifying metadata, preparing meaningful questions for subject matter experts, understanding how data is being used, and most importantly – evaluating the ROI of data quality improvements. Building data quality monitoring functionality into the applications that support business processes provides the ability to measure the effect that poor data quality can have on decision-critical information.
Steve Sarsfield explains:
“In order to know if you're winning in the fight against poor data quality, you have to keep score...use data quality scorecards to understand the detail about quality of data...and aggregate those scores into business value metrics...solid metrics...give you a baseline against which you can measure improvement over time.”
People Power
Although incredible advancements continue, technology alone cannot provide the solution. Data governance and data quality both require a holistic approach involving people, process and technology. However, by far the most important of the three is people. In my experience, it is always the people involved that make projects successful.
Steve Sarsfield explains:
“The most important aspect of implementing data governance is that people power must be used to improve the processes within an organization. Technology will have its place, but it's most importantly the people who set up new processes who make the biggest impact.”
Conclusion
Data governance provides the framework for evolving data quality from a project to an enterprise-wide initiative. By facilitating the collaboration of business and technical stakeholders, aligning data usage with business metrics, and enabling people to be responsible for data ownership and data quality, data governance provides for the ongoing management of the decision-critical information that drives the tactical and strategic initiatives essential to the enterprise's mission to survive and thrive in today's highly competitive and rapidly evolving marketplace.
Strange Case of Dr Jekyll and Mr Hyde was Robert Louis Stevenson's classic novella about the duality of human nature and the inner conflict of our personal sense of good and evil that can undermine our noblest intentions. The novella exemplified this inner conflict using the self-destructive split-personality of Henry Jekyll and Edward Hyde.
The duality of data quality's nature can sometimes cause an organizational conflict between the Business and IT. The complexity of a data quality project can sometimes work against your best intentions. Knowledge about data, business processes and supporting technology are spread throughout the organization.
Neither the Business nor IT alone has all of the necessary information required to achieve data quality success.
As a data quality consultant, I am often asked to wear many hats – and not just because my balding head is distractingly shiny.
I often play a hybrid role that helps facilitate the business and technical collaboration of the project team.
I refer to this hybrid role as using the split-personality of Dr. Technology and Mr. Business.
Dr. Technology
With relatively few exceptions, IT is usually the first group that I meet with when I begin an engagement with a new client. However, this doesn't mean that IT is more important than the Business. Consultants are commonly brought on board after the initial business requirements have been drafted and the data quality tool has been selected. Meeting with IT first is especially common if one of my tasks is to help install and configure the data quality tool.
When I meet with IT, I use my Dr. Technology personality. IT needs to know that I am there to share my extensive experience and best practices from successful data quality projects to help them implement a well architected technical solution. I ask about data quality solutions that have been attempted previously, how well they were received by the Business, and if they are still in use. I ask if IT has any issues with or concerns about the data quality tool that was selected.
I review the initial business requirements with IT to make sure I understand any specific technical challenges such as data access, server capacity, security protocols, scheduled maintenance and after-hours support. I freely “geek out” in techno-babble. I debate whether Farscape or Battlestar Galactica was the best science fiction series in television history. I verify the favorite snack foods of the data architects, DBAs, and server administrators since whenever I need a relational database table created or more temporary disk space allocated, I know the required currency will often be Mountain Dew and Doritos.
Mr. Business
When I meet with the Business for the first time, I do so without my IT entourage and I use my Mr. Business personality. The Business needs to know that I am there to help customize a technical solution to their specific business needs. I ask them to share their knowledge in their natural language using business terminology. Regardless of my experience with other companies in their industry, every organization and their data is unique. No assumptions should be made by any of us.
I review the initial requirements with the Business to make sure I understand who owns the data and how it is used to support the day-to-day operation of each business unit and initiative. I ask if the requirements were defined before or after the selection of the data quality tool. Knowing how the data quality tool works can sometimes cause a “framing effect” where requirements are defined in terms of tool functionality, framing them as a technical problem instead of a business problem. All data quality tools provide viable solutions driven by impressive technology. Therefore, the focus should always be on stating the problem and solution criteria in business terms.
Dr. Technology and Mr. Business Must Work Together
As the cross-functional project team starts working together, my Dr. Technology and Mr. Business personalities converge to help clarify communication by providing bi-directional translation, mentoring, documentation, training and knowledge transfer. I can help interpret business requirements and functional specifications, help explain business and technical challenges, and help maintain an ongoing dialogue between the Business and IT.
I can also help each group save face by playing the important role of Designated Asker of Stupid Questions – one of those intangible skills you can't find anywhere on my resume.
As the project progresses, the communication and teamwork between the Business and IT will become more and more natural and I will become less and less necessary – one of my most important success criteria.
Success is Not So Strange
When the Business and IT forges an ongoing collaborative partnership throughout the entire project, success is not so strange.
In fact, your data quality project can be the beginning of a beautiful friendship between the Business and IT.
Everyone on the project team can develop a healthy split-personality.
IT can use their Mr. Business (or Ms. Business) personality to help them understand the intricacies of business processes.
The Business can use their Dr. Technology personality to help them “get their geek on.”
Data quality success is all about shiny happy people holding hands – and what's so strange about that?
People, process and technology. All three are necessary for success on your data quality project. By far, the most important of the three is people. However, who exactly are some of the most important people on your data quality project?
Or to phrase the question in a much more entertaining way...
Who are The Three Musketeers of Data Quality?
1. Athos, the Executive Sponsor - Provides the mandate for the Business and IT to forge an ongoing and iterative collaboration throughout the entire project. You might not see him roaming the halls or sitting in on most of the meetings. However, Athos provides oversight and arbitrates any issues of organization politics. Without an executive sponsor, a data quality project can not get very far and can easily lose momentum or focus. Perhaps most importantly, Athos is also usually the source of the project's funding.
2. Porthos, the Project Manager - Facilitates the strategic and tactical collaboration of the project team. Knowledge about data, business processes and supporting technology are spread throughout your organization. Neither the Business nor IT alone has all of the necessary information required to achieve data quality success. Porthos coordinates discussions with all of the stakeholders. Business users are able to share their knowledge in their natural language and IT users are able to “geek out” in techno-babble. Porthos clarifies communication by providing bi-directional translation. He interprets end user business requirements, explains technical challenges and maintains an ongoing dialogue between the Business and IT. Yes, Porthos is also responsible for the project plan. But he realizes that project management is more about providing leadership.
3. Aramis, the Subject Matter Expert - Provides detailed knowledge about specific data subject areas and business processes. Aramis reviews the reports from the data quality assessments and provides feedback based on his understanding of how the data is actually being used. He helps identify the data most valuable to the business. Aramis will often be an excellent source for undocumented business rules and can quickly clarify seemingly complex issues based on his data-centric point of view.
Alexandre Dumas fans will recall the novel's plucky primary protagonist was an outsider who became the Fourth Musketeer and yes, data quality has one too:
4. D'Artagnan, the Data Quality Consultant - Provides extensive experience and best practices from successful data quality implementations. Most commonly, d'Artagnan is a certified expert with the data quality tool you have selected. D'Artagnan's goal is to help you customize a technical solution to your specific business needs. Unlike the Dumas character, your d'Artagnan usually doesn't accept the Musketeer commission at the end of the project. Therefore, his primary responsibility is to make himself obsolete as quickly as possible by providing mentoring, documentation, training and knowledge transfer.
Your data quality project will typically have more than one person (and obviously not just men) playing each of these classic roles although you may use different job titles. Additionally, there will be many other important people on your project playing many other key roles, such as data architect, business analyst, application developer and system tester - to name just a few.
Data quality truly takes a team effort. Remember that you are all in this together.
So if anyone asks you who is the most important person on your project, then just respond with the Musketeer Motto:
In Dante’s Inferno, these words are inscribed above the entrance into hell. The Roman poet Virgil was Dante’s guide through its nine circles, each an allegory for unrepentant sins beyond forgiveness.
The Very Model of a Modern DQ General will be your guide on this journey through nine of the most common mistakes that can doom your data quality project:
1.Thinking data quality is an IT issue (or a business issue)- Data quality is not an IT issue. Data quality is also not a business issue. Data quality is everyone's issue. Successful data quality projects are driven by an executive management mandate for the business and IT to forge an ongoing and iterative collaboration throughout the entire project. The business usually owns the data and understands its meaning and use in the day to day operation of the enterprise and must partner with IT in defining the necessary data quality standards and processes.
2. Waiting for poor data quality to affect you- Data quality projects are often launched in the aftermath of an event when poor data quality negatively impacted decision-critical enterprise information. Some examples include a customer service nightmare, a regulatory compliance failure or a financial reporting scandal. Whatever the triggering event, a common response is data quality suddenly becomes prioritized as a critical issue.
3. Believing technology alone is the solution- Although incredible advancements continue, technology alone cannot provide the solution. Data quality requires a holistic approach involving people, process and technology. Your project can only be successful when people take on the challenge united by collaboration, guided by an effective methodology, and of course, implemented with amazing technology.
4. Listening only to the expert- An expert can be an invaluable member of the data quality project team. However, sometimes an expert can dominate the decision making process. The expert's perspective needs to be combined with the diversity of the entire project team in order for success to be possible.
5. Losing focus on the data- The complexity of your data quality project can sometimes work against your best intentions. It is easy to get pulled into the mechanics of documenting the business requirements and functional specifications and then charging ahead with application development. Once the project achieves some momentum, it can take on a life of its own and the focus becomes more and more about making progress against the tasks in the project plan, and less and less on the project's actual goal, which is to improve the quality of your data.
This common mistake was the theme of my post: Data Gazers.
6. Chasing perfection- An obsessive-compulsive quest to find and fix every data quality problem is a laudable pursuit but ultimately a self-defeating cause. Data quality problems can be very insidious and even the best data quality process will still produce exceptions. Although this is easy to accept in theory, it is notoriously difficult to accept in practice. Do not let the pursuit of perfection undermine your data quality project.
7. Viewing your data quality assessment as a one-time event- Your data quality project should begin with a data quality assessment to assist with aligning perception with reality and to get the project off to a good start by providing a clear direction and a working definition of success. However, the data quality assessment is not a one-time event that ends when development begins. You should perform iterative data quality assessments throughout the entire development lifecycle.
8. Forgetting about the people- People, process and technology. All three are necessary for success on your data quality project. However, I have found that the easiest one to forget about (and by far the most important of the three) is people.
9. Assuming if you build it, data quality will come- There are many important considerations when planning a data quality project. One of the most important is to realize that data quality problems cannot be permanently “fixed" by implementing a one-time "solution" that doesn't require ongoing improvements.
New York City, 2022 - Technical Architect Robert Thorn and Business Analyst Sol Roth have been called in by Business Director Harry Harrison and IT Director Tab Fielding to investigate an unsolved series of anomalies that have been plaguing the company's Dystopian Automated Transactional Analysis (DATA) system.
Harry Harrison (Business Director):
“Thank you for coming on such short notice. We hope that you will be able to help us with our DATA problems.”
Robert Thorn (Technical Architect):
“You're welcome.”
Sol Roth (Business Analyst):
“I am sure that we can help. Can you provide us with an overview of the situation?”
Harry Harrison (Business Director):
“We hired quality expert William Simonson from Soylent Consulting to fix our DATA problems.”
Robert Thorn (Technical Architect):
“Soylent Consulting? Never heard of them.”
Sol Roth (Business Analyst):
“They are the professional services division of Green, Incorporated.”
Harry Harrison (Business Director):
“Yes, that's right - the High-Energy, Environmentally-Minded Corporation”
Tab Fielding (IT Director):
“More like the High-Rate, Weak-Minded Corporation, if you ask me.”
Harry Harrison (Business Director):
“Well anyway, Mr. Simonson first met with me to receive the business requirements.”
Sol Roth (Business Analyst):
“Receive? You mean he was handed the completed business requirements document?”
Harry Harrison (Business Director):
“Yes, of course.”
Sol Roth (Business Analyst):
“So...he didn't meet directly with anyone on the business team?”
Harry Harrison (Business Director):
“No, I write the business requirements document so that meeting with the business team is unnecessary.”
Sol Roth (Business Analyst):
“O...K...then what happened?”
Tab Fielding (IT Director):
“Mr. Simonson met with me to receive the functional specifications.”
Robert Thorn (Technical Architect):
“Receive? So...did you write the functional specifications?”
Tab Fielding (IT Director):
“Yes, I write the functional specifications after reading Mr. Harrison's business requirements document.”
Robert Thorn (Technical Architect):
“Reading? So...you didn't even meet directly with Mr. Harrison?”
Tab Fielding (IT Director):
“No, before today's meeting, I haven't even seen him or anyone from the business team in months.”
Robert Thorn (Technical Architect):
“O...K...then what happened?”
Tab Fielding (IT Director):
“Mr. Simonson spent a few months coding, implemented our solution, then told us he was ‘going home.’”
Harry Harrison (Business Director):
“And now our DATA is worse than ever!”
Sol Roth (Business Analyst):
“And both of you are wondering how it came to this?”
Harry Harrison (Business Director) and Tab Fielding (IT Director):
“Yes!”
Robert Thorn (Technical Architect):
“I'll tell you how. Because you both forgot the most important aspect of DATA.”
Tab Fielding (IT Director):
“What are you talking about?”
Robert Thorn (Technical Architect):
“It's people. Data's Quality is made by People. You've gotta tell them. You've gotta tell them!”
Harry Harrison (Business Director):
“I promise, Thorn. I promise. We will tell executive management.”
Robert Thorn (Technical Architect):
“You tell everybody. Listen to me, both of you! You've gotta tell everybody that Data Quality is People!”
Jim Harris in