The Diffusion of Data Governance
Jim Harris in Blogs, Data Quality, Debates tagged Data Governance, Philosophy Monday, July 5, 2010 at 1:00PM 
Marty Moseley of Initiate recently blogged Are We There Yet? Results of the Data Governance Survey, and the blog post includes a link to the survey, which is freely available—no registration required.
The Initiate survey says that although data governance dates back to the late 1980s, it is experiencing a resurgence because of initiatives such as business intelligence, data quality, and master data management—as well as the universal need to make better data-driven business decisions “in less time than ever before, often culling data from more structured and unstructured sources, with more transparency required.”
Winston Chen of Kalido recently blogged A Brief History of Data Governance, which provides a brief overview of three distinct eras in data management: Application Era (1960-1990), Enterprise Repository Era (1990-2010), and Policy Era (2010-?).
As I commented on Winston’s post, I began my career at the tail-end of the Application Era, and my career has been about a 50/50 split between applications and enterprise repositories since history does not move forward at the same pace for all organizations, including software vendors—by which, I mean that my professional experience was influenced more by working for vendors selling application-based solutions than it was by working with clients who were, let’s just say, less than progressive.
Diffusion of innovations (illustrated above) is a theory developed by Everett Rogers for describing the five stages and the rate at which innovations (e.g., new ideas or technology) spread through markets (or “cultures”), starting with the Innovators and the Early Adopters, then progressing through the Early Majority and the Late Majority, and finally ending with the Laggards.
Therefore, the exact starting points of the three eras Winston described in his post can easily be debated because progress can be painfully slow until a significant percentage of the Early Majority begins to embrace the innovation—thereby causing the so-called Tipping Point where progress begins to accelerate enough for the mainstream to take it seriously.
Please Note: I am not talking about crossing “The Chasm”—which as Geoffrey A. Moore rightfully discusses, is the critical, but much earlier, phenomenon occurring when enough of the Early Adopters have embraced the innovation so that the beginning of the Early Majority becomes an almost certainty—but true mainstream adoption of the innovation is still far from guaranteed.
The tipping point that I am describing occurs within the Early Majority and before the top of the adoption curve is reached.
Achieving 16% market share (or “cultural awareness”) is where the Early Majority begins—and only after successfully crossing the chasm (which I approximate occurs somewhere around 8% market share). However, the difference between a fad and a true innovation occurs somewhere around 25% market share—and this is the tipping point that I am describing.
The Late Majority (and the top of the adoption curve) doesn’t begin until 50% market share, and it’s all downhill from there, meaning that the necessary momentum has been achieved to almost guarantee that the innovation will be fully adopted.
For example, it could be argued that master data management (MDM) reached its tipping point in late 2009, and with the wave of acquisitions in early 2010, MDM stepped firmly on the gas pedal of the Early Majority, and we are perhaps just beginning to see the start of MDM’s Late Majority.
It is much harder to estimate where we are within the diffusion of data governance. Of course, corporate cultural awareness always plays a significant role in determining the adoption of new ideas and the market share of emerging technologies.
The Initiate survey concludes that “the state of data governance initiatives is still rather immature in most organizations” and reveals “a surprising lack of perceived executive interest in data governance initiatives.”
Rob Karel of Forrester Research recently blogged about how Data Governance Remains Immature, but he is “optimistic that we might finally see some real momentum building for data governance to be embraced as a legitimate competency.”
“It will likely be a number of years before best practices outnumber worst practices,” as Rob concludes, “but any momentum in data governance adoption is good momentum!”
From my perspective, data governance is still in the Early Adopter phase. Perhaps 2011 will be “The Year of Data Governance” in much the same way that some have declared 2010 to to be “The Year of MDM.”
In other words, it may be another six to twelve months before we can claim the Early Majority has truly embraced not just the idea of data governance, but have realistically begun their journey toward making it happen.
What Say You?
Please share your thoughts about the diffusion of data governance, as well as your overall perspectives on data governance.
Related Posts
MacGyver: Data Governance and Duct Tape
Jack Bauer and Enforcing Data Governance Policies
Follow OCDQ
If you enjoyed this blog post, then please subscribe to OCDQ via my RSS feed, my E-mail updates, or Google Reader.
You can also follow OCDQ on Twitter, fan the Facebook page for OCDQ, and connect with me on LinkedIn.
Reader Comments (13)
Jim,
Like you, I entered the enterprise IT world around the same time. I was amazed not to hear about Data Governance until relatively recently; I mean, if it was so big, then how could I have missed it?
Turns out that I was not alone. There are still many people who've yet to hear about the term, much less use its tenets to effect the kind of change that Data Governance promises. Perhaps we're moving to the right of the curve after all.
Good post.
ps
Fascinating post as ever Jim.
I think we need plenty of research like this to help us visualize just how early we are in these phases of maturity. The danger is that because we all live in these very active online and offline communities that discuss DG/DQ/MDM/BI etc. all day long we somehow get comfortable in the belief that we're moving forward.
Obviously, putting my editor's hat on for a minute, I would love to think that all this content, discussion and industry momentum we're a part of is having an effect but when you actually see the numbers of site references then I think we're still way down in the Innovator / Early Adopter phase.
I think we're seeing a lot of tactical implementations to resolve issues that are biting the executives in the butt, but cases of wholesale data governance across the enterprise are still statistically few and far between when we look at the volume of organizations out there.
But hey, that presents a great opportunity for all of us - right?
Jim, I agree that we're still in the early stages here. Most companies we work with only really get Data Quality if they've been bitten or if they have to (SOX).
I was working on a data migration last week where the client knowingly loaded all of the data and intended to patch it later. They're already suffering since the data is too dynamic and is changing rapidly on the source.
I recently wrote that data governance has just crossed the chasm. With your definition of the "tipping point", I think you're right that data governance is somewhere between the chasm and the tipping point.
Some other surveys paint a more optimistic picture.
The Economist survey in 2008 on information governance find that 39% of the organizations have an information governance body in place, while another 15% are in the process of forming one. That survey contains a large portion of financial surveys companies (24%) and higher percentage of executives (27% CEO/President/MD). And a relatively higher percentage of respondents from Western Europe which has a higher affinity for governance of all varieties.
An Accenture CIO survey in 2007 finds that 19% of organizations have an enterprise data governance program in place. The 2008 Information Difference data governance survey finds 36% of organization having already implemented data governance, with 14% dedicating full time staff.
To counter that, I recently went to a CIO conference and sat with a group of 8 CIOs at lunch. They came from smaller organizations, and data governance was not even on the horizon.
In other words, it goes back to your point of uneven adoption, depending on vertical, geography, and company size. These surveys may also point to a disparity between how executive and rank-and-file employees see data governance.
Great post as always, Jim. And thanks for the reference.
Winston
Thank you very much for your comments, Phil, Dylan, Dave, and Winston.
Your feedback is greatly appreciated!
@Phil — Yes, until about a year ago, I too hadn't heard much of anything about data governance, which lead me to invest the time and energy into learning more about it. Although the industry definitely has a fondness for making everything old seem new again by giving it a shiny new term, I believe data governance deserves its own calling card, and that its time may finally be arriving.
@Dylan — Yes, the Expert-o-Sphere (I just made that term up to encapsulate not only the blogosphere, Twitterverse, and wider social networking space, but also, as you said, all of very active online and offline communities in our industry) can definitely provide the impression of progress. However, our collective perception may slowly becoming a reality.
@Dave — Yes, data quality and data governance are often reactively adopted only after harsh reality SOX'it to 'em! :-)
@Winston — Yes, different surveys, and different hype cycles, definitely have a role to play in the industry's perception of the adoption of new trends. As you stated, the uneven adoption, depending on vertical, geography, and company size, and disparity between how executive and rank-and-file employees see data governance, continues and until some of that levels out, true progress may continue to be delayed. However, to paraphrase Rob Karel, even delayed progress is still progress.
From the LinkedIn Group for Data Quality Pro, Marty Moseley commented:
“I hope you're right, but fear it may take longer than that, simply because data governance is a highly political undertaking, and the benefits are still "soft," so data governance gets de-prioritized in favor of more tangible programs...
Mr Ray-of-Sunshine...”
And I responded:
Thanks Marty (aka “Mr Ray-of-Sunshine”) — Yes, politics is, by far, the most significant corporate cultural challenge delaying the true progress of data governance. However, this daunting reality always reminds me of the famous words of Otto von Bismarck: “Politics is the art of the possible.”
Therefore, I believe that Data Governance is also the Art of the Possible.
Early stages still...you bet!
It seems (to me) that although some awareness and program implementation are underway, most of these are driven by the management, rather than the executive level.
Customer centric projects such as CRM may be top level priorities, but understanding what is required (data governance, information architecture, metadata management, etc) for long term success is something senior executives are not too interested in championing.
The result of this, corporate programs driven by middle level management, are more costly to do because of the effort required to raise awareness and gain understanding and alignment, before any actual work can take place.
Thanks for an always interesting and valid post!
Jim,
Thanks again for the reference. I think I'll blog about my own history of Data Governance tomorrow!
Hopefully, it will be interesting and informative.
Not to rest on our achievements to date, I continue to push for even more automation and richness in managing the Declarations of Data Governance.
Cheers, everyone!
A wonderful piece on data governance!
Thoroughly enjoyed it - especially the reminder of where I too began (late '80s 'large systems implementation').
However I would like to raise an issue or two - not in contention, but in reflection, as I have found myself moving from ERP-based solution delivery towards addressing 'pre-project' issues and onwards into the 'readiness' of clients to receive the 'communion' of ERP.
Part of that transition process came about at an end-user role where I implemented what I called 'project boards' simply to find out who was messing about with what, why and who was supposed to be paying for it and what conflicts were occurring - a form of governance if you like.
From that came the early (2004) realization (for me at least...) that the underlying issue was often that the 'owners' of such things as processes and data were simply not involved in 'IT' projects, some of which were unashamed attempts at bolstering the 'technology' with a shiny new toy. Underlying all that was the lack of true 'ownership' (much of which was overcome by a scathing SOX audit in late '04) of the data - or perhaps we should think more of the processes that source the data.
Where all this is going is that Data Governance (or MDM as it is now, but watch this space as practitioners and clients realize the inadequacies of mere MDM) has always been with us if we were 'doing it right' (thank our lucky stars nobody was or we'd all be on the dole queue!).
However, don't be tempted to plonk 'governance' on top of a problem (rather the way SOX did, for all its good points) - 'governance' is a discipline within extant activities (think 'constitutional' government and you aren't far off...), not something that can be dumped on top (think 'communism' if you like...!) that needs a careful hand in its application to a) work and b) not be a fad that goes away when the underlying targets realize they can get away without it...
@Marty — “I hope you're right, but fear it may take longer than that, simply because data governance is a highly political undertaking, and the benefits are still "soft," so data governance gets de-prioritized in favor of more tangible programs...” Oh, yes, the 'politics'...yet is is possible to create a business case (we do prepare business cases for our projects, don't we..?) for 'governance' that is tangible. Trouble is that when you do actually present it, it can often show the tangible damage that lack of governance has caused, which causes the usual 'egg on face' that has to be covered up quickly. We are not thanked for pointing that out either! Nevertheless, the business case, with the appropriate metrics, can be quite the bolster to our case! Then you have to deal with the 'silo' mentality ... It can be done!
Somebody asked me once if we can have governance without technology or technology without governance. The answer in both cases is of course, yes, but the results will be glaringly substandard and potentially destructive.
I have worked extensively to develop the 'zero-impact' and 'zero-overhead' approaches to 'governance'. I would be happy to carry on my ramblings if anyone is listening...
Otherwise I wait with interest the next comments.
Best Regards,
Marcus
Thank you very much for your comments, Jill, Marty, and Marcus.
Your feedback is greatly appreciated!
@Jill — Yes, a lack of true senior executive sponsorship and support can be the kiss of death for any enterprise initiative, but especially for a data governance program. This is one of the many reasons that I am looking forward to attending your webinar on Guerilla Data Governance.
@Marty — I am looking forward to reading your upcoming blog post about the history of Data Governance, as well as attending your upcoming webinar about the Declarations of Principles, Policies, Procedures/Processes, Business Rules, and Metrics for Data Governance.
@Marcus — First, thanks for your very detailed comments. As for what exactly data governance is, this is probably as good a time as any for me to plug my Declaration of Data Governance blog post :-)
From the LinkedIn Group for Data Governance, Terri Spencer commented:
“In your opinion, what companies have done an effective job in implementing sustainable data governance with business support and a realistic strategy?”
And I responded:
Thanks for your comment, Terri.
I wish I could name countless examples of early data governance success stories. However, the very fact that we are so early in the adoption of data governance as a true enterprise-wide best practice with business support, executive sponsorship, and a well-defined, realistic strategy--is the main reason that I cannot provide examples (the other being that I cannot talk publicly about my clients).
Although specific companies are not mentioned, the Initiate survey, which is freely available—no registration required, that is included in their blog post I referenced, does provide interesting insights, and does include feedback from a variety of different industries as well as different company configurations (number of employees, number of business units, etc.)
From the LinkedIn Group for Data Quality Pro, Lucy Cording commented:
“I think that we are seeing the move towards data governance more and more. Data legislation, improved technology to help, and the strive towards a Single Customer View within organizations, to name a few, all are working towards putting this up on people's agendas.”
And I responded:
Thanks for your comment, Lucy,
Yes, I definitely agree that we are seeing an increasingly forward momentum building toward the adoption of data governance by more organizations.
Best Regards,
Jim
Over on the SmartData Collective, Neil Raden commented:
“I was speaking with Winston a few days ago about this issue. Winston uses what I think is a fairly novel way to describe data governance, that doesn't really involve data at all. He calls it policy. In this way, DG software is rather simple and plays a support role. The problem is, it is only software vendors and implementers (and analysts and conference organizers) who are really pushing DG. As an implementer myself, my firm is not currently qualified to pursue a wide-ranging changement role to implement policy. I'm not sure anyone is.
I have some ideas how this could work, though. I think it is impossible to implement policy as an abstraction above systems, data and work without a semantic framework.
In that sense, technology could take the lead to start and once policies for some subject area are in place, behavioral changes may, potentially, follow. With some shepherding. I'm willing to give it a try.”
And I replied:
I definitely agree that data governance requires a change management approach, and all truly effective change management has to focus on behavioral changes. Regarding the connection between Policy and Data Governance, I explained my perspective in my recent blog post: Declaration of Data Governance.
Great post, Jim. I really like your thoughts on the future of data governance. I wonder what 2011 "The Year of Data Governance" will look like? We have a community for IM professionals (www.openmethodology.org) that discusses related topics and have bookmarked this post. Looking forward to reading more of your work and sharing with our community.