Do you believe in Magic (Quadrants)?
Jim Harris in Books, Data Quality, Debates, News, Vendors tagged Gartner Magic Quadrant, Philosophy Thursday, July 1, 2010 at 3:00AM If you follow Data Quality on Twitter like I do, then you are probably already well aware that the 2010 Gartner Magic Quadrant for Data Quality Tools was released this week (surprisingly, it did not qualify as a Twitter trending topic).
The five vendors that were selected as the “data quality market leaders” were DataFlux, IBM, Informatica, SAP Business Objects, and Trillium.
Disclosure: I am a former IBM employee, an IBM Information Champion, and I currently blog for the DataFlux Community of Experts.
Please let me stress that I have the highest respect for both Ted Friedman and Andy Bitterer, as well as their in depth knowledge of the data quality industry and their insightful analysis of the market for data quality tools.
In this blog post, I simply want to encourage a good-natured debate, and not about the Gartner Magic Quadrant specifically, but rather about market research in general. Gartner is used as the example because they are perhaps the most well-known and the source most commonly cited by data quality vendors during the sales cycle—and obviously, especially by the “leading vendors.”
I would like to debate how much of an impact market research really has on a prospect’s decision to purchase a data quality tool.
Let’s agree to keep this to a very informal debate about how research can affect both the perception and the reality of the market.
Therefore—for the love of all high quality data everywhere—please, oh please, data quality vendors, do NOT send me your quarterly sales figures, or have your PR firm mercilessly spam either my comments section or my e-mail inbox with all the marketing collateral “proving” how Supercalifragilisticexpialidocious your data quality tool is—I said please, so play nice.
The OCDQ View on OOBE-DQ
In a previous post, I used the term OOBE-DQ to refer to the out-of-box-experience (OOBE) provided by data quality (DQ) tools, which usually becomes a debate between “ease of use” and “powerful functionality” after you ignore the Magic Beans sales pitch that guarantees you the data quality tool is both remarkably easy to use and incredibly powerful.
However, the data quality market continues to evolve away from esoteric technical tools and toward business-empowering suites providing robust functionality with easier to use and role-based interfaces that are tailored to the specific needs of different users, such as business analysts, data stewards, application developers, and system administrators.
The major players are still the large vendors who have innovated (mostly via acquisition and consolidation) enterprise application development platforms with integrated (to varying degrees) components, which provide not only data quality functionality, but also data integration and master data management (MDM) as well.
Many of these vendors also offer service-oriented deployments delivering the same functionality within more loosely coupled technical architectures, which includes leveraging real-time services to prevent (or at least greatly minimize) poor data quality at the multiple points of origin within the data ecosystem.
Many vendors are also beginning to provide better built-in reporting and data visualization capabilities, which is helping to make the correlation between poor data quality and suboptimal business processes more tangible, especially for executive management.
It must be noted that many vendors (including the “market leaders”) continue to struggle with their International OOBE-DQ.
Many (if not most) data quality tools are strongest in their native country or their native language, but their OOBE-DQ declines significantly when they travel abroad. Especially outside of the United States, smaller vendors with local linguistic and cultural expertise built into their data quality tools have continued to remain fiercely competitive with the larger vendors.
Market research certainly has a role to play in making a purchasing decision, and perhaps most notably as an aid in comparing and contrasting features and benefits, which of course, always have to be evaluated against your specific requirements, including both your current and future needs.
Now let’s shift our focus to examining some of the inherent challenges of evaluating market research, perception, and reality.
Confirmation Bias
First of all, I realize that this debate will suffer from a considerable—and completely understandable—confirmation bias.
If you are a customer, employee, or consultant for one of the “High Five” (not an “official” Gartner Magic Quadrant term for the Leaders), then obviously you have a vested interest in getting inebriated on your own Kool-Aid (as noted in my disclosure above, I used to get drunk on the yummy Big Blue Kool-Aid). Now, this doesn’t mean that you are a “yes man” (or a “yes woman”). It simply means it is logical for you to claim that market research, market perception, and market reality are in perfect alignment.
Likewise, if you are a customer, employee, or consultant for one of the “It Isn’t Easy Being Niche-y” (rather surprisingly, not an “official” Gartner Magic Quadrant term for the Niche Players), then obviously you have a somewhat vested interest in claiming that market research is from Mars, market perception is from Venus, and market reality is really no better than reality television.
And, if you are a customer, employee, or consultant for one of the “We are on the outside looking in, flipping both Gartner and their Magic Quadrant the bird for excluding us” (I think that you can figure out on your own whether or not that is an “official” Gartner Magic Quadrant term), then obviously you have a vested interest in saying that market research can “Kiss My ASCII!”
My only point is that your opinion of market research will obviously be influenced by what it says about your data quality tool.
Therefore, should it really surprise anyone when, during the sales cycle, one of the High Five uses the Truly Awesome Syllogism:
“Well, of course, we say our data quality tool is awesome.
However, the Gartner Magic Quadrant also says our data quality tool is awesome.
Therefore, our data quality tool is Truly Awesome.”
Okay, so technically, that’s not even a syllogism—but who said any form of logical argument is ever used during a sales cycle?
On a more serious note, and to stop having too much fun at Gartner’s expense, they do advise against simply selecting vendors in their “Leaders quadrant” and instead always advise to select the vendor that is the better match for your specific requirements.
Features and Benefits: The Game Nobody Wins
As noted earlier, a features and benefits comparison is not only the most common technique used by prospects, but it is also the most common—if not the only—way that the vendors themselves position their so-called “competitive differentiation.”
The problem with this approach—and not just for data quality tools—is that there are far more similarities than differences to be found when comparing features and benefits.
Practically every single data quality tool on the market today will include functionality for data profiling, data quality assessment, data standardization, data matching, data consolidation, data integration, data enrichment, and data quality monitoring.
Therefore, running down a checklist of features is like playing a game of Buzzword Bingo, or constantly playing Musical Chairs, but without removing any of the chairs in between rounds—in others words, the Features Game almost always ends in a tie.
So then next we play the Benefits Game, which is usually equally pointless because it comes down to silly arguments such as “our data matching engine is better than yours.” This is the data quality tool vendor equivalent of:
Vendor D: “My Dad can beat up your Dad!”
Vendor Q: “Nah-huh!”
Vendor D: “Yah-huh!”
Vendor Q: “NAH-HUH!”
Vendor D: “YAH-HUH!”
Vendor Q: “NAH-HUH!”
Vendor D: “Yah-huh! Stamp it! No Erasies! Quitsies!”
Vendor Q: “No fair! You can’t do that!”
After both vendors have returned from their “timeout,” a slightly more mature approach is to run a vendor “bake-off” where the dueling data quality tools participate in a head-to-head competition processing a copy of the same data provided by the prospect.
However, a bake-off often produces misleading results because the vendors—and not the prospect—perform the competition, making it mostly about vendor expertise, not OOBE-DQ. Also, the data used rarely exemplifies the prospect’s data challenges.
If competitive differentiation based on features and benefits is a game that nobody wins, then what is the alternative?
The Golden Circle
I recently read the book Start with Why by Simon Sinek, which explains that “people don’t buy WHAT you do, they buy WHY you do it.”
The illustration shows what Simon Sinek calls The Golden Circle.
WHY is your purpose—your driving motivation for action.
HOW is your principles—specific actions that are taken to realize your Why.
WHAT is your results—tangible ways in which you bring your Why to life.
It’s a circle when viewed from above, but in reality it forms a megaphone for broadcasting your message to the marketplace.
When you rely only on the approach of attempting to differentiate your data quality tool by discussing its features and benefits, you are focusing on only your WHAT, and absent your WHY and HOW, you sound just like everyone else to the marketplace.
When, as is often the case, nobody wins the Features and Benefits Game, a data quality tool sounds more like a commodity, which will focus the marketplace’s attention on aspects such as your price—and not on aspects such as your value.
Due to the considerable length of this blog post, I have been forced to greatly oversimplify the message of this book, which a future blog post will discuss in more detail. I highly recommend the book (and no, I am not an affiliate).
At the very least, consider this question:
If there truly was one data quality tool on the market today that, without question, had the very best features and benefits, then why wouldn’t everyone simply buy that one?
Of course your data quality tool has solid features and benefits—just like every other data quality tool does.
I believe that the hardest thing for our industry to accept is—the best technology hardly ever wins the sale.
As most of the best salespeople will tell you, what wins the sale is when a relationship is formed between vendor and customer, a strategic partnership built upon a solid foundation of rapport, respect, and trust.
And that has more to do with WHY you would make a great partner—and less to do with WHAT your data quality tool does.
Do you believe in Magic (Quadrants)?
How much of an impact do you think market research has on the purchasing decision of a data quality tool? How much do you think research affects both the perception and the reality of the data quality tool market? How much do you think the features and benefits of a data quality tool affect the purchasing decision?
All perspectives on this debate are welcome without bias. Therefore, please post a comment below.
PLEASE NOTE
Comments advertising your products and services (or bashing competitors) will not be approved.
Reader Comments (8)
Engaging post Jim, I have some strong views on the Quadrant, I may write about it shortly but my main concern is that the research doesn't do enough to promote innovation and seems to contradict a lot of what I'm hearing on the ground.
True there is an innovation sector in the quadrant but to even get on the quadrant a technology provider must maintain an installed base of at least 75 production customers. Now, that is a big task for any startup in a fiercely competitive environment.
Over on Data Quality Pro we've reported on several young data quality startups that I think have disruptive technologies and a formidable track record of launching successful data quality products in the past that were bought out by the "big boys" (Trillium and Informatica) but they get only one line apiece. (Full disclosure - neither of these vendors are sponsors of Data Quality Pro at the time of writing).
For me, market research should focus on the leading edge as well as the old guard. By talking to existing customers of existing products are you not restricting your research to a specific market? Shouldn't market research be more forward looking?
In terms of the strengths of several vendors, I've taken recent calls from a number of our members who have completed RFP's and their accounts differ wildly to the strengths reported. I therefore think that the research needs to be broader, as you say, if you're speaking to existing customers of existing products then it's likely to skew the results. What about speaking to those companies who simply cannot find a solution for their needs or have recently completed in-depth RFP's? They also represent the market after all.
Jim, very well written piece.
An observation: As I have worked for one of the: “We are on the outside looking in, flipping both Gartner and their Magic Quadrant the bird for excluding us” vendors I have noticed that the quadrant seems to be more important to the (“big five”) consulting firms than to the direct buyers.
An experience: As you say: “It must be noted that many vendors (including the “market leaders”) continue to struggle with their International OOBE-DQ.” As we are based in continental Europe we have had some success with doing a test as part of a sales competition proving that we have better results, especially in matching (hope this remark will pass moderation).
An opinion: Selling data quality tools is still though not so much about beating the others as it is beating the alternative of doing nothing about data quality. Usually I name “Laissez-Faire” as our main competitor.
Hey Jim, thanks for stirring up the pot a bit, I'm betting this one is going to generate quite a few "strong" opinions...
Regarding the impact of the market research, I'm in the camp that the research has significantly higher impact within these smaller/niche markets then it would for larger audiences such as those you might find in the "Database Magic Quadrant".
From my little window on the world, I believe Data Quality and other Data Management specialties are very much in an "immature" stage of their life cycle as practices. These practices are not what Gartner is evaluating and since these practices are "immature" you'll find folks moving into the practices with "other" backgrounds.
With all this said, when folks move into these "immature" practices and determine they need a tool, they will look to the analysts for insight because they don't yet have personal experience with a preferred tool.
These folks are most likely going to look to “High Five” as the first few to evaluate, and chances are they are too busy with day-to-day data issues that they will not look far beyond the list. This is pretty much exactly how our experience purchasing a tool played out, except it took a very long time for us to "sell" the purchase of the tool to our senior management.
Thanks for the post and best regards...Rich Murnane
Ah, such a good post, and such a good question Jim!
I used to be in a large enterprise, and often used market research and magic quadrants (as well as tea leaves, various procedures involving chicken bones, and other highly scientific methods) to help select enterprise software.
Now I've switched to the vendor side of things, and I have founded an upstart startup in the data quality and transformation realm. Of course we are focusing on the ease of use, insanely good customer service, and "the right features" not all the features. (Hint: Our tool did not make the leaders quadrant this year :-) )
In terms of how these types of market research devices influence purchases, I think in large enterprises they often play a major role in the final decision. The more money being spent, the more reassuring it is to see your preferred vendor in that little box.
Wikipedia, source of all valid facts for bloggers, claims that the term "Fear Uncertainty and Doubt" originated from a former IBM sales person (this was in the seventies, when IBM would have been the mainframe magic quadrant leader, had it existed). This sales person went out on his own, was competing with IBM, and trying to overcome the famous "No one ever got fired for buying IBM".
I think it's possible that "No one ever got fired for picking a tool in the leaders quadrant" is maybe pushing it a bit--but some of that basic thinking applies.
Thank goodness there are still the chicken bones and tea leaves to help those vendors that are quadrant challenged.
Excellent post Jim – thanks!
Many of your points about DQ-MDM-DI software solutions are applicable to most software solutions.
Coming from many years of market and competitive intelligence for the software industry, I so agree with your points about differentiation (lack thereof) between most vendors in a particular solution category:
-- Almost all software solutions very quickly become part of highly commoditized markets
-- There are no “silver bullets” for defeating other software vendors
-- Feature/functions/benefits comparisons are much more useful for software vendor internal products teams than for sales teams – hopefully to encourage true innovation and differentiation
-- RFIs / RFPs, and the accompanying processes, do more damage than good
-- It’s much more useful for software vendors to detail how their offerings provide end-to-end solutions for customer scenarios (real needs/problems) than the persisting, and troubling, horizontal platform approach
Cheers,
@juliebhunt
P.S. My thoughts on how most analysts missed the mark on defining real solution markets for Web Content Management:
Web Experience Management software solutions and markets
Thank you very much Dylan, Henrik, Rich, James, and Julie for your comments.
Your feedback is greatly appreciated!
@Dylan — Yes, most research seems more focused on the status quo of the market, and less focused on the true innovation happening in the data quality industry, which like most industries, is generally coming from the smaller vendors, and typically, as you mentioned, startups.
@Henrik — Yes, I agree with your observation about consulting firms. As an independent, I get contacted frequently for contract opportunities, and almost all of them are for projects using a data quality tool from one of the “High Five.” Also, excellent point about “Laissez-Faire” remaining the main competitor of Data Quality.
@Rich — Yes, smaller/niche markets like data quality probably direct more attention both from and to market research. Also, great point about the difference between practice maturity and tool evaluation. And I agree that the “High Five” are often the default for many, especially those too busy to perform a detailed evaluation.
@James — Great point about how it's probably true that “no one ever got fired for picking a tool in the leaders quadrant.” Additionally, I think that focusing on “the right features” not all the features is lost on many vendors, and regardless of their size. One of the the main reasons that the Features Game ends in a tie is precisely because vendors mistake true innovation with “yeah, we have Feature X now too!” This Mediocrity Arms Race (as opposed to specialization and true innovation) results in many data quality tools with “robustly broad but shallow functionality.” At least that's what my chicken bones and tea leaves are telling me.
@Julie — Thanks for your detailed feedback. I definitely agree that the horizontal platform approach, which definitely has potential if done with the right focus, has, for the most part, caused “the quest to build the best platform” resulting simply in a well-integrated, but nonetheless underwhelming, suite of functionality, which often, as you noted, can struggle to provide true end-to-end solutions--which, ironically, was the whole point of building the platform in the first place.
I'd like to believe too but my name isn't Jack and these beans aren't worth...beans!
Confirming who are the biggest players with the most money is not exactly informative but does confirm that they think they are "more bettah."
What ever happened to true research and analysis of emerging technologies and companies in existing markets?
Is it true that only the biggest wallets get on the grid or are the analysts not paid to find these edge players?
Over on the SmartData Collective:
Len Dubois commented:
I believe Magic Quadrants are a tool that clients of Gartner, and any one else that can get their hands on them use as one data point in their decision making process. Analytic reports, like any other data point, are as useful or dangerous as the user wants/needs it to be. From a buyer’s perspective, a MQ can be used for lots of things:
1. To validate a market
2. To identify vendors in the marketplace
3. To identify minimum qualifications in terms of features and functionality
4. To identify trends
5. To determine a company’s viability
6. To justify one’s choice of a vendor
7. To justify value of a purchase
8. Worse case scenario: defends one choice of a failed selection
9. To demonstrate business value of a technology
I also believe they use the analysts, Ted and Andy in this instance, as a sounding board to validate what they believe or learned from other data points, i.e. references, white papers, demos, friends, colleagues, etc. In the final analysis though I know that clients usually make their selection based on many things, the MQ included. One of the most important decisions points is the relationship they have with a vendor or the one they believe they are going to be able to develop with a new
vendor…and no MQ is going to tell you that.
Timo Elliott commented:
The Quadrants clearly do affect the purchasing cycle (which is why people pay for the analysis in the first place, why you chose it as a blog post title, and why I read it!). And the quads do try to take account of at least some of the "why", because the "ability to execute" includes things like customer satisfaction. If nothing else, the quadrants certainly frame the debate and guides the resulting conversations (e.g. "so why should I choose you even though you're not a leader -- what's the discount?"). By the way, anybody who's interested in a behind-the-scenes look at an analyst's life should check out http://analysterical.com, by Gartner analyst Andy Bitterer himself.
AnnMaria commented:
It depends on the knowledge of the people doing the purchasing. In my case, market research such as this would have negligible effect. I'd do my own research, talking to people who are most similar to the industry and needs I am trying to match. If it was something I used, that would be a big factor. I'd get my hands on several products and try doing the things I need done. I might use research like this as further support when I recommend going with product X.
HOWEVER ... when decisions are made by a management team that has limited technical expertise, in my experience, they are influenced more by market research. I've also been (tangentially) involved with purchases when a non-technical team comes back and has decided to go with product Y because of what the market research says. They then have to go back and sell this to their technical staff. Sometimes that works. Sometimes, not so much.
And I replied:
Thank you very much for your comments, Len, Timo, and AnnMaria.
Your feedback is greatly appreciated!
@Len — Excellent point about market research being as useful or as dangerous as the user wants/needs it to be, and thanks for providing the great list of reasons for at least using market research as one of the data points in the purchasing decision. I definitely agree that the relationship formed between vendor and customer is more important than any market research.
@Timo — Great point about the title of my post, "How important is market research?" certainly wouldn't have received many clicks or comments, and I agree that, at the very least, the Gartner Magic Quadrant (and all other market research) certainly helps frame the debate. I too recommend ANALYSTerical - it's almost as excellent as Andy's market analysis, and it's incredibly funny too.
@AnnMaria — Both of your points are excellent. Market research should be balanced by the prospect's own analysis, but sometimes market research is used as the substitute for their lack of experience to perform the research themselves.