OOBE-DQ, Where Are You?
Jim Harris in Data Quality, Debates, Technology tagged Philosophy Saturday, January 9, 2010 at 8:53PM Much of enterprise software is often viewed as a commercial off-the-shelf (COTS) product, which, in theory, is supposed to provide significant advantages over bespoke, in-house solutions. In this blog post, I want to discuss your expectations about the out-of-box-experience (OOBE) provided by data quality (DQ) software, or as I prefer to phrase this question:
OOBE-DQ, Where Are You?
Common DQ Software Features
There are many DQ software vendors to choose from and all of them offer viable solutions driven by impressive technology. Many of these vendors have very similar approaches to DQ, and therefore provide similar technology with common features, including the following (Please Note: some vendors have a suite of related products collectively providing these features):
- Data Profiling
- Data Quality Assessment
- Data Standardization
- Data Matching
- Data Consolidation
- Data Integration
- Data Quality Monitoring
A common aspect of OOBE-DQ is the “ease of use” vs. “powerful functionality” debate—ignoring the Magic Beans phenomenon, where the Machiavellian salesperson guarantees you their software is both remarkably easy to use and incredibly powerful.
So just how easy is your Ease of Use?
“Ease of use” can be difficult to qualify since it needs to take into account several aspects:
— Installation and configuration
— Integration within a suite of related products (or connectivity to other products)
— Intuitiveness of the user interface(s)
— Documentation and context sensitive help screens
— Ability to effectively support a multiple user environment
— Whether performed tasks are aligned with different types of users
There are obviously other aspects, some of which may vary depending on your DQ initiative, your specific industry, or your organizational structure. However, the bottom line is hopefully the DQ software doesn't require your users to be as smart as Brainiac (pictured above) in order to be able to figure out how to use it, both effectively and efficiently.
DQ Powers—Activate!
Ease of use is obviously a very important aspect of OOBE-DQ. However, as Duke Ellington taught us, it don't mean a thing, if it ain't got that swing—in order words, if it's easy to use but can't do anything, what good is it? Therefore, powerful functionality is also important.
“Powerful functionality” can be rather subjective, but probably needs to at least include these aspects:
— Fast processing speed
— Scalable architecture
— Batch and near real-time execution modes
— Pre-built functionality for common tasks
— Customizable and reusable components
Once again, there are obviously other aspects, especially depending on the specifics of your situation. However, in my opinion, one of the most important aspects of DQ functionality is how it helps (as pictured above) enable Zan (i.e., technical stakeholders) and Jayna (i.e., business stakeholders) to activate their most important power—collaboration. And of course, sometimes even the Wonder Twins needed the help of their pet space monkey Gleek (i.e., data quality consultants).
OOBE-DQ, Where Are You?
Where are you in the OOBE-DQ debate? In other words, what are your expectations when evaluating the out-of-box-experience (OOBE) provided by data quality (DQ) software?
Where do you stand in the “ease of use” vs. “powerful functionality” debate?
Are there situations where the prioritization of ease of use makes a lack of robust functionality more acceptable?
Are there situations where the prioritization of powerful functionality makes a required expertise more acceptable?
Please share your thoughts by posting a comment below.
Follow OCDQ
If you enjoyed this blog post, then please subscribe to OCDQ via my RSS feed or my E-mail updates.
You can also follow OCDQ on Twitter, fan the Facebook page for OCDQ, and connect with me on LinkedIn.
Reader Comments (9)
Good post Jim.
In my daily work, I use two different tools depending on what problem is to be solved.
The one tool is an easy to use desktop application and the other tool is a full server framework.
Some while ago, I blogged about my experience using the analogy of driving data quality in two lanes.
Interesting and insightful post Jim.
For us, the whole ease of use vs. functionality debate was included in the business case for the purchase. We identified the business requirements, included pros and cons of the ease of use vs. functionality and made recommendations on the vendors based on the results of the pros vs. cons vs. requirements.
Also important to note, and included in our business case, was to question if the “ease of use” requires an intensive effort or costly training program, especially if your goal is to engage business users.
So to answer your questions, I would say if you have your requirements identified, and you do your homework on the benefits/risks/costs of the software, you should have all the information you need to make a logical decision based on the present situation. Which, of course, will change somewhere down the line as everything does.
And for goodness sake (did I say goodness!), when things do change, always start with identifying the requirements.
Never assume the requirements are the same as they used to be :)
Thanks!
Jim,
Interesting post. I know more about enterprise software than DQ-specific applications, but I find the tradeoff to be similar to other applications with which I have worked.
Being a self-admitted geek, I will always opt for more power at the risk of "user-friendliness."
I know that I can always figure out "Band-Aids", including importing "cleansed" data into the system through different means. For the general public, though, I suspect that they would err on the side of user-friendliness.
Of course, I'm not sure about how many people in an organization would actually use different DQ tools.
Is this solely the domain of the "super user"?
Let's be controversial ....
"There are many DQ software vendors to choose from and all of them offer viable solutions driven by impressive technology." - I think we must be living on different planets, Jim.
When it comes to international data, you can count the number of solutions that actually improve data quality on the fingers of one hand. Maybe it's because I understand what I see when the data comes out, but I am largely unimpressed by most offerings.
As Phil Simon said in his blog post, Data Chaos and Five Truisms of Data Quality, (and I am paraphrasing) - with international data, all bets are off!
Graham
@Henrik I really like your “driving data quality in two lanes” analogy.
Do you see any distinction such as one tool for business users and one tool for technical users? Or is the distinction, more to the point you stated, based on the specific problem being solved?
@Jill Excellent overview of using a detailed business case and business requirements to drive the vendor selection process.
I am always amazed more organizations don't do this (although many will claim they do, of course). And great point about the need to always verify that previously identified requirements are still representative of the organization's current needs.
@Phil Your question about whether DQ is the solely the domain of the super user is perhaps fundamental to this debate. Some tools seem to be built with only the super user in mind, putting “ease of use” in the eye of the beholder, so to speak, since many complex tasks actually do appear easy to super users and Gleeks (i.e., expert consultants).
@Graham Great point!
And by the way, I think we are living on the same planet - I just occasionally broadcast from Alpha Centari to play “nice-nice” with all my vendor friends. :-)
Thanks for your comments, your feedback is greatly appreciated.
Jim, to answer your question - I guess there is always a business (oriented) user involved. What has worked for me several times is using the user friendly tool in an interactive process with business users solving downstream data quality issues, and then taking the proven business rules to upstream processes in more technical focused implementations.
Graham's remarks on international data reminds me in my most recent international solution that deals with matching B2B customer data from all over the world with the D&B Worldbase holding +150 million entities, I used three different tools:
(1) For candidate selection
(2) For advanced similarity assignment
(3) For building a dedicated user interface being friendly for that particular purpose
Interesting debate Jim.
I think one of the most frustrating things I saw recently was a vendor who had managed to submit a link on Wikipedia (a feat in itself believe me!) that basically pitched something along the lines of "...forget all those expensive high-end solutions, what you need is our sub $500 solution that fixes data quality in an afternoon...".
Out of the box is one thing but a complete solution in an afternoon?
I think the most important trend in recent years is where vendors are really starting to understand how DQ workflows should integrate with the knowledge workforce. I'm seeing several products really get this and create simple UI's and functions based on the role of the knowledge worker involved. These tools have a great balance between usable interface for business specific roles but also a great deal of power features under the bonnet. That is the software I typically recommend but again it is also about budget, these solutions may be too expensive for some organisations.
There is a danger here though of adding powerful features to knowledge workers who don't fully understand the ramifications of committing those updates to that master customer list. I still think we'll see IT playing a major role in the DQ process for some time to come, despite the business-focused marketing we're seeing in vendor land.
Another good post Jim.
I'm with Henrik & Graham on this one.
“Ease of Use” vs. “Powerful Functionality” should be applied the the user skills and experience.
We've written a fair bit of software around data cleansing and data quality for our own clients requirements. From an industry strength point of view we have always written them to be powerful, as the users are internal - already trained and skilled in the data processes.
However, in reality, these would not be commercially acceptable as they are, without simplifying and improving user interaction, i.e., best performance with easiest use. Tweaking these processes will ensure that the either performance or functionality is compromised. Only the users acceptable threshold can guide the software.
Yes Graham you are right about International data - reality sadly means that out of the box solutions for this type of data will always provide an "ok" job at best. This again goes back to skill and user experience. This is certainly a niche area, finding skilled people is the challenge here.
Perhaps we should be talking about Common DQ User Features :)
From the Data Cleansing User Group on LinkedIn, Jacqueline Roberts commented:
“Excellent post, but at an extremely high frustrated moment of a third release of a COTS solution in the PIM category of the "magic quadrant" and there seems to be a lot of magic in evaluation in the PIM category.
I believe that the design focus is on the software developer perception of data processing in a perfect world, leaving out the data issues / data processes. What was missing in the team development model is the expertise of the business data analysts.
We now officially operate in a "work around" model of data cleansing and processing, which completely goes against my belief of Master Data Management and Data Provenance practices, but I also have a data job to do which requires mass data through put.”