Promoting Poor Data Quality
Jim HarrisA few months ago, during an e-mail correspondence with one of my blog readers from Brazil (I’ll let him decide if he wishes to remain anonymous or identify himself in the comments section), I was asked the following intriguing question:
“Who profits from poor data quality?”
The specific choice of verb (i.e., “profits”) may have been a linguistic issue, by which I mean that since I don’t know Portuguese, our correspondence had to be conducted in English.
Please don’t misunderstand me—his writing was perfectly understandable.
As I discussed in my blog post Can Social Media become a Universal Translator?, my native language is English, and like many people from the United States, it is the only language I am fluent in. My friends from Great Britain would most likely point that I am only fluent in the American “version” of the English language, but that’s a topic for another day—and another blog post.
When anyone communicates in another language—and especially in writing—not every word may be exactly right.
For example: Muito obrigado por sua pergunta!
Hopefully (and with help from Google Translate), I just wrote “thank you for your question” in Portuguese.
My point is that I believe he was asking why poor data quality continues to persist as an extremely prevalent issue, especially when its detrimental effects on effective business decisions has become painfully obvious given the recent global financial crisis.
However, being mentally stuck on my literal interpretation of the word “profit” has delayed my blog post response—until now.
Promoting Poor Data Quality
In economics, the term “flight to quality” describes the aftermath of a financial crisis (e.g., a stock market crash) when people become highly risk-averse and move their money into safer, more reliable investments. A similar “flight to data quality” often occurs in the aftermath of an event when poor data quality negatively impacted decision-critical enterprise information.
The recent recession provides many examples of the financial aspect of this negative impact. Therefore, even companies that may not have viewed poor data quality as a major risk—and a huge cost greatly decreasing their profits—are doing so now.
However, the retail industry has always been known for its paper thin profit margins, which are due, in large part, to often being forced into the highly competitive game of pricing. Although dropping the price is the easiest way to sell just about any product, it is also virtually impossible to sustain this rather effective, but short-term, tactic as a viable long-term business strategy.
Therefore, a common approach used to compete on price without risking too much on profit is to promote sales using a rebate, which I believe is a business strategy intentionally promoting poor data quality for the purposes of increasing profits.
You break it, you slip it—either way—you buy it, we profit
The most common form of a rebate is a mail-in rebate. The basic premise is simple. Instead of reducing the in-store price of a product, it is sold at full price, but a rebate form is provided that the customer can fill out and mail to the product’s manufacturer, which will then mail a rebate check to the customer—usually within a few business weeks after approving the rebate form.
For example, you could purchase a new mobile phone for $250 with a $125 mail-in rebate, which would make the “sale price” only $125—which is what the store will advertise as the actual sale price with “after a $125 mail-in rebate” written in small print.
Two key statistics significantly impact the profitability of these type of rebate programs, breakage and slippage.
Breakage is the percentage of customers who, for reasons I will get to in a moment, fail to take advantage of the rebate, and therefore end up paying full price for the product. Returning to my example, the mobile phone that would have cost $125 if you received the $125 mail-in rebate, instead becomes exactly what you paid for it—$250 (plus applicable taxes, of course).
Slippage is the percentage of customers who either don’t mail in the rebate form at all, or don’t cash their received rebate check. The former is the most common “slip,” while the latter is usually caused by failing to cash the rebate check before it expires, which is typically 30 to 90 days after it is processed (i.e., expiration dated)—and regardless of when it is actually received.
Breakage, and the most common form of slippage, are generally the result of making the rebate process intentionally complex.
Rebate forms often require you to provide a significant amount of information, both about yourself and the product, as well as attach several “proofs of purchase” such as a copy of the receipt and the barcode cut out of the product’s package.
Data entry errors are perhaps the most commonly cited root cause of poor data quality.
Rebates seem designed to guarantee data entry errors (by encouraging the customer to fill out the rebate form incorrectly).
In this particular situation, the manufacturer is hyper-vigilant about data quality and for an excellent reason—poor data quality will either delay or void the customer’s rebate.
Additionally, the fine print of the rebate form can include other “terms and conditions” voiding the rebate—even if the form is filled out perfectly. A common example is the limitation of “only one rebate per postal address.” This sounds reasonable, right?
Well, one major electronics manufacturer used this disclaimer to disqualify all customers who lived in multiple unit dwellings, such as an apartment building, where another customer “at the same postal address” had already applied for a rebate.
Conclusion
Statistics vary by product and region, but estimates show that breakage and slippage combine on average to result in 40% of retail customers paying full price when making a purchasing decision based on a promotional price requiring a mail-in rebate.
So who profits from poor data quality? Apparently, the retail industry does—sometimes.
Poor data quality (and poor information quality in the case of intentionally confusing fine print) definitely has a role to play with mail-in rebates—and it’s a supporting role that can definitely lead to increased profits.
Of course, the long-term risks and costs associated with alienating the marketplace with gimmicky promotions take their toll.
In fact, the major electronics manufacturer mentioned above was actually substantially fined in the United States and forced to pay hundreds of thousands of dollars worth of denied mail-in rebates to customers.
Therefore, poor data quality, much like crime, doesn’t pay—at least not for very long.
I am not trying to demonize the retail industry.
Excluding criminal acts of intentional fraud, such as identity theft and money laundering, this was the best example I could think of that allowed me to respond to a reader’s request—without using the far more complex example of the mortgage crisis.
What Say You?
Can you think of any other examples of the possible benefits—intentional or accidental—derived from poor data quality?
Reader Comments (8)
Poor data quality is welcomed whenever there are no rules or procedures in place to make a sound decision.
Whenever gut feeling or intuition or anecdotal evidence based decisions are to be justified upon failure, it is better to have figures to bend than certified numbers to rely upon.
I've seen, in companies with large margins, directors take initiatives just to make something new and compelling. Intentionally, no metrics to measure them was set up to claim success in any case.
Luckily there are few of them.
Interesting line of thinking.
Eventually, there's probably only more money in taxes. Costs coming from poor quality data and business processes have to be calculated into the prices of the products the companies sell.
This way leading to higher (than necessary) turnovers and VAT. The ironic part is that the extra "Value Added" tax resulting from poor data quality is actually "Negative Value Extracted Tax" or NVET.
Added to that the fines these companies have to pay, all leads to a higher cash level at your local treasury department. That's how they can keep the taxes so low!
Thanks for your comments, Augusto and Frank. Your feedback is greatly appreciated.
@Augusto (aka Stray__Cat) — Excellent points about how poor data quality is welcomed wherever no rules or formal procedures have been established that would support making sound business decisions, as well as how anecdotal evidence is far easier to manipulate than certified data analysis.
@Frank — I really like the concept of a NVET (Negative Value Extracted Tax), perhaps an international Poor Data Quality (PDQ) Tax should be established. This might get more organizations to prioritize data quality PDQ (pretty damn quick) :-)
So I've captured your unconscious attention? Cool ...
Let´s solve the enigma: I think quality - especially data quality - is not a natural, instinctive human behavior. On the contrary - quality data is dangerous!
Except for people like you and me - obsessive compulsive is a perfect definition of us - who cares about data quality?
Nobody! Only the ones with their money on fire - the owners. But they still don´t know that is perfectly possible to save tons of dollars if you record data right the first time!
Everyone else - IT folks, Call Center folks, Lawyers, Administrators, Used car sellers, everybody profits when data quality is asymmetrical - there is a Nobel Prize about this - Joseph Stiglitz, 2001.
There is a whole scrap-and-rework do it again industry, waiting for desperate calls, selling solutions that will never fix the real problem - poor data quality.
Information is power, information is money.
What do you think?
@Blog reader from Brazil — In 2009, the total international revenue generated from the sale of data quality software licenses was estimated at $400 million (USD). Therefore, and especially since that estimate doesn't even include the cost of the related professional services (i.e., training, consulting, and staff augmentation), I definitely agree that there is a very profitable scrap-and-rework industry built upon the very sturdy foundation of poor data quality.
Bringing the concept of information asymmetry, for which George Akerlof, Michael Spence, and Joseph Stiglitz were awarded the 2001 Nobel Prize in Economics, into this discussion is an excellent point as well.
Information asymmetry certainly applies to the retail industry example I used, as well as other situations where the seller has better information than the buyer, including the used-car salespeople you mentioned, as well as mortgage brokers, loan originators, stockbrokers, Realtors, and real estate agents (all of whom had their parts to play, knowingly or not, in contributing to the complex causes of the global financial crisis).
Quoting Joseph Stiglitz, “Information economics represents a fundamental change in the prevailing paradigm within economics. Problems of information are central to understanding not only market economics but also political economy. Unfettered markets often not only do not lead to social justice, but do not even produce efficient outcomes. Interestingly, there has been no intellectual challenge to the refutation of Adam Smith’s invisible hand: individuals and firms, in the pursuit of their self-interest, are not necessarily, or in general, led as if by an invisible hand, to economic efficiency.”
So in conclusion, I completely agree with your premise that “information is power, information is money.” The paradox is although this drives organizations to buy the very data quality solutions being sold by software and consulting vendors, that I believe are actually trying to help in the vast majority of cases, there is also a silent vested market interest to never actually solve the poor data quality problem -- because there is too much money to be made from its persistence.
Via e-mail, Joe Erl commented:
Fraudsters (internal and external) also benefit.
If data quality is poor, then it is easier to commit fraud; but once fraud is committed poor data quality makes investigation of the fraud network harder.
Interesting post, Jim.
I can think of a few more people that profit from poor quality data: banks, insurers and software vendors.
Banks and insurers can profit if beneficiaries of policies cannot be found for payment. Although there are penalties and escrows to be paid, in some cases not paying the policy out can be beneficial. Software vendors who make data quality tools certainly profit from poor data quality as well :)
I will be thinking of this more in the future to determine if I have a client that might profit from data quality issues.
A good post makes you think of things in a new way. This one was good!
In the case of Name and Address data quality (or Contact Information data quality, in general), there are some definite benefits for Data Quality service providers, if the input data quality remains at a relatively low level.
For instance, there are businesses out there nowadays that will append data like email, phone number, mobile number, social network ids, etc... to the names and addresses stored in company's marketing database.
It's a lot easier to get a "match" if the name and address coming in is a bit more ambiguous - a last name of "Smith" or "Wong" in high-rise apartment apartment, for example, is MUCH easier to match than the higher quality "Marcus Smith" in "Apartment 13a". The problem is, that email you appended off of the ambiguous match is far less likely to actually be the person you're trying to market to... and therefore a lot more likely not to respond to your marketing efforts.
Like anything else, the more criteria you add to a search (or, the higher the data quality), the more you narrow the list of results. Since most companies either charge by the rate at which they append, or rely on a high append rate and price per M, there are significant dollars involved.