Are Applications the La Brea Tar Pits for Data?
Jim Harris in Sponsored Blog Posts tagged Enterprise CIO Forum, HP, IT Thursday, May 12, 2011 at 11:00AM This blog post is sponsored by the Enterprise CIO Forum and HP.
In a previous post, I explained application modernization must become the information technology (IT) prime directive in order for IT departments to satisfy the speed and agility business requirements of their organizations. An excellent point raised in the comments of that post was that continued access to legacy data is often a business driver for not sunsetting legacy applications.
“I find many legacy applications are kept alive in read-only mode, i.e., purely for occasional query/reporting purposes,” explained Beth Breidenbach. “Stated differently, the end users often just want to be able to look at the legacy data from time to time.”
Gordon Hamilton commented that data is often stuck in the “La Brea Tar Pits of legacy” applications. Even when the data is migrated during the implementation of a new application (its new tar pit, so to speak), the legacy data, as Breidenbach said, is often still accessed via the legacy application, which could be dangerous, as Hamilton noted, because the legacy data is diverging from the version migrated to the new application (i.e., after migration, the legacy data could be updated, or possibly deleted).
The actual La Brea Tar Pits were often covered with water, causing animals that came to drink to fall in and get stuck in the tar, thus preserving their fossils for centuries—much to the delight of future paleontologists and natural history museum enthusiasts.
Although they are often cited as the bane of data management, most data silos are actually application silos because historically data and applications have been so tightly coupled. Data is often covered with an application layer, causing users that enter, access, and use the data to get stuck with the functionality provided by its application, thus preserving their use of the application even after it has become outdated (i.e., legacy)—much to the dismay of IT departments and emerging technology enthusiasts.
When so tightly coupled with data, applications—not just legacy applications—truly can be the La Brea Tar Pits for data, since once data needed to support business activities gets stuck in an application, that application will stick around for a very long time.
If applications and data were not so tightly coupled, we could both modernize our applications and optimize our data usage in order to better satisfy the speed and agility business requirements of our organizations. Therefore, not only should we sunset our legacy applications, we should also approach data management with the mindset of decoupling our data from its applications.
This blog post is sponsored by the Enterprise CIO Forum and HP.
Related Posts
A Sadie Hawkins Dance of Business Transformation
Why does the sun never set on legacy applications?
The IT Pendulum and the Federated Future of IT
Suburban Flight, Technology Sprawl, and Garage IT
Reader Comments (8)
Good post, Jim. I never really thought about it that way. The legacy data shouldn't be viewed in isolation. Will getting to it via legacy applications introduce problems, including security concerns?
Thanks for your comment, Phil. Accessing data through legacy applications can introduce problems, since the functionality of the legacy application may lack the required data security (as you noted), as well as other data and process controls, which likely are among the features and benefits of the new application(s) attempting to supplant the legacy application.
This is a difficult challenge for IT, which may be providing the new technology needed to properly support the current data-related business requirements of the organization, but when the data needed to support business activities remains stuck in (or is simply still used from within) legacy applications, data and process controls can be too easily circumvented.
Excellent post, Jim.
It reminded me of John Morris' admonition in Practical Data Migration to always have the System Retirement Policies in place early in your data migration plan. John stressed, as your post does, that unless there is a formal policy to retire the legacy application after the migration it will in all likelihood "stick around".
John suggests that the system retirement policy should contain a set of minimum functionality that the new system must achieve before the legacy system can be retired. Then when the new system reaches that minimum level of functionality then the retirement policy kicks in, and since the retirement policy is already approved, there really is no argument for keeping it hanging around - "stinkin' up the joint" with it's old moldy data droppings.
Cheers, Gordon
Ahhh...the good old really, really old La Brea Tar Pits!
There must be tools out there to liberate relevant data from old applications no longer needed. I know I can get years-old brokerage and banking account data...I suspect that data was liberated from legacy applications, no? Are there applications where it is impossible to migrate the data to a more modern application? I suspect it's easier in some cases just to keep the legacy application going to allow access to the imprisoned data, no?
Great post, we have an obligation to hold, and occasionally make available, 7 years worth of financial data. Doesn't mean online however.
In one recent estimate, the mapping of the old schema to our new application to keep up our end of the supposed bargain was twice the estimate of running two systems until we reached our archive point where the data is stored offline or shipped via DVD to the customer!
When we ran stats on the access, no one had accessed greater than 2 years worth.... I quizzed the account folks to find out that 'we always do it, don't know why, we didn't think it cost anything to hold it there'. Groan...
Thanks Gordon, John, and Sid for your comments.
@Gordon — I also laud John Morris' recommended best practice of System Retirement Policies — but I must lament that most organizations do not follow it :-)
@John — It's definitely easier to keep the legacy application going to allow access to the imprisoned data — and this path of least resistance is well traveled by most organizations :-)
@Sid — Yes, many organizations follow data management default practices instead of best practices . . . Groan, indeed :-)
From the LinkedIn Group for Enterprise CIO Forum, Pearl Zhu commented:
“Hi, Jim, as the other popular discussion you posted about Legacy Application Modernization, this is also a great topic on how critical it is to optimize the data in modern organizations. In today's information explosive era, the enterprise is all about data--structured data, unstructured data, legacy data, near real time data--the whole information management solution is also data management, MDM, data warehouse, data integration, data optimization., etc. Since the right type of data are the fundamentals of BI (one of the modern applications), analytics sources, and the need for more real-time insight and foresight, as well as expedited good decision making, legacy (outdated) data causes all sorts of hassles.”
And I responded:
Thanks for another great comment, Pearl.
Yes, it could be easily argued that outdated data could be more dangerous to the organization than outdated applications since data-driven decisions will obviously be affected by the timeliness (and other quality characteristics) of the data.
And, as you noted, the variety of data is increasing as much as its volume, and the required velocity of effective business decisions is getting nearer and nearer to real time.
Best Regards,
Jim
Great post as ever Jim.
As Gordon above points out, by creating System Retirement Policies at the start of the migration we immediately put on the table the thorny issues of who is going to take responsibility for shutting down this old application and more importantly, what are the precise conditions they need to see before it can happen. This could be testing of the new application and data, archival of the old data somewhere, reconciliation validation, it varies.
A big problem though is when these systems persist and that typically means we've not fully understood all of our to-be functions. If people still need to keep diving into the old system to perform valid business activities then we have failed to:
a) Replicate the legacy data on the new database
b) Replicate the legacy business function on the new application
c) Modify our business process to terminate that legacy function
I've seen lots of issues with legacy systems persisting beyond their close-down date, people taking copies of data to create local spreadsheets because they prefer the data in the "old format" for example.
One area I am seeing growth is where people are using MDM technologies to help support data migration projects, they effectively uncouple the legacy data from its application, expose it to the new application and perform a faster and less riskier migration. I think this emphasizes your point about the benefit of having less forced linkage between data and applications in modernized systems.