Big Data el Memorioso
Jim Harris in Sponsored Blog Posts tagged Big Data, Enterprise CIO Forum, HP Thursday, January 19, 2012 at 11:00AM This blog post is sponsored by the Enterprise CIO Forum and HP.
Funes el memorioso is a short story by Jorge Luis Borges, which describes a young man named Ireneo Funes who, as a result of a horseback riding accident, has lost his ability to forget. Although Funes has a tremendous memory, he is so lost in the details of everything he knows that he is unable to convert the information into knowledge and unable, as a result, to achieve wisdom.
In Spanish, the word memorioso means “having a vast memory.” Without question, Big Data has a vast memory comprised of fast-moving large volumes of varying data seemingly providing details about everything your organization could ever want to know about our increasingly digitized and pixelated world. But what if Big Data is the Ireneo Funes of the Information Age?
What if Big Data el Memorioso is the not-so-short story in which your organization becomes so lost in the details of everything big data delivers that you’re unable to connect enough of the dots to convert the information into knowledge and unable, as a result, to achieve the wisdom necessary to satisfice specific business needs?
Adrian Bridgwater recently compared this challenge to “trying to balance a stack of papers on a moving walkway, in a breeze, without knowing the full length or speed of the walkway itself. If you want to extend the metaphor one step further — there are other passengers on our walkway and they could bump into us and/or add papers to our stack. Oh, did I mention that the pieces of paper might not even all be the same size, shape, or color — and some may have tattered edges and coffee stains?”
In other words, as Bridgwater went on to explain, “our information optimization goals will typically include the need to manage information and assess its quantitative and qualitative values. We will also need to analyze streams of both structured and unstructured data, the latter including video, emails, and other less ‘straight edged’ data.”
While examining some of the technology options that can assist with this challenge, Paul Muller recently remarked “whether it be structured, unstructured, big, small, real-time, or historical — data of all kinds are top-of-mind for business executives. It may already feel like you’re drowning in data, but it’s important to get to grips with the changing technology landscape to ensure you’re not drowning in an incoherent mess of information management architectures too.”
Edd Dumbill recently wrote an introduction to the big data landscape, which concluded that “big data is no panacea. You can find patterns and clues in your data, but then what?” As Dumbill recommends, you need to know where you want to go. You need to know what problem you want to solve, i.e., you need to pick a real business problem to guide your implementation.
Without this implementation guide, big data will have, as Borges said of Funes, “a certain stammering greatness,” but amount to, as William Shakespeare said in The Tragedy of Macbeth, “a tale told by an idiot, full of sound and fury, signifying nothing.”
This blog post is sponsored by the Enterprise CIO Forum and HP.
Related Posts
Neither the I Nor the T is Magic
Information Overload Revisited
A Decision Needle in a Data Haystack
Dot Collectors and Dot Connectors
Data, Information, and Knowledge Management
Is your data complete and accurate, but useless to your business?
The Real Data Value is Business Insight
Reader Comments (3)
From the LinkedIn Group for Enterprise CIO Forum, Pearl Zhu commented:
“Hi, Jim, enjoy your big data wisdom starting with an interesting little story. I would say, big data could mean more of a culture evolution than a technology evolution, the point is: instead of counting on the gut feeling to make decisions, big data now encourages us to pursue data-driven patterns and facts, then, as you pointed out, big data is the way to the end, not the end, the purpose of analytics is not to get stuck to the details, but accumulate the wisdom, in order to spark more innovative products or services, optimize customer experience or improve the working environment, etc.”
And I responded:
Thanks for your excellent (as always) comment, Pearl.
I definitely agree with you that big data is more of a corporate cultural evolution than a technology evolution. But since technology is almost always easier to work with than corporate culture, I fear that some organizations will simply try throwing technology (e.g., Hadoop and NoSQL) at big data only to be disappointed with the results, or lack thereof.
Best Regards,
Jim
I think his helps illustrate that one size does not fit all. You can't take a singular approach to how you design for big data. It's all about identifying relevance and understanding that relevance can change over time.
There are certain situations where it makes sense to leverage all of the data, and now with high performance computing capabilities that include in-memory, in-DB and grid, it's possible to build and deploy rich models using all data in a short amount of time. Not only can you leverage rich models, but you can deploy a large number of models that leverage many variables so that you get optimal results. On the other hand, there are situations where you need to filter out the extraneous information and the more intelligent you can be about identifying the relevant information the better.
The traditional approach is to grab the data, cleanse it and land it somewhere before processing or analyzing the data. We suggest that you leverage analytics up front to determine what data is relevant as it streams in, with relevance based on your organizational knowledge or context. That helps you determine what data should be acted upon immediately, where it should be stored, etc. And of course there are considerations about using visual analytic techniques to help you determine relevance and guide your analysis, but that's an entire subject just on its own!
Mark Troester
SAS
Twitter: @mtroester
From the TDWI Business Intelligence and Data Warehousing LinkedIn Group, Lalitha Nataraj commented:
“Nice one. Can you find me some examples of business problems which have been tackled by big data?”
And I responded:
Thanks for your comment and question, Lalitha.
The easiest examples of business problems tackled by big data come from some of the biggest Internet companies.
Google, whose description of their proprietary MapReduce algorithm served as the inspiration for the open source development of Hadoop, is a company built on big data with their indexing and ranking of web pages driving their search engine dominance, and their per-click advertising business model that still accounts for approximately 80% of their financial success. Amazon and Apple are excellent examples of platforms built on the big data of online sales transactions and recommendation engines, and the Facebook empire is built on the big data of social networking.
Of course, these companies are also examples of the potential dark side of big data, as shown by the data privacy implications and related legal and government regulatory challenges plaguing Google and Facebook.
However, companies of all sizes, and in all industries, are starting to investigate how big data can help with their business problems, and so in the coming months and years, I believe that we will be hearing more of those more practical stories and real-world case studies.
Best Regards,
Jim
And Jaime Rubio commented: More examples out of the cloud, but on the ground, where you’d have to tap Big Data:
- Financial transactions (fraud detection, accounting)
- Insurance liability calculation
- Cell phone calls analysis for rate plan design
- Daily purchases on a big retail chain
- Votation polls
And I responded: Thanks Jaime for providing some excellent additional examples of the usefulness of Big Data.