“Hi, Jim, nice to see you expand the data wind into cloud wings, the blog is resourceful by puling and well-organizing a few good cloud blogs to deliver more holistic view of cloudification:

-- Legacy Infrastructure Modernization and Convergence: Cloud provides the unprecedented opportunities for business to re-visit their EA, ITA, DA, and manage their information system with elasticity and agility

-- Application Portfolio Rationalization: SAAS solutions may also help business to revitalize their application portfolio via consolidation, integration, rationalization and optimization

-- From CapEx to OpEx: Beyond fast delivery and easy provisioning, Cloud has the economical flexibility to allow businesses to experiment with new innovations, delight customers and engage employees”

And I responded:

Thanks for the additional insights about the benefits of cloudification, Pearl :-)

And Paul Calento commented:

“Specific to your mention of converged infrastructure, one of the challenges isn’t the opportunity itself, but the fact that for many companies the term doesn’t reflect harmony but (at least in some part) orphaned systems and applications that exist and can't be merely ripped and replaced.

In that scenario, instead of calling it converged infrastructure you could call it duct tape and baling twine infrastructure.

One key, perhaps to resolving this and the as a service trend is starting with hard choices during the asset evaluation and management state.”

@Jen — Yes, I too have used a library card catalog analogy in the past, as well as a music analogy based on albums and album covers, back when albums were compact disks or vinyl records that you could use for a show-and-tell presentation in real life. I came up with the iTunes analogy after being chided by a much younger member of my extended family who was confused why there were no songs in my iTunes library — and even more confused by my attempted explanation of music that didn’t come from the Internet :-)

@Crysta — Pandora (which I am listening to as I type this) is another fascinating example since metadata is needed in order for Pandora to create radio stations for you by artist, track, or composer, as well as the great metadata that they have for lyrics (my favorite feature), artist biographies, and similar artists.

Pandora’s Music Genome Project is a great example of the power of combining metadata and data, which uses almost 400 song attributes and a complex data mining algorithm to organize them, which also discovers other songs and artists similar to the ones you indicate that you like (data quality geeks like me would call this a data matching user feedback process).

“Nice one. Can you find me some examples of business problems which have been tackled by big data?”

And I responded:

Thanks for your comment and question, Lalitha.

The easiest examples of business problems tackled by big data come from some of the biggest Internet companies.

Google, whose description of their proprietary MapReduce algorithm served as the inspiration for the open source development of Hadoop, is a company built on big data with their indexing and ranking of web pages driving their search engine dominance, and their per-click advertising business model that still accounts for approximately 80% of their financial success. Amazon and Apple are excellent examples of platforms built on the big data of online sales transactions and recommendation engines, and the Facebook empire is built on the big data of social networking.

Of course, these companies are also examples of the potential dark side of big data, as shown by the data privacy implications and related legal and government regulatory challenges plaguing Google and Facebook.

However, companies of all sizes, and in all industries, are starting to investigate how big data can help with their business problems, and so in the coming months and years, I believe that we will be hearing more of those more practical stories and real-world case studies.

Best Regards,

Jim

And Jaime Rubio commented: More examples out of the cloud, but on the ground, where you’d have to tap Big Data:

- Financial transactions (fraud detection, accounting)
- Insurance liability calculation
- Cell phone calls analysis for rate plan design
- Daily purchases on a big retail chain
- Votation polls

And I responded: Thanks Jaime for providing some excellent additional examples of the usefulness of Big Data.

There are certain situations where it makes sense to leverage all of the data, and now with high performance computing capabilities that include in-memory, in-DB and grid, it's possible to build and deploy rich models using all data in a short amount of time. Not only can you leverage rich models, but you can deploy a large number of models that leverage many variables so that you get optimal results. On the other hand, there are situations where you need to filter out the extraneous information and the more intelligent you can be about identifying the relevant information the better.

The traditional approach is to grab the data, cleanse it and land it somewhere before processing or analyzing the data. We suggest that you leverage analytics up front to determine what data is relevant as it streams in, with relevance based on your organizational knowledge or context. That helps you determine what data should be acted upon immediately, where it should be stored, etc. And of course there are considerations about using visual analytic techniques to help you determine relevance and guide your analysis, but that's an entire subject just on its own!

Mark Troester
SAS
Twitter: @mtroester

“Great work Jim, thanks! This is one area that organizational/technical/business tactics are key.

The biggest challenge is getting from maturity level 0 to level 1 - puzzle analogy: corners and edges first. Developing repeatable and proven approaches that are tailored to each company's unique situation is the thing that makes establishing Data Governance so challenging (and fun). This is a cross discipline activity that requires Business, Organizational, Intrapreneurial, Technical, and Interpersonal smarts. A tall order indeed.”

And I responded:

Thanks for your great comment, John.

Corners and edges first, indeed. And, as you noted, each company's unique situation will determine where those corners and edges will be as the data governance journey begins.

Best Regards,

Jim

“Hi, Jim, enjoy your big data wisdom starting with an interesting little story. I would say, big data could mean more of a culture evolution than a technology evolution, the point is: instead of counting on the gut feeling to make decisions, big data now encourages us to pursue data-driven patterns and facts, then, as you pointed out, big data is the way to the end, not the end, the purpose of analytics is not to get stuck to the details, but accumulate the wisdom, in order to spark more innovative products or services, optimize customer experience or improve the working environment, etc.”

And I responded:

Thanks for your excellent (as always) comment, Pearl.

I definitely agree with you that big data is more of a corporate cultural evolution than a technology evolution. But since technology is almost always easier to work with than corporate culture, I fear that some organizations will simply try throwing technology (e.g., Hadoop and NoSQL) at big data only to be disappointed with the results, or lack thereof.

Best Regards,

Jim