Hadoop, there it is!

Editor, IBM Big Data & Analytics Hub, IBM

In 2013, we learned all about Hadoop as subject matter experts from around the globe put pen to paper (or stepped in front of the camera or microphone) and graciously mapped their angle on this game-changing open source software framework.

Last January, some of the industry’s boldest thought leaders laid out predictions for how they believed Hadoop (and similar tools) would evolve and be used in the future. The prevalent theme was that Hadoop itself is not the sum total of big data, and successful companies will require a custom-fit solution that is easier to use as time goes on. Usability was key with experts like Big Data Product Marketing Manager Christy Maver as she explained how software like InfoSphere BigInsights answers the question “What can I do with this data?”

More detail on exactly how IBM InfoSphere BigInsights works with the Hadoop framework to produce superior results can be found in this solution demo where Nicolas Morales details innovative capabilities like automatic refresh of underlying data, repeat reports on your schedule and ease of export to a variety of data formats and direct upload to your data warehouse.


Tina Groves warned us about the potential pitfalls of eliminating the data warehouse and moving completely to Hadoop, one of which comes in the form of double counting: when results are overstated due to referencing a transaction more than once. Valuable advice to take into 2014.

For those intimidated by the very idea of big data, don’t panic—Eric Sall assures us that “big data is for everybody” in his informative podcast on the five big data use cases. In particular, he highlights Constant Contact’s use of InfoSphere BigInsights to analyze 35 billion annual emails in order to advise clients of best dates and times to send emails. Want to learn more about how Constant Contact is using this IBM software to decode big data? David Pittman points out this great accompanying video during his interview with Sall, and it is embedded here for your convenience:


To round out a year of solid Hadoop tips, coverage and detailed explanantions, Tina Groves implores us to ask not what your BI tool can do for you, but to instead consider “What is the best way to leverage a Hadoop infrastructure with my BI tool?” After all, ability does not necessarily indicate readiness. Read more in her post "Where does Hadoop fit in a Business Intelligence data strategy?"

Because some readers are really scrollers, I've served up a big data infographic to demonstrate how Hadoop plays a role in taming big data. As this framework continues to evolve each year, we're certain to see exponential increase in sophistication, usability and effectiveness from Hadoop and related technologies.