Information empowerment for your Hadoop ecosystem

Portfolio Marketing Manager, Information Integration & Governance, IBM

If you’re using Hadoop or a Hadoop-based solution, you already know these platforms are excellent at storing and processing large data volumes. But organizations trying to leverage Hadoop for data integration find that they’re quickly outgrowing this solution for the following reasons:

  • Exploding amounts of data being sent to the data lake
  • Expanding communities of data providers and consumers
  • Higher levels of regulatory requirements needed to understand and protect certain classes of data

Not only is Hadoop ill-equipped to respond to these increasing demands, but it was never intended to carry out the essential functions of data integration such as governance support, metadata management or flexible data delivery styles. In other words, continuing to rely on Hadoop for data integration is like building a house out of straw. The basic shape may look fine, but it’s just a matter of time until the whole thing collapses.

How can you empower your IT professionals to cleanse, monitor, transform and deliver reliable and trustworthy data? Plus, how can you enable these functions while ensuring greater scalability and reliability, increased productivity across all departments, faster responsiveness and decision-making, and superior employee and customer trust? address these customer requirements, IBM recently released BigInsights BigIntegrate and BigInsights BigQuality V11.5. These tools, which build on IBM InfoSphere Information Server and run within the Hadoop platform, offer the following capabilities:

  • Enhance your organization’s data quality through metadata management, data profiling capabilities and massively scalable, shared-nothing, in-memory data integration
  • Build Hadoop-based applications without having to be retrained on newly emerging languages that require manual hand coding
  • Define policies and rules that pinpoint the data that’s of greatest interest to your organization
  • Quickly ensure information is being used correctly, eliminating the need to carry out this process manually
  • Consistently work with trustworthy information that’s governed in real time
  • Identify and unlock the value of your data sources more rapidly

If you want to learn more about this topic, don't miss the Strata+Hadoop World conference in New York. As part of the conference, on Thursday, October 1 at 11:20 AM in room 3D 06/07, we will present “Optimize Your Data Integration to Launch New Financial Products with Confidence.” This session will show how IBM’s new solution is being leveraged by financial services clients today. I hope to see you at the conference.