Transforming the agile data warehouse in the age of the in-memory cloud

Big Data Evangelist, IBM
Transforming agile DW blog figure 1.jpg
Image courtesy of Wikimedia Commons and used with permission.

IBM Analytics Warehouse for Bluemix is now available

Agility is a business vision. Your data platform should be agile so that you can continue to adapt to an ever-changing competitive climate.

Agile infrastructures can serve as the common foundation for all of your data requirements. Agility is all about speed of response and flexibility to turn on the proverbial dime in any new direction that your business requires, like an Olympic-class athlete. 

What’s most important is that your data platforms have the agility to operate cost-effectively at any of the following:

  • Any scale of business: You should be able to acquire a low-volume data platform and modularly scale it out in storage, processing, memory and I/O capacity as your needs change. Your platform should elastically scale up and down as requirements oscillate. Your end-to-end infrastructure should also be able to incorporate platforms of diverse scales with those platforms specialized to particular functions and all of them interoperating in a common fabric.
  • Any speed of business: You should be able to acquire a low-velocity data platform and modularly accelerate it through incorporation of faster software, faster processors, faster disks, faster cache and more DRAM as your need for speed grows. You should be able to integrate your data platform with a stream computing platform for true real-time ingest, processing and delivery. And your platform should also support concurrent processing of diverse latencies, from batch to streaming, within a common fabric.
  • Any scope of business: You should be able to acquire a "low-variety" data platform (perhaps a RDBMS dedicated to marketing) and be able to evolve it as needs emerge into a multifunctional system of record supporting all business functions. Your data platform should have the agility to enable speedy inclusion of a growing variety of data types from diverse sources. It should have the flexibility to handle structured and unstructured data, as well as events, images, video, audio and streaming media with equal agility. It should be able to process the full range of data management, analytics and content management workloads. It should serve the full scope of users, devices and downstream applications.

To determine whether your organization has an agile data strategy, ask yourself the following questions:

  • Does our data strategy incorporate conformed independent data marts, centralized multi-domain architectures, hub-and-spoke environments, federated cloud or some innovative hybrid of those approaches?
  • Does our data strategy involve building flexible hooks into the infrastructure so that it can be easily extended to support in-database analytics, stream computing, unstructured content and other emerging requirements?
  • Does our data strategy focus on providing a “single version of the truth” through a comprehensive data warehousing (DW), data quality, data governance and master data management capability?

IBM has been a leader in agile data platforms for many years, as evidenced by the breadth and depth of our solution portfolio that embodies this vision. For example, we continue to integrate new agile technologies, such as in-memory and cloud computing, into our entire data platform portfolio. Over a year ago, we announced several key new solutions that keep us at the forefront of this new agile-data order. The incorporation of BLU Acceleration technology (which is dynamic, in-memory and columnar) into IBM DB2 10.5 and IBM Informix 12.1 demonstrated that we've evolved our online transaction processing (OLTP) systems to enable speed-of-thought velocities without sacrificing their batch-oriented cores.

And IBM has long been a leader in agile DW. That's because you can't respond effectively at any scale, speed or scope unless you have a "single version of the truth" repository of official business data (for example: finance, HR and customer records) to anchor it all. Our latest generation DW platform, IBM PureData System for Analytics, lets you flexibly grow your DW infrastructure up and out as your needs evolve. PureData's expert integrated system architecture incorporates such agile concepts as fit-to-purpose design, flexible licensing and deployment and self-service "build, load and go."

So it's no surprise that IBM has made agile DW the heart of our cloud portfolio as well. We've architected PureData and all of our other PureSystems platforms with "cloud-first" design principles. This will facilitate the day when our customers decide to evolve their investments in IBM DW and other solutions into cloud deployments that are simple, scalable, high-performance, flexible, on-demand, always-on and ubiquitously accessible.

IBM Bluemix with Analytics Warehouse.png

We made a significant announcement on this direction in November 2013 when we launched the technology preview of IBM BLU on IBM's public cloud platform, supporting agile DW and analytics for everyone. This cloud DW service leverages our breakthrough in-memory technology, IBM DB2 with BLU Acceleration, and works with our proven Cognos BI solution. The open beta was launched in late February at IBM Pulse and, in addition to running in-memory analytics on the IBM Softlayer public cloud, beta users have also had the option of allowing IBM run it for them through our Bluemix cloud service marketplace.

Now we're announcing that the service, now known as IBM Analytics Warehouse, is generally available to all customers for your agile DW and analytics needs. IBM Analytics Warehouse is designed to provide a single agile platform for all applications required for most DW, BI and analytics projects. This pay-as-you-go public-cloud service combines high-performance with ease-of-use and cost-efficiency to enable 8 to 25 times faster reporting and analytics and more than 1,000 times faster queries in specific use cases. It achieves these results with the core capabilities of dynamic in-memory processing, actionable compression, parallel vector processing and data skipping that were heretofore only available to users of premises-based DB2 BLU and Informix deployments.

As such, IBM Analytics Warehouse is specifically designed with more users in mind than just IT departments, DBAs and developers. In fact, it is geared for the broader market of line-of-business users, small to midmarket businesses and others who need an agile, cloud-based DW/analytics platform. And within IT organizations, Analytics Warehouse is well-suited for educational uses, prototypes, development, testing and pilots.

The key agile data use cases for IBM Analytics Warehouse are:

  • Agile analytic marts: Analytics Warehouse allows organizations to deploy powerful analytic data warehousing in the cloud in less than an hour.
  • Agile development and test environments for DB2 Shops: Analytics Warehouse is suited for development and test environments that augment existing production environments.
  • Agile big data pilots and proofs of concept: Analytics Warehouse allows users to explore big data analytics easily, at no risk and with no infrastructure investment required.

Here's what comes with IBM Analytics Warehouse:

  • High performance columnar, in-memory-optimized database powered by IBM DB2 with BLU Acceleration
  • Enterprise grade security with zero touch data encryption and unified security management
  • Integrated with R, a comprehensive statistics solution designed for data scientists
  • Web-based administration with training videos and user support built right into the console
  • Full set of drivers for easy connection to third party applications
  • Mobile access enablement
  • Accessibility using third-party tools that you may already know and work with using freely available drivers
  • Easy integration with corporate and enterprise deployments and with other big-data platforms
  • Support for industry-standard SQL access

IBM Analytics Warehouse is part of a larger agile ecosystem.

For starters, IBM Analytics Warehouse is one of over 50 new Bluemix services, covering a comprehensive range of enterprise infrastructure functionality, that have recently become generally available.

On top of that, IBM Analytics Warehouse is a key pillar in the full range of new data analytics cloud services now available under Bluemix. The full list, with access links, as as follows:

Furthermore, IBM Analytics Warehouse is the lowest cost entry point for DW in IBM’s industry leading big data platform, Watson Foundations. This comprehensive information management platform provides analytics on any type of data (from structured, semi-structured and unstructured to data at rest and streaming in real time) along with a wide array of complementary ecosystem tools. Watson Foundations also serves as the core infrastructure for Watson, IBM’s unique cognitive-based system that allows users to drive analytics through natural language.

In all of these ways, IBM Analytics Warehouse can transform your DW into an agile infrastructure. It can propel your data analytic infrastructure into the 21st century, in which all business will operate at thought-speed by making the most of in-memory computing in the cloud.

For further details and to get started with IBM Analytics Warehouse, please visit this page.

For further details on the entire Bluemix service catalog, please visit this page.

For IBM's news release on the Bluemix service launch and the momentum since the beta period, please visit this page.