System z Rocks (Again)

The mainframe isn’t dead—in fact, it’s turning heads as a data warehouse platform

When IBM announced IBM® DB2® database in 1983, the product was mainframe-based and positioned as a new technology foundation for decision-support applications—the term data warehouse had not been coined yet. Organizations jumped on DB2, but often used it to run online transaction processing (OLTP) workloads. In response, IBM added lots of performance-enhancing features to DB2, and OLTP became the primary workload running on DB2 systems all over the world. As time went by, the predominance of OLTP fed a perception that mainframes were not a great fit for business intelligence (BI) applications. Now, however, IBM System z® mainframe computers and DB2 are a popular combination for BI. How is it that DB2 and the mainframe—the combination born for BI—are once more seen as a primo platform for data warehousing?

The mainframe market: Very healthy, thank you

The “mainframes are dead” pronouncements reached a crescendo in the late 1990s, and some folks continue to stick to that line. What a load of bunk. The market for IBM mainframe servers remains very robust. In fact, the System z server line is thriving because it solves major challenges that almost every company is facing, starting with the drive toward “greener” computing. System z servers are highly space- and energy-efficient, delivering a lot of computing power per unit of floor space occupied and unit of energy consumed. Another mainframe-boosting trend is the desire for continuous availability and consistently good performance of online applications, underscored by users’ trust that sensitive data will be well protected. System z has long been the benchmark computing platform when it comes to availability, scalability, workload management, and security (advantages that are even more pronounced when several IBM z/OS systems work together in a shared-data, Parallel Sysplex cluster). Just because mainframes are hot technology doesn’t mean that z-based data warehousing should be on the rise—but that’s what’s happening. Why? For one thing, there is an increasing interest in getting more BI work done closer to the warehouse source data, and a great deal of that source data is on mainframe systems. Getting the data warehouse close to the data supply facilitates frequent, near-real-time updating of warehouse data, a huge plus for many organizations. Further, many companies now see their data warehouse systems to be just as mission critical as their run-the-business OLTP applications. Tolerance for unscheduled outages is low, especially when failures trigger financial penalties built into service-level agreements. That often leads organizations to build their warehouse on a System z foundation.

DB2 for z/OS steps up

While DB2 debuted on the mainframe platform as a DBMS designed for decision support, DB2 for Linux, UNIX, and Windows (LUW) delivered BI-friendly features that were lacking in DB2 for z/OS. This gave some folks the impression that IBM was pushing distributed system servers as the DB2 platform of choice for data warehouse applications. In recent years, DB2 for z/OS got its BI groove back, thanks to enhancements in DB2 8 and 9, including:

  • 64-bit addressing (DB2 8): Data warehouse applications are often I/O-intensive, and it helps to be able to allocate really big buffer pools.
  • Materialized query tables (DB2 8): By providing prebuilt intermediate result sets that would otherwise have to be materialized at query execution time (often related to aggregation functions or table join operations), MQTs can dramatically reduce query run times.
  • In-memory work files (DB2 8, extended in DB2 9): When an intermediate result set is materialized at query execution time and is subsequently re-accessed by DB2, that re-access is accelerated through the use of in-memory work files.
  • Indexes on expressions (DB2 9): BI queries are often complex and regularly involve predicates that contain column expressions. The ability to create indexes on column expressions can deliver orders of magnitude performance improvements.
  • Index compression (DB2 9): In data warehouse environments, the disk space used for indexes can exceed the space used for tables. Index compression can help reduce index disk space consumption significantly.
  • Richer SQL: Includes common table expressions and recursive SQL (DB2 9); INTERSECT and EXCEPT for result set comparison (DB2 9); online analytical processing (OLAP) functions such as RANK, DENSE_RANK, and ROW_NUMBER (DB2 9); TRUNCATE for quickly emptying data from a table (DB2 9).

Add some financial incentives

A BI application can be a particularly cost-effective mainframe workload, thanks to a couple of financial incentives provided by IBM:

  • zIIP engines: System z Integrated Information Processors (zIIPs) are specialized mainframe CPUs that help lower the cost of computing: they cost less than general purpose processors, and they do not factor into mainframe software pricing. DB2 query parallelism—a big performance booster for data warehouse queries—is a zIIP-eligible system activity. zIIPs can also handle some of the work associated with queries that get to DB2 by way of the Distributed Data Facility, using the Distributed Relational Database Architecture (DRDA) protocol (often through IBM DB2 Connect)—something that’s very common in DB2-based data warehouse environments.
  • DB2 for z/OS Value Unit Edition pricing: For certain types of application workloads (and data warehousing is one of the eligible types), DB2 for z/OS can be acquired for a one-time charge.

A green, super-scalable, super-available, and super-secure server platform. Advanced BI technology. Budget-friendly financial incentives. DB2 and System z offer a compelling solution for organizations seeking a rock-solid foundation on which to build mission-critical data warehouses. How to get from A to B, then? That’s where IBM InfoSphere® software can help.

Mainframes still in the mainstream

Far from being extinct, mainframe servers maintain a solid—and growing—enterprise presence.

  • During the fourth quarter of 2008, IBM led the market for high-end servers (those costing more than US$250,000), with 63.5 percent of factory revenue share.(1) System z’s share of this mainframe market has nearly doubled since 2000.(2)
  • This growth is not just a matter of long-established mainframe applications getting bigger. Installed capacity of new workloads on IBM mainframes grew significantly in the first half of 2008 versus the same period in 2007.(3)
  • System z growth extends beyond traditional large-scale-computing markets. IBM mainframe revenue in emerging markets such as Brazil, Singapore, Thailand, and the Philippines grew 21 percent in the first six months of 2008 compared to the first half of 2007, according to IBM.(3)

InfoSphere: Design, populate, accelerate

A data warehouse does your organization good when it is populated and providing actionable information to users. IBM InfoSphere Warehouse on System z provides an integrated toolset that helps you get there faster, particularly when the warehouse source data is managed by DB2 for z/OS. The toolset includes a Design Studio that helps to model data for OLAP access (including physical database design and data movement flows); the SQL Warehousing Tool (SQW), which delivers a SQL-based data movement and transformation capability; Cubing Services for optimizing multidimensional reporting and analysis, including a caching capability that can greatly improve performance for queries expressed in the industry-standard Multidimensional Expressions (MDX) query language and that supports popular end-user tools such as IBM Cognos 8 Business Intelligence and Microsoft Excel; and an administration console to manage the runtime environment. So, get with the in crowd. The mainframe platform that runs your mission-critical OLTP applications is also a great choice for data warehousing. After all, BI is in the DNA of DB2 for z/OS.