Colin White and I recently wrote a white paper for IBM titled Technology Innovations for Enhanced Database Management and Advanced BI. In it we discussed the fact that IT leaders across all industries need to review and enhance their information architecture to support new requirements, such as big data, advanced business intelligence, cloud computing, and mobile devices. These leaders must assess how they can exploit recent technological innovations in data management and business intelligence (BI) capabilities to satisfy these needs. To make sense of their assessments though requires a deeper understanding in three critical areas—the What, the Why and the How behind these requirements.
These describe the drivers for the technologies that help both business and IT acquire, analyze and understand all sources of information flowing into the enterprise today. The biggest driver for most implementers consists of the new and unusual sources of data they must handle. The next driver comes from the business itself: Information workers want to be able to create their own reports and analyses; they want self-service BI. Next is the demand for performance—for operational or real-time BI and for supporting increasingly complex analyses performed on higher volumes of data. Another driver comes from the constant cry: Reduce IT costs!
New technologies must fit into ever-shrinking IT budgets. You can’t enhance or extend existing BI environments with new technologies if they break the bank. And the final driver must be to enhance IT’s ease of use and improve their flexibility. IT must be able to install, integrate and maintain these new technologies without significant or additional effort or cost.
These consist of the benefits derived from improved Business/IT efficiencies and effectiveness achievable through the utilization of technological advances. Both online transaction processing (OLTP) and business intelligence (BI) systems are undergoing great changes. The OLTP systems that drive operational business processes of our modern enterprises are experiencing massive increases in transaction volumes especially in the area of web traffic. BI systems are also experiencing significant growth. BI has evolved from simple reporting functionality to very sophisticated, complex diagnostic, investigative and optimization capabilities. Like OLTP systems, the volume and types of data being handled by BI systems are growing as organizations become increasingly interested in analyzing new sources of data such as machine sensor, web and social media data. Changes in both OLTP and BI are leading to highly complex and mixed workloads, which are stretching the capabilities of most IT systems. The challenge is to maintain service level requirements in the face of this growth. Organizations therefore need to be thoughtful in their choice of hardware and software solutions if they are to leverage the business benefits offered by the many innovations in both data management and business intelligence.
These are the technological advances, derived from the What and Why factors, that enable the modern enterprise possible today. Organizations have spent over 50 years automating their business processes and during that time, there have been some key technology advances that have had a major positive impact on IT, which in turn has led to significant benefits to the business. Four of these innovations are directly relevant to our paper:
- Online transaction processing systems in the early 1960s
- Relational database management systems in the early 1980s
- Data warehousing and business intelligence in the early 1990s
- Big data, the latest technology innovation to bring significant business benefits to organizations
You will find more detail in the paper itself on these innovations as well as information on the two main categories of technology innovation—advanced BI and enhanced data management.
Regardless of whether new technologies are used to improve existing systems or to build new ones, it is clear that a one size fits all approach to technology selection is not viable given the complexity of today’s business needs and growing transaction and data volumes. Instead organizations will need to deploy technologies based on a range of different, and often mutually exclusive, business and IT requirements. A flexible and integrated information architecture like the one shown in our paper is therefore required.
IT also has a number of different deployment options. Application systems can be deployed on-premises or in a cloud-based operating environment. On-premises systems can be optimized by IT, or purchased as pre-optimized hardware and software appliances. The choice therefore is between:
- The flexibility offered by software solutions that can be optimized by IT for either OLTP or BI processing.
- The convenience of appliances, pre-optimized for specific OLTP and BI workloads.
- The agility and fast deployment provided by a cloud-based computing approach.
There is no single solution that meets all needs, and IT is likely to use all three options based on business and IT requirements. The vendors that provide all three options with solid integration and interoperability capabilities between them are likely to be the winners, given the increasing complexity of today’s IT infrastructure.
Big data coupled with advanced BI and the significant improvements to RDBMS technology dramatically improve IT’s ability to deploy both OLTP and BI solutions that enhance and extend existing systems. Careful selection and deployment of these technologies can bring significant benefits to the business while preserving the value found in the existing components. Organizations should therefore review and improve their information architectures so that they are in a position to take advantage of this new generation of business process and BI innovations.
As with any IT projects, the starting point is determining needs of the business, and then evaluating how any given technology satisfies those needs. Balancing business benefits against the IT costs of achieving those benefits is also a key consideration.
Listen to a Boulder BI Brain Trust podcast featuring Claudia Imhoff covering IBM big data product annnouncements