Workload-optimized Systems? Patterns of Expertise for Built-in Solution Best Practices

Big Data Evangelist, IBM

Expertise is precious and hard-won. Perhaps you’ve cultivated it in-house through long-term investment in staff skills and training, in which case you risk having it walk all out the door at any time. Perhaps you’ve recruited experts externally, and paid a premium for their specialties. Perhaps you have independent consultants on retainer who know your environment as well as or better than any of your staff.

People come and go, but patterns of expertise are a resource that should endure as a stable business asset. Ideally, your organization should have expert patterns integrated into all analytic, database, application, middleware, computing, storage, networking and other business infrastructure platforms.

What is an integrated expertise pattern? It represents an encapsulation of best practices drawn from the time-proven practical know-how of myriad IT deployments. It is the pre-defined, preconfigured, pre-optimized solution architecture, enabling repeatable deployment with full lifecycle management, monitoring, security and so forth. For the workloads you need to process, the pattern should:

  • Give your business infrastructure platform the flexibility of a general-purpose system with the elasticity of cloud services and the simplicity of workload-optimized hardware/software appliances;
  • Encapsulate application components, virtual images, applications, databases, queues, resource connectors, business process models, batch jobs, mediations schemas, policies and rules (e.g., high availability, SLAs, security, multi-tenancy, isolation), and other artifacts essential to the system life cycle;
  • Automate many functions while enhancing the productivity of IT professionals in the deployment, provisioning, configuration, integration, tuning, management and upgrading of the underlying infrastructure, including all physical and virtual components; and
  • Provide a shareable resource that your IT teams can leverage and extend as a deployment tool in single- and multi-node environments.

If you’ve deployed a sophisticated data warehouse (DW) appliance such as IBM Netezza, you are probably familiar with value of integrated expertise patterns. You expect that the DW appliance comes preconfigured with an architecture, features and tooling that have been field-proven. A DW appliance proves its value by embodying established best practices to get you up and running and to automate tedious chores—such as storage optimization—thereby enabling your database administrators, data scientists, business intelligence developers and knowledge workers to be as productive as possible from day one.

Within a DW appliance, the management tooling should embed integrated expertise patterns. These patterns—think of them both as productivity tools and solution accelerators—should help you see the value of the integrated system, both in a proof-of-concept context and over the useful life of the unit. At the very least, integrated expertise patterns should facilitate delivery of quick value in the appliance’s core use cases— reporting, query, dashboarding, in-database analytics, etc.—and in support of all requisite data integration, governance and other functions.

The appliance should come with integrated expertise patterns specific to key DW, big data and advanced analytic applications. The chief artifact that the patterns leverage should include at least, prepackaged source connectors; ETL and data quality scripts; data schemas and glossaries; hierarchies and catalogs; predictive models; key performance indicators; report and dashboard designs; calculations; rules; and policies.