Workload-optimized Systems? Clean-slate Design for Optimal Appliance Performance

Big Data Evangelist, IBM

An appliance is a fit-for-purpose, repeatable node within your broader big-data architecture. Many appliances will be optimized to support various mixes of big-data workloads, while others will be entirely specialized to a particular function that they perform with lightning speed and elastic scalability.

If it is designed properly and supports all the requisite interfaces, a big-data appliance can be a specialized “drop-in” to your existing infrastructure. The appliance might be deployed to support specific roles, loads and applications–such as staging, pre-processing, sandboxing and query offload–while allowing you to extend your investment in legacy platforms, tools and applications. You might also mix and match diverse appliances from various vendors, or various generations of appliance from a single vendor such as IBM, within a common big-data or cloud architecture.

When deployed into a specialized “drop-in” role, the appliance can incorporate “clean-slate design” principles. What this means is simply that vendors can optimize and innovate in the internal design of each new appliance, improving performance, scalability, resiliency and so forth without being constrained by the artifacts of legacy platforms. Engineers can adopt clean-slate design practices only if as they maintain external interfaces to ensure seamless cross-generation interoperability. Where big-data appliances are concerned, these external interfaces include, at a minimum, common denominators such as SQL and MapReduce.

A perfect example of clean-slate design is the IBM Netezza appliance product family. We continue to innovate from one product generation to the next, while maintaining a strict adherence to standards so that customers never need worry about whether the appliances work with their investments in business intelligence, extract transform load, and other key investments. Our clean-slate approach has given the freedom to eschew inessential data warehousing artifacts or features that you might find in our competitors’ offerings. For example, the Netezza appliance product family has never incorporated indexes, physical tuning, partitions, cubes, materialized views, results caching or manual compression. And we have never required manual regeneration of failed drives, manual rewrite or relocation of bad sectors, and other manual storage administration functions.

Clean-slate design is simply the recognition that the way we have integrated hardware, software, applications and networking into past appliances should not constrain our practical imagination going forward. When we think about the insides of our boxes, we’re always thinking outside the box.

Related Posts on Workload-optimized Systems

James is blogging all week about topics related to workload-optimized systems. Read his previous posts:

Check back all week for other posts in the series.