Workload-optimized Systems? Scale In, Out and Up for Balanced Big Data Configurations

Big Data Evangelist, IBM

Big data is, fundamentally, a cloud-computing approach to advanced analytics and data management. The images that come to mind when somebody says “cloud computing” are a) increasingly sprawling server farms and b) increasingly huge server racks arranged in endless rows within these farms. These “scale-out” and “scale-up” approaches anchor the cloud cosmos; some call them “horizontal” and “vertical” scaling, respectively. Regardless, this is scaling that you can see with the naked eye and even kick with the naked foot.

If you only look at the computing universe on a macro level, you’re likely to overlook scalability’s equally impressive nano-frontier. At the nano-level, scale-in architecture focuses on engineering ever more densely packed interconnections among ever tinier processing, storage, memory and other components. It also involves tooling densely packed integrated systems with elastic provisioning and flexible virtualization features. And it involves integrating networking, storage, resilience and system management capabilities into a single system that is easy to deploy and manage either in stand-alone mode or within your multi-rack server farms.

Scale-in architectures enable you to add workloads and boost performance within existing densely configured nodes, each of which should be an expert integrated system. You can execute dynamic, unexpected workloads with linear performance gains while making most efficient use of existing server capacity. And you can significantly scale your big data storage, application software and compute resources per square foot of precious data-center space.

For enterprises that are serious about scaling their big data at all levels, a balanced strategy should incorporate all three approaches–scale-up, scale-out and scale-in–depending on the workloads you’re trying to optimize.

Related Posts on Workload-optimized Systems

James is blogging all week about topics related to workload-optimized systems. Read yesterday's post, "Workload-optimized Systems? Patterns of Expertise for Built-in Solution Best Practices." Check back all week for other posts in the series.