Deploying analytics microservices in the cloud

Big Data Evangelist, IBM

Service-oriented architecture (SOA) may sometimes feel like an outmoded IT industry paradigm from the first decade of the 2000s, but it could not be more relevant to the cloud computing arena of the mid-2010s and beyond. SOA, under that term, has been out of vogue for almost a decade now, as you can see on its Wikipedia page references from the era of the most recent president named Bush. Many of the discussions since then have been repeating the same terminology without advancing the discussion much.

Nevertheless, the interoperability pressures that spawned the SOA mania haven’t gone away. From the start, the point of SOA was to help users reduce their lock in to specific vendor platforms that offered core application and middleware functions on inflexible stacks. It’s all about maximizing users’ ability to share and reuse distributed services over a loosely coupled, multiprotocol, interoperability fabric made up of dumb pipes. Not just that, but SOA is also a paradigm that enables all these services on whatever platform to be accessed through fine-grained standard interfaces and to evolve independently while maintaining stable, service invocation interfaces.

The new SOA

None of of those interoperability points is less relevant to IT professionals today than it was 10 years ago. From what I can see, SOA has evolved into microservices architecture, a term I’ve been hearing with increasing frequency in recent months in industry circles. But the SOA connection didn’t spring to mind until I saw the recent article “Microservices” by James Lewis and Martin Fowler. The authors define microservices as an “architectural style…[for] developing a single application as a suite of small services, each running in its own process and communicating with lightweight mechanisms, often an HTTP resource API.” As they describe it, a microservices architecture rides on such practices as service-oriented componentization, capability-aligned service factoring and service deployment and management automation.

Lewis and Fowler’s point of view is thoroughly consistent with SOA as it’s been popularly understood for many years. The flexible deployment capabilities implied in that microservices definition go to the heart of SOA, which has long been practiced in the most complex cloud services environments. Enabling distributed microservices to function as a distributed application may require a complex middleware fabric of reliable messaging, transactional rollback and long-running orchestration capabilities. These shared services need to span diverse microservices interaction patterns, such as hierarchical, parent-child, peer-to-peer or various blends thereof.

However, something new in the microservices architecture exists that wasn’t prominent in SOA discussions of yesteryear. Specifically, Lewis and Fowler call out a growing trend—edge-oriented deployment of application intelligence and decentralized management of data governance—that wasn’t part of industry discussions in SOA’s heyday. At the very least, they certainly weren’t in the industry mainstream when I blogged about SOA in September 2005.

What’s new this time around is that, in the intervening years, big data analytics and the Internet of Things moved to the forefront of everybody’s IT strategy. To the extent that any of those technologies was on the SOA community’s collective radar in the mid-2000s, it was in big data analytics predecessors such as very large databases (VLDBs) and such precursors of the Internet of Things as machine-to-machine computing.

The microservices ahead you look at the distributed computing cloudscape of 2016 and beyond, clearly edge-oriented, big data analytics microservices are fast on their way to ubiquity. These microservices are taking the form of analytics—especially machine learning, deep learning and natural-language processing—that leverage the rich pallet of sensors on every Internet of Things endpoint. These embedded analytics microservices will be ubiquitous in every smart device, material or other artifact that comes to market. And data scientists will be building the embedded analytics microservices that enable all these Internet of Things–enabled artifacts to handle most of this processing locally and more rapidly and flexibly than any cloud service.

Actually, the micro part is a bit superfluous in this new context of the Internet of Things and big data analytics. The edge-oriented analytics services may not be any more micro—in other words, granular in functional scope—than the properly factored services called for in established SOA practices. Nevertheless, the endpoints themselves where these services execute—smart sensors, smartphones and so on—will indeed be more micro—or rather, nano—in their resource constraints than the server clusters assumed in classic SOA.

The view through the fog

Another way of looking at microservices architecture is simply that it’s SOA in the new era of Internet of Things–centric, big data analytics–infused fog computing. Several resources provide some informative perspectives: