Recommendations Galore: Part 1

Explore the analytics-powered process engines of the new economy

Big Data Evangelist, IBM

Recommendation engines are the largely unseen process platforms that power the new economy. What they do is automatically generate tailored and context-sensitive recommendations to guide decisions and actions to be taken by humans, automated systems, or various combinations thereof. Embedded in operational applications, these analytic-driven platforms enable continuous optimization of marketing, sales, customer service, and other business processes in keeping with predictive models, business rules, cognitive-computing algorithms, and other domain-specific metadata and logic.

Recommendation engines—also known by such names as next-best-action platforms and decision automation engines—came to mind as I walked the expo floor at IBM Information On Demand (IOD) 2013. Of course, among them was the IBM® Enterprise Marketing Management (EMM) Product Recommendation Solution offering that incorporates its IBM Coremetrics® portfolio.

In the broader perspective, I saw recommendation engines everywhere and nowhere in particular. Viewing the vast range of IBM and partner technologies on display, I saw many demonstrations that focused on e-commerce applications such as offer targeting, advertising optimization, and customer experience management. Recommendation engines of one sort or another power all these applications, and these engines all rely on big data and analytics to tune their actions in keeping with situational context.


Common infrastructure

When demoing a recommendation engine, what you're actually showing is an externally evident business application—such as those mentioned previously—rather than the seemingly invisible guts of the beast. Depending on its capabilities, there are many different functional layers to a recommendation engine. The following principal pieces of infrastructure can be found in many real-world recommendation engines.

Big-data analytics platforms

Platforms for big data analytics may be enterprise data warehouses, customer data marts, or even Apache Hadoop or other big data platforms. Regardless of the underlying data technology, a big data analytics platform’s core jobs are to provide an essential repository for customer data and an execution engine for in-database analytic processing used to generate recommendations. Examples of such data include customer profiles, transaction histories, and accepted and rejected past offers. Examples of in-database analytics include predictive analytics, natural language processing, and model scoring.

Orchestration platforms

Platforms delivering orchestration—such as IBM Business Process Manager—provide the workflow environment that drives the flow of recommendation generation and delivery tasks. For example, the business process platform might execute the flow of interactions between call center, fulfillment, and billing staff and the applications needed for speedy delivery of product orders. Organizations might also combine process orchestration models, predictive upsell models, and deterministic business rules to drive customer-facing, next-best offers. By the same token, they can automate back-end processes such as order fulfillment, manufacturing, distribution, logistics, and delivery from other models and rules relevant to these process domains.

Business-rules platforms

Platforms that define business rules—such as IBM Operational Decision Manager—allow application developers to define the deterministic business logic that calculates the optimal recommendation for each circumstance. These tools support development of the rules that determine the offers an organization makes to its customers under various scenarios. For example, these tools can be used to define complex eligibility rules that govern the types of customers to whom various types of offers may be extended, based on constraints such as age, citizenship, and socioeconomic status. A financial services firm, for example, might have complex rules governing specific offers, such as, “Don’t offer a term life insurance policy if the customer already has one,” or “Don’t offer a credit card if the customer's risk profile is unacceptable.”

Advanced analytics platforms

Platforms for advanced analytics—such as IBM SPSS® Analytic Catalyst suite—enable data scientists to build and refine the statistical models that feed predictive confidence into the recommendations the engine is generating. For example, call center agents might see a ranked list of auto-generated offers to guide their discussions with customers. If the principal requirement is to hold on to the customer, the highest-ranked offer might be identified through an inline churn model that incorporates predictive variables. These variables are mostly correlated with retaining customers in a specific segment, such as unmarried women over age 50 who make six-figure incomes. The predictive churn models evaluate the likelihood of particular customers accepting specific offers. Data scientists might also build graph models that identify the behavioral variables within social communities—such as a positive tweet from a trusted authority—that influence people's decisions to accept one recommendation rather than another.

Stream computing platforms

Platforms that deliver stream computing—such as IBM InfoSphere® Streams analytics for data in motion—enable dynamic updates to the models and rules that drive recommendations and keep them relevant to changing customer situations and requirements. For example, stream computing can ensure that real-time customer click streams from the portal integrate into predictive customer experience optimization models. These models dynamically present the optimal arrangement of offers, options, and menus to customers when they are trying to make up their minds to buy or not to buy a particular product.


The human factor

Recommendation engines are indeed everywhere. However, the fact that they are usually encountered as near-invisible slabs of assembled infrastructure—rather than as discrete hardware platforms or software packages—may make them seem a bit mysterious. By the same token, the complex pool of big data and process logic that powers them—algorithms, predictive models, business rules, orchestrations, metadata, and so on—may make it seem as if they can think for themselves.

That capability is an illusion. What truly powers a recommendation engine are the data governance professionals, data scientists, business-rules developers, orchestration modelers, and subject-domain experts who collectively write and tune their intricate process logic. In a general sense, they hold collective responsibility for the automated decisions and actions taken by the recommendation engines. These smart people leverage the tools of big data and data science to build decision logic that often produces highly unique, personalized, and situation-specific recommendations and guidance. And they write logic that drives interactions on the portal, in the call center, in customers’ smartphone interfaces, and in other channels.

In the conclusion of this two-part column, I’ll address why recommendation engines are often perceived as something slightly mysterious. In the meantime, please share any thoughts or questions in the comments.


IBM Website References


[followbutton username='jameskobielus' count='false' lang='en' theme='light']
[followbutton username='IBMdatamag' count='false' lang='en' theme='light']