Using IBM Watson Foundations to read emotions

President, Clabby Analytics, Clabby Analytics

This week I met with Tim Llewellynn, CEO and cofounder of nViso, a world leader in facial imaging technology, to discuss how the company uses Watson Foundations (IBM’s Big Data & Analytics platform, explained later). I learned that nViso makes facial analytics tools that can help consumer-focused businesses better understand their customers by analyzing human non-verbal signals. Further, the company uses Watson Foundations to provide the analytics and data management infrastructure that support facial image analysis.

A closer look at nViso shows that its core expertise is facial imaging analysis. To conduct this analysis, the company:

  • Digitally captures data streams on cloud and mobile platforms
  • Uses facial imaging software to analyze that captured digital data to look for human non-verbal signals (emotional responses)
  • Enables customers to use the data that it collects to start asking new and different questions about customer preferences, and can become more attuned to consumer reactions to marketing campaigns
  • Automates the process of analyzing customer emotional reactions, making it easier and less costly to gather consumer information. 


As for Watson Foundations, a closer look shows that this platform is a combination of “use cases” (types of queries that need to be answered), and the supporting technologies needed to answer these queries. The key concept to understand here is that different types of queries need the data they analyze organized in different ways; a variety of data management tools, as well as analytics software, is used to analyze this data. In IBM’s parlance, these data, data management, analytics and -systems environments are called “zones.” For more on how use cases and zones work together in Watson Foundations, see my recent blog on the Big Data & Analtyics Hub entitled “An Analyst’s Examination of Watson Foundations.” In this post I described how Watson Foundations helps customers frame their queries around “use cases,” and how foundational back-end data “zone” technologies consisting of systems and related database and data management software help customers answer different types of queries. 

Now apply this Watson Foundations concept to what nViso does: building facial recognition software. This software analyzes digital emotional reactions to products and situations, and once this data has been captured, many questions can be poised, such as, What is happening to make these customers react this way? Why are customers happy or unhappy with a particular product or proposed action? or What is likely to happen if we change a product to reflect this feedback? The answers to these questions may require the data to be structured differently for analysis, and this is where the “zone” concept comes in. Data can be put in:

  1. An information ingestion or operational information zone made up of various technologies that can best serve this type of analysis
  2. A real-time analysis zone that identifies and acts on results as they are happening
  3. An exploration, landing or archive zone
  4. A data warehouses or data mart zone supported by various analytics appliances

nViso, for its part, needs only to focus on its specialty--facial imaging--and all of the back-end “zone stuff” is taken care of by IBM using its Watson Foundations use cases and zones approach.

A closer look at nViso: The growing use cases for facial imaging technology

nViso facial imaging technology can be used in a variety of ways, but one of the most interesting examples that Llewellynn described has to do with the reinvention of manual marketing focus groups with automated facial imaging tools that can help collect streamed data and place it into a format that can be analyzed. By automating and digitizing the manual marketing focus group process, enterprises that want to test consumer reactions to its products can more easily conduct consumer research through surveys with greater frequency at significantly lower cost. Further, it can be argued that the data collected and analyzed by systems may produce more “provable” results (it is less subject to human interpretation and associated biases). 

To understand how facial imaging can help to reinvent focus groups, consider how marketing organizations have gathered customer information in the past. During focus group meetings customers are asked for their perceptions of a given product, their reactions to marketing campaigns and so on in an attempt to gather statistically relevant feedback regarding consumer and buyer preferences. Focus group mediators gather feedback, human analysis is then performed on that feedback and important decisions are made regarding when and how to go to market (or even not go to market). But, as Llewellynn points out, there is another dimension that can be weighed in this kind of focus group scenario: the non-verbal emotional reaction of customers to product designs, tastes, spokespeople and marketing campaigns. Prospective customers reveal much about their emotional reactions to a product through non-verbal communications, and nViso has the tools to capture that information digitally, analyze it and help customers make more informed decisions.

Llewellynn points out that there are other ways to use his facial imaging technologies including using in-store monitors to capture customer in-the-moment reactions to products and product displays that are difficult for people to recall. The data collected through this monitoring would, of course, be anonymized, or used in interactive kiosks where self-administered interviews can be conducted as customers leave stores, to find out what their shopping experience was like. Such concepts have already been demonstrated in retail banking environments with the Bank of New Zealand (BNZ).

Summary observations

One point I did not mention in this blog is how nViso brings its products to market. The company offers on-premise software, as well as makes its software and back-end linkage to Watson Foundations available as a cloud service (using IBM’s SmartCloud/SoftLayer environment). I especially liked the cloud offering; it makes it simple for companies that don’t want to worry about building, governing and managing the big data and analytics software to deploy nViso solutions.

I also asked Llewellynn why he chose IBM as a business partner (after all, surely some of the other database companies could have offered similar services). What Llewellynn told me was that the reason nViso chose this partnership is because of IBM’s clear vision about the future of data and analytics, the company’s solid investment in analytics and IBM’s strong record for execution (regularly meeting deadlines and expectation). Llewellynn indicated that he did not find this same same set of characteristics when evaluating other vendors. This mimics my findings in the analytics market--I see IBM investing more heavily than any other vendor in this market space and this investment is driving innovative new ways to conduct analytics. I’d be interested if readers agree with this point. 

During the course of my discussion with Llewellynn, I found myself wondering if nViso technology could be used to conduct widespread, web-based customer analysis. Imagine if a prospective buyer was willing to sit down in front of a webcam and provide product feedback. Think about thousands of customers reacting to products before they are brought to market--what marketing organizations would be willing to pay for this kind of data! 

My key take-away from this interview was this: facial imaging could become really important as a marketing tool over the next several years, and with Watson Foundations serving as a big data and analytics software platform available as a cloud service, building a data analysis environment and conducting this kind of analysis has been greatly simplified.

Read more about IBM Watson Foundations in my post recapping an enlightening interview with John Hagerty, IBM program director for Big Data & Analytics.