The outthink imperative: The many paths to cognitive illumination

Big Data Evangelist, IBM

We can’t outthink our problems in the 21st century by using the same cognitive approaches that fell short in the past. Outthinking our way to a better future requires that we leverage data-driven guidance from fresh sources. The world’s issues are too large for any one person to have all the answers. Likewise, the challenges confronting humanity are too complex, dynamic and multifaceted for any one algorithm, statistical model or machine-learning program to address definitively.

With advances in cognitive computing and machine learning, people are starting to realize that within specific scenarios computers can provide smart guidance in myriad circumstances. Cognitive systems can algorithmically learn from fresh data and dynamic circumstances. They can generate unprecedented outputs that may not have previously occurred to any human expert. In the process, these systems can automatically guide users to the right data-driven decisions in every situation.

The unifying theme behind today’s announcements at IBM Insight 2015, and others of very recent vintage, is that there are many paths to cognitive illumination and many involve a creative blend of human and algorithmic guidance. In this new era, you can use self-service, cloud-centric tools to easily analyze huge amounts of data and a deep pool of cognitive smarts without needing to be a domain expert or data scientist. And you can leverage the full range of analytics—cognitive, descriptive, predictive and prescriptive—that are suited to a particular decision scenario. In this regard, several key recent announcements were made: 

  • Guided exploration: Announced more than two weeks ago and currently in beta, IBM Watson Analytics Expert Storybooks offer guidance through a natural-language interface and prebuilt cognitive-computing templates to deliver the domain expertise of subject-matter experts (SMEs). Expert storybooks provide starting points to guide users through the best practices of understanding data, digging deeply into it and communicating insights. They automate the steps of data access and refinement, predictive analysis and visual storytelling. They use advanced analytics to find hidden items in data and to spark new questions. The storybooks enable ad hoc exploration of domain-specific data to uncover timely, functionally relevant insights. And they facilitate self-service guided discovery of the most relevant data and analytics on the Watson cloud-based platform. 
  • Guided discovery: Announced today, IBM Cognos Analytics enables teams of knowledge workers to leverage traditional business analytics capabilities—query, reporting and dashboards—and the visual data discovery and accelerated statistical modeling tools generally associated with data scientists. Just as important, Cognos Analytics provides guidance to the new breed of power users and citizen data scientists through a new feature—intent-driven modeling. The solution unlocks insights efficiently through a guided experience that interprets the user’s keyword-evident intent in the development of models, dashboards and reports. The tool responds with model proposals based on that intent and on the available data. All the while, it takes the user to a visual representation of the model that it’s assembling for them, enabling them to rapidly test and modify it. In this way, Cognos Analytics helps users to ask the right questions and can recommend the best way to present the insights or build a report or model. 
  • Guided modeling – IBM Analytics for Apache Spark: Announced in June as an open beta but formally launched today, IBM Analytics for Apache Spark as a Service is a subscription-based, IBM Bluemix cluster-computing service that delivers the power of Apache Spark in the cloud. It provides an interactive modeling environment that guides data scientists and analytics in the creation of high-performance, machine-learning models. Developers can iteratively build, iterate and execute in-memory models in a unified environment that leverages Spark SQL, Spark Streaming and Python, while an object storage service helps simplify management of deep data sets that may include various combinations of unstructured data and rich media. 
  • Guided curation: Announced today, IBM Insight Cloud Services provide repeatable Bluemix industry solutions that leverage curated data sets from IBM Business Partners, The Weather Company (TWC) and Twitter. Industry data packages in insurance, government, energy and utilities, healthcare and other sectors support cloud-based explorations by data scientists and analysts. When explored through Watson and Analytics for Apache Spark, these curated data sets can guide organizations more rapidly to insights that they can operationalize inside business applications. all these ways, outthinking the best experts of yesteryear is becoming easier for everyone by using cognitive analytics well suited for and available today on the cloud. Are you hungry for your own guided illumination? Several links provide on-ramps for each of the new solutions discussed here:

In addition, deepen data science explorations in business intelligence, Watson Analytics, the open and unified analytics platform, Apache Hadoop and Spark. You many also find these additional IBM resources on Spark helpful: 

And if you’re interested in engaging with data scientists exploring the use of Spark, machine learning and cognitive computing, connect live and direct at Datapalooza, 10–12 November 2015, in San Francisco, California.

Start here for a deep dive on Insight Cloud Services.