Insight: Finding the wow-factor in your data
As we kick off each new year, we think big thoughts. We make big plans. We look for that big “wow!” insight that will set us on a great new trajectory. But the insight itself doesn’t typically materialize from nothing in a big bang. It emerges when we connect the dots among bits of information that were previously not connected—or were connected in a different way. The raw data was there all along, but now we see it in a new way. Wow!
Sometimes this connect-the-dots activity is slow and painstaking. Sometimes it happens in a burst of realization or inspiration. Yet, however it happens, it can disrupt business practices, belief systems, industries, even ways of life. As a result, much interest is generated in gaining insights and determining what can be done to accelerate them. If there’s an important insight to be had in your own industry, don’t you want your organization to be the one that benefits?
In education, assumptions about what underprivileged students from a community high school could achieve were disrupted by the performance of a team from Carl Hayden Community High School in Phoenix, Arizona. It defeated other teams from MIT and other respected institutions in an underwater robotics competition designed for college students. Attendees at Insight 2015 learned from Fred Lajvardi—Science, Technology, Engineering and Math (STEM) teacher and coach—how diversity, teamwork and previously untapped potential were leveraged to create insight and innovation. This inspiring success was featured in the documentary Underwater Dreams and in the motion picture Spare Parts.
Has that success really disrupted secondary education? Not yet, to my knowledge. But perhaps it will kick-start an evolution in a direction that benefits students and their communities across the US and around the world.
In business, when we think of market disruption, companies such as Amazon and Uber come to mind. At Insight 2015, Elle Shelley from Local Motors described potential disruption in the automotive industry as the movement of car creation from large factories to micromanufacturing. What will be the market impact of the 3–D printed cars that Local Motors plans for 2017? We won’t know for a couple of years, but perhaps a new era will emerge in which both mass-produced and micro-produced automobiles coexist, just as printed books and ebooks do today.
In addition to insights that lead to major market disruptions, smaller insights are helping organizations operate more efficiently and engage their customers more effectively than ever, as well as identify new opportunities day after day. Some environments do a better job than others of enabling insight and then encouraging related innovation.
If rigid organizations and brittle processes can hamper insight and innovation, what are some of the characteristics of environments that foster insight? Here are a few examples:
- Easy access to all types of information, in any form and wherever it resides, for anyone who needs it because we have no way of knowing which data elements will fuel the next big insight
- A foundation of high-quality information that can be trusted—regardless of where it is stored, when it arrived or where it originated—because well-informed decisions are based on trusted information
- Protection of personally identifiable information and other confidential data from inappropriate access because data scientists and citizen analysts shouldn’t have to worry about whether or not the data they access is authorized for use
- Open sharing of nonconfidential information because communities of interest can spawn creativity and accelerate solutions
- Adaptability to hybrid environments because today’s organizations include both on-premises and cloud-based data and processes, structured and unstructured information, internal organizational data and data from external sources, familiar information and unused dark data, and proprietary and open-source software
- Advanced analytics capabilities, whether provided in tools for specialists or embedded in applications so citizen data scientists without special skills can take advantage of them, because the application of analytics to data can put information into a whole new light
The IBM Analytics Technology platform is designed to help organizations create that insight-nurturing environment, based on technology that is open, hybrid and trusted. What’s open about it? For one thing, it is designed to add value regardless of what type of technology is already in place within the organization. For another, it illustrates the IBM commitment to Apache Spark, the fastest growing community project today.
Transforming a commitment to Spark into concrete contributions, IBM has added more than 60 improvements to the core of Spark and has freely offered SystemML—its machine-learning language—to the community. IBM has already built multiple analytics and commerce solutions on Spark, and now a Spark-as-a-service offering is available on the IBM Bluemix platform.
So where does insight come from? Whether you’re an engineer, a data scientist or a businessperson, watch this one-minute video. Then you’ll be ready to answer the question in terms that a ten year old can understand.