Internet of Things: A continuum of change with opportunities galore
Internet of Things technology continues to grow at a fascinating pace, and increasing opportunities to innovate in an Internet of Things world proliferate. The skill and technology layers needed to make the Internet of Things a reality for today’s world starts with machine learning, deep learning, graph databases and real-time data management. And they continue with advanced analytics, sensor data frameworks, infrastructure platforms such as Apache Hadoop, NoSQL and programming languages such as C++, Go, Java and Python.
The most recent wins in this new land of opportunity are the National Institutes of Health (NIH)’s Precision Medicine Initiative (PMI), fraud analytics in healthcare and financial analytics with advanced clustering and classification techniques on mobile infrastructure. More opportunities exist in terms of space exploration, smart cars and trucks, and new forays into energy research. And don’t forget the smart wearable devices and devices for pet monitoring, remote communications, healthcare monitoring, sports training and many other innovations.
5 steps to innovation
How do we achieve success in this journey and become innovative? What are the critical success factors? What are the risks? The processing of Internet of Things data requires a step-by-step approach:
- Acquire data from all sources. These sources include automobiles, devices, machines, mobile devices, networks, sensors, wearable devices and anything that produces data.
- Ingest all the acquired data into a data swamp. The key to the ingestion process is to tag the source of the data. Streaming data that needs to be ingested can be processed as streaming data and can also be saved as files. Ingestion also includes sensor and machine data.
- Discover data and perform initial analysis. This process requires tagging and classifying the data based on its source, attributes, significance and need for analytics and visualization.
- Create a data lake after data discovery is complete. This process involves extracting the data from the swamp and enriching it with metadata, semantic data and taxonomy and adding more quality to it as is feasible. This data is then ready to be used for operational analytics.
- Create data hubs for analytics. This step can enrich the data with master data and other reference data, creating an ecosystem to integrate this data into the database, enterprise data warehouse and analytical systems. The data at this stage is ready for deep analytics and visualization.
The key to note here is that steps 3, 4 and 5 are all helping in creating data lineage, data readiness with enrichment at each stage and a data availability index for usage.
Critical factors for success
While the steps for processing data are similar to what we do in the world of big data, the data here can be big, small, wide, fat or thin, and it can be ingested and qualified for usage. Several critical success factors can result from this journey:
- Data: You need to acquire, ingest, collect, discover, analyze and implement analytics on the data. This data needs to be defined and governed across the process. And you need to be able to handle more volume, velocity, variety, formats, availability and ambiguity problems with data.
- Business goals: The most critical success factor is defining business goals. Without the right goals, the data is neither useful, nor are the analytics and outcomes from the data useful.
- Sponsors: Executive sponsorship is needed for the new age of innovation to be successful. If no sponsorship is available, then the analytical outcomes, the lineage and linking of data, and the associated dashboards are all not happening and will be a pipe dream.
- Subject experts: The people and teams who are experts in the subject matter are needed to be involved in the Internet of Things journey; they are key to the success of the data analytics and using that analysis.
- Sensor data analytics: A new dimension of analytics is sensor data analytics. Sensor data is continuous and always streaming. It can be generated from an Apple iWatch, Samsung smartphone, Apple iPad, a smart wearable device, or a BMW i series, Tesla or hybrid car. How do we monetize from this data? The answer is by implementing the appropriate sensor analytics programs. These programs require a team of subject and analytics experts to come together in a data science team approach for meeting the challenges and providing directions to the outcomes in the Internet of Things world. This move has started in many organizations but lacks direction and needs a chief analytics officer or chief data officer role to make it work in reality.
- Machine intelligence: This success factor refers to an ecosystem of analytics and actions built on system outcomes from machines. These machines work 24/7/365 and can process data in continuum, which requires a series of algorithms, processes, code, analytics, action-driven outcomes and no human interference. Work taking place for more than 25 years in this area has led to outcomes such as IBM Watson; TensorFlow, an open source library for numeric computation; Bayesian networks; hidden Markov model (HMM) algorithms; and Decision theory and Utility theory models of web 3.0 processing. This field is the advancement of artificial intelligence algorithms and has more research and advancement published by Apache Software Foundation, Google, IBM and many universities.
- Graph databases: In the world of the Internet of Things, graph databases represent the most valuable data processing infrastructure. This infrastructure exists because data will be streaming constantly and be processed by machines and people. It requires nodes of processing across infrastructure and algorithms with data captured, ingested, processed and analyzed. Graph databases can scale up and out in these situations, and they can process with in-memory architectures such as Apache Spark, which provides a good platform for this new set of requirements.
- Algorithms: The algorithm success factor holds the keys to the castle in the world of the Internet of Things. Several algorithms are available, and they can be implemented across all layers of this ecosystem.
No success is possible without identifying associated risks and pitfalls. In the world driven by the Internet of Things, the risks and pitfalls are all similar to those we need to handle on a daily basis in the world of data. The key here is that data volume can cause problems created by excessive growth and formats.
Lack of data
A vital area to avoid within the risks and pitfalls is a lack of data, which is not identifying the data required in this world driven by the Internet of Things architecture. This pitfall can lead to disaster right from the start. Be sure to define and identify the data to collect and analyze, its governance and stewardship, its outcomes and processing—it’s a big pitfall to avoid.
Lack of governance
Data lacking governance can kill a program. No governance means no implementation, no required rigor to succeed and no goals to be measured and monitored. Governance is a must for the program to succeed in the world of the Internet of Things.
Lack of business goals
No key business actions or outcomes can happen when there are no business goals established. Defining business goals can provide clear direction on which data and analytics need to be derived with Internet of Things data and platforms. Two important requirements for these goals helps avoid this important pitfall: one is executive sponsorship and involvement, and the other is governance. Do not enter into this realm of innovative thinking and analytics without business goals.
Lack of analytics
No analytics can lead to total failure and facilitates non-adoption and a loss of interest in the Internet of Things program. Business users need to be involved in the program and asked to define all the key analytics and business applications. This set of analytics and applications can be documented in a roadmap and delivered in an implementation plan. A lack of analytics needs to be avoided in all programs related to the Internet of Things.
Lack of algorithms
No algorithms can create no results and translates to non-adoption of the program. A few hundred algorithms can be implemented across Internet of Things platforms and data. These algorithms need to be understood and defined for implementation, which requires some focus and talent in the organization both from a leadership and team perspective. Algorithms are expected to evolve over time and need to be defined in the roadmap.
The use of incorrect applications tends to occur from business users with a lack of understanding of the data on the Internet of Things platform, and it is a pitfall to avoid early on. The correct applications can be defined as proof-of-value exercises and executed to provide clarity of the applications. Proof of value is a cost-effective solution architecture build out and scalability for the Internet of Things platform.
Failure to govern
If no effective data governance team is in place, implementing, or attempting any data or analytics, can be extremely challenging. This subject has been a sore point to be resolved in all aspects of data, but has not been implemented successfully very often. For any success in the Internet of Things, the failure to govern pitfall needs to be avoided with a strong and experienced data governance team in place.
More insights in your journey
Hopefully you will find these shared insights useful, and you can continue these discussions in your own implementations. In addition, be sure to check out the IBM Watson Data Platform, which can be an enabler for developing cognitive Internet of Things applications.
Editor’s note: This article by Krish Krishnan, president and CEO at Sixth Sense Advisors, Inc., is offered for publication in association with the Big Data Conference, 13–15 March 2017, in Washington, DC at the Crystal City Hilton. The event is sponsored by Data Management Forum. Additional information is available in the Big Data Conference flyer or this summary flyer.