Streaming media and narrative power of video content analytics

Big Data Evangelist, IBM

Even the simplest video stream is equivalent to a massive database in terms of its intrinsic information content.

In a video stream, every picture (in other words, every frame) tells a story, much like every record in a database. Not just that, but each pixel within each frame might tell its own very specific story, equivalent to a field within in a database record. Sometimes the tiniest details reveal huge insights. And that stream may consist of dozens of frames per second, with their sequence telling another, higher-order story, much like a database table. If the stream continues indefinitely, like a table of arbitrary length, the video essentially tells a never-ending narrative.

People can't possibly comprehend every nuance within in a never-ending video stream. That's equivalent to asking them to explore every possible correlation within a petabyte-scale big-data repository. Of course, you can make the task manageable by asking them to characterize the contents of shorter, simpler video streams. But people's time and attentions are limited, which means they'll never be able to view every video source, much less describe it at every possible level of detail. It can't be done with human resources alone.

playback ss.JPG
Image courtesy of Openclipart and used with permission

But, for sure, it is technologically possible to analyze the contents of the vast range of video sources that will never be viewed by flesh-and-blood humans. Video content analytics tools are humanity's unblinking eyes, capable of continuously filtering the world's media streams at scale. As outlined in this recent blog post on video analytics from AnalyticBridge, these tools can even tune into minute visual details well below the threshold of human perception.

On their own, video content analytics algorithms can parse the fine details within and between successive frames of specific streams. As described in the cited blog, these tools can be optimized for specific fine-grained capabilities as pattern recognition (for example: faces, license plates and specific objects), gesture recognition (such as greetings and assaults), location detection (like objects/people within frame context, vis-a-vis each other, crowding and tailgating), motion detection (of objects, people and so on), event detection (such as intrusions, removals and abandonments), production-style detection, dynamic video masking and camera tamper detection.

In practice, though, most video content analytics tools are not designed to roll up stream-specific insights into narratives of what's taking place across two or more streams. In addition, it's rare to find video content analytics tools that correlate the extracted insights with intelligence from other, non-video sources. Knitting together the larger narrative within which the video content is certainly possible, but to make it happen you should be correlating the outputs of your video content analytics with other intelligence within big-data repositories.

To the extent that video content analytics figure into larger value narratives, it's often through their ability to output real-time alarms and alerts based on automated detection of various prespecified events, such as intrusions, accidents, explosions, obstacles, thefts, loiterers and crowds. As the blog states: "The objects and their movement are then compared to preset behavioral and motion parameters, and alarm sequences are initiated if certain criteria are met or exceeded."

Typically, these event-driven alerts are received by separate applications, such as forensic analysis and security incident monitoring systems. The alert data is often logged, correlated with other data and used to drive predictive, prescriptive and historical analyses. To the extent that video-extracted data can be correlated with identity, geospatial and other contextual intelligence, it can be used to construct a comprehensive, dynamic narrative of what the video may reveal.

Video surveillance applications thrive on these sorts of correlations. But so, conceivably, could retail applications that leverage imaging analytics for behavioral targeting. In fact, I alluded to the latter scenario in this post that I wrote over a year ago. Another term you might use to refer to this customer-centric video-infused narrative is "journey."

Tread lightly as you engage customers in a video-enriched value journey. If you're using video analytics to fine-tune your targeting for marketing, sales and other purposes, you can easily stray into privacy-sensitive territory. It's unnerving when we realize that someone or something is paying close attention to our every move.

After all, no one wants to live under an unblinking eye.