Predict, Prevent, and Act on Security Threats

Nature of analytics: Security intelligence from real-time insight helps mitigate black swan events

Manager of Portfolio Strategy, IBM

What does a black swan have to do with enterprise security? Bird aficionados well know that spotting a black swan is unusual. Black swans, which can often be confused with magpies, are hard to find among the 10,000 species of birds in the world. As a result, the black swan has become synonymous for a high-profile, hard-to-predict, and rare event that is beyond the realm of normal expectations.

Unfortunately, black swan events are all too common when it comes to data security. Today’s sophisticated criminals leverage a vast arsenal of techniques. They are highly motivated, patient, persistent, and well funded—sometimes even state-sponsored. They watch and wait for the right moment to commit a breach.

Cybercriminals are taking advantage of today’s highly instrumented and interconnected world to hide their attacks, just like a black swan hides in the flock. And every day the world creates 2.5 quintillion bytes of data,1 creating camouflage for a new generation of cybercriminals.

Data security in the spotlight

Security threats and attacks get a lot of press coverage, and many dedicated sites such as InformationWeek's Dark Reading site2 and institutions such as the Ponemon Institute3 try to help make sense of the attacks. Cybercrime dramas also make for good television viewing and major motion picture releases. New story lines about cybercriminal high jinks and nefarious activities continue to increasingly fill the airwaves.

And media organizations are not the only ones making good dramas. The IBM big data team just released its “The Nature of Analytics” trilogy. The first installment, “The Swan of All Fears,” features an elusive black swan chased down by IBM® InfoSphere® Streams streaming analytics software in a leading role. This IBM video portrays InfoSphere Streams as the intrepid hero successfully averting serious attacks and engaging in a battle of wits with the world’s most dangerous forces.

In all seriousness, organizations are spending a lot of time, money, and resources on thwarting security attacks. A recent Infosecurity website news article covering the RSA Conference 2014 event’s keynote address states that spending in the security market is around USD46 billion.4 Yet many organizations are missing the mark. The cost, complexity, and severity of attacks are increasing.

Current approaches to network security can combat known threats, but they are not as good at finding new associations or uncovering patterns. As a result, organizations are opening the door to advanced persistent threats (APTs), spear phishing, hacktivism, and other dangers.

Within all the noise of big data, organizations need sophisticated real-time analytics to find a relatively weak signal. Without deep insight, most threats cannot be detected. The goal is to predict, prevent, and act on threats to minimize damage, maintain a strong brand image, and keep employees, businesses, and information safe and secure.


Big data analytics–provided security intelligence

Security intelligence derived from big data analytics provides real-time insight across all data types, including traditional security data types—log files and audit trails—and big data types—social data, photos, sensor data, and email. This intelligence enables organizations to sift through massive amounts of data generated inside and outside of the organization to uncover hidden relationships, detect patterns, and remove security threats.

Essentially, security analytics blend real-time analytics on data in motion with historical analysis on data at rest. Organizations can achieve the following results when deploying security-specific analytics:

  • Enhanced intelligence and surveillance insight. Analyzing data in motion and data at rest helps organizations find new associations or uncover patterns and facts. This real-time—or near-real-time—insight can be invaluable for detecting new kinds of threats.
  • Real-time cyberattack prediction and mitigation. Analyzing network traffic helps organizations discover new threats early and react quickly before they propagate.
  • Crime prediction and protection. Analyzing data from the Internet such as email and Voice over Internet Protocol (VoIP), smart devices such as location and call detail records, and social media can help law enforcement enhance detection of criminal threats and collect evidence. Instead of waiting for a crime to be committed, organizations can address the threat proactively.

For example, an organization can use real-time streaming security analytics for deep packet inspection, which allows them to monitor web traffic, Domain Name System (DNS) lookups, network flow, and port and protocol usage. The outcome of this analysis may reveal precisely which web servers are infected with malware, identify suspicious domain names, pinpoint leaked documents, and help deliver intelligence on patterns of data access.

This detailed analysis informs data protection policies, so organizations can apply the right approach where it is needed most. For example, analytics helps organizations know which data to mask, which documents to redact, and which data sources—including databases, data warehouses, and big data platforms—to monitor. This analysis helps identify imbalanced situations in which organizations use lax security on highly accessed data but have extremely tight policies—such as encryption and monitoring—on rarely used sources. Top security solutions should enable organizations to carry out the following tasks:

  • Write custom queries against historical data
  • Analyze and apply different sets of analytics to find past incidents
  • Visualize results in varied ways to uncover attacks
  • Mine new data types in real time for security threats
  • Define sensitive data and share definitions across the enterprise
  • Discover and classify sensitive entities to be protected
  • Mask and redact sensitive data
  • Monitor all activity, including data, networks, applications, and more

Big data analytics security outcomes

A security strategy should be designed to keep assets safe and improve capabilities to predict and prevent attacks. A security strategy specific to big data enables organizations to achieve the following goals:

  • Stop threats quickly. This goal helps reduce the cost, duration, and severity of attacks. It also can strengthen brand reputation and client loyalty while protecting stakeholders.
  • Scale security policies. Scaling policy across traditional and big data environments helps organizations create sophisticated security analytics efficiently. It also helps analyze new data sources automatically and update business policies in real time.
  • Focus on the greatest risk. Setting priorities helps uncover unknown threats and shift resources as required. A security strategy utilizing big data analytics helps organizations understand false positives, remove blind spots, and track the effectiveness of different policies.

By achieving these goals, organizations can dramatically reduce black swan events and deter the threat of cyberattacks in their environments. And what can codfish tell you about your business? Look for this upcoming installment starring InfoSphere Streams.

Please share any thoughts or questions in the comments.

1 Big Data at the Speed of Business,” big data and analytics overview,   Nature of Analytics – Black Swan
2 InformationWeek Dark Reading website.
3 Ponemon Institute website.
4 RSA Conference 2014: Organizations Spending Too Much on Security Technology,” Infosecurity website, news item, February 2014.


• Empower security analysts with big data analytics,” IBM Software ebook, March 2014.
• Smarter security intelligence,” IBM Software solution brief, October 2013.


[followbutton username='madiakc' count='false' lang='en' theme='light']
[followbutton username='IBMdatamag' count='false' lang='en' theme='light']