Big Data, Meet Internet Security

Communications Manager, IBM Security Systems

As many of you likely know, IBM has been building and extending new analytics capabilities for years now and across every industry imaginable. Some of these capabilities are purpose-built for specific functions and requirements, while some of our other tools are more like an analytics framework that can adapted to specific workloads and challenges. These types of solutions are often the product of the close relationship between our technology teams and the organizations we're working with to solve challenges. We bring our expertise in handling and visualizing extremely large data sets of both structured and unstructured data and they bring their core domain expertise. They know what data they have and what they hope to get out of it and we bring a set of capabilities to help them do that.

Regardless of the type of system eventually constructed, there are some consistent themes popping up in the world of data and analytics, one of which is around specificity. Whether we are talking about consumer marketing strategies or political campaigns, understanding the individual in terms of broad demographics is becoming a bit of an outdated construction. People are complicated and uniquely motivated, and while understanding broad demographic trends is better than nothing, beginning to understand individuals is a lot better than understanding broad demographics.

This isn't a new idea either, the good direct mail campaigns of yesteryear were always deeply interested in how to cater to their audience as specifically as possible.

Now we are seeing some of that thinking spill into the world of internet and computer security, especially in the case of the more advanced attackers. For a long time, security technology was designed in a manner similar to how people used to think about marketing. The technology was designed to stop broad classes of threats instead of specific individuals. Attackers exploited web applications so we built application vulnerability scanning technology. They sent phishing emails with attachments that had never before seen malware to senior executives, so we built network defense technologies that became increasingly more advanced in the way they block even unknown attacks. They used legitimate network credentials to access and exfiltrate data so we started baselining user behavior in order to detect deviations from the norm. I could keep going, but the issue is that much of what you're hearing might sound similar to how people used to market to large demographics. It's addressing classes of behavior but not terribly concerned with understanding one person.

The issue with this approach is that in marketing, not quite understanding a person might make your tactics less effective and could hurt you around the margins (granted, this is where many businesses live or die), but in security, not being able to understand a specific attacker means you can't defend your network against the most advanced threats, the ones who can damage your business/organization the most. The reason these attackers are so dangerous is because they are frequently not generically after credit card data or something along those lines, they want intellectual property, business plans, long-term access to strategically significant networks so they can monitor communications for years and years. Sure, it's no good if your website is down for two hours. It's a lot worse if someone going after you is on your network for two years.

We need new strategies for understanding and combating the threats that matter the most, and this is where big data and the business analytics space begins to converge with security intelligence. IBM's security intelligence capabilities currently allow us to go through as much as several billion security events every day (things like network and firewall logs, network flow data, etc) and by applying analytics, take those couple billion events and boil them down to a handful of events that require priority attention and human investigation. However, as attackers are becoming more sophisticated and the data they pursue ever changing, the data security professionals need to make sense of is also changing. More advanced organizations are looking for new data sources like full email text, business process data, communications channels scrapes, etc., for the security insights that could be contained within them. To pursue this end they need to extend their existing capabilities to accommodate the increasing volume and variety of data, which is where we turn to the world of business intelligence, large data warehouses and new ways of slicing and visualizing these different types of data.

By combining QRadar with the IBM BigInsights platform we are dramatically expanding the scale and scope of investigation that a security team can pursue. This is enabling them to see potential threats on the network not in terms of broad classes of threats, but a specific threat after a specific target. Organizations will still need much of the security technology they've deployed over the years to keep the volume of incidents down, but they need to expand their skills and understanding of the underlying challenge if they want to successfully combat the most determined, talented and well funded attackers.

To learn more about the capabilities of this combined QRadar and IBM solution, visit IBM Security Intelligence with Big Data and watch the following video.