Hitachi

U.S.A.

Hitachi Group Global Network

Americas

Asia

Europe

Middle East and Africa

Oceania

Close

The Next Generation of MSS: Big Data, Artificial Intelligence and Machine Learning
You are here: Home \ Artificial Intelligence \ The Next Generation of MSS: Big Data, Artificial Intelligence and Machine Learning
Artificial-Intelligence_Hitachi-Systems-Security
Posted on Tuesday, December 4th, 2018 by

The true key to an actionable risk management strategy is data

 

This is the third article in a 3-part series focused on the rise of risk management over traditional threat hunting provided by MSSPs through Managed Security Services (MSS).

 

Read part 1 here: The Next Generation of Managed Security Services: Beyond IT

Read part 2 here: The Next Generation of MSS: Proactive Risk Management

 

In the past two posts, we touched on the overview of the evolving managed security service (MSS) landscape, and how proactive threat intelligence has become an essential need of next-generation MSS providers.

But how exactly does it all come about? What lies beneath? What gets us from theory to a real-world, brass-tacks, bad-guy-squashing security strategy?

The answer to those questions is simple, and it’s everywhere: data.

 

The Role of Data

 

Everything that is connected to the network in today’s organization generates data, and a lot of it. This data could be in the form of logs or just a monitoring data stream.

But all of this data hides a very important key – patterns. Gleaning insights from these patterns would give us a window to predict and protect; predict threats, and protect critical assets in organizations.

So, how does data go from being just data to a true ‘asset’ when it comes to information security? And why does it matter in the context of MSS platforms?

In this post, we will be going over three things that would connect all these dots, enable cutting-edge threat hunting, and deliver proactive security:

  1. Advanced Data Sources
  2. Big Data
  3. Artificial Intelligence and Machine Learning

 

1.   Advanced Data Sources

 

Traditionally, MSSP platforms have ingested various kinds of logs such as application logs, firewall logs, network device logs, etc. These conventional data sources have been very good for reacting to threats, general monitoring, and responding to incidents.

However, they are not enough when we want to get proactive about threats and security. For a more comprehensive approach, these logs need to be accompanied by other sources such as data feeds from Internet of Things (IoT) devices, Operational Technology (OT) devices, social media, dark/deep web sources, etc.

A good MSS platform would correlate between these various data sources to create a full scenario that would aid in effective threat hunting – both proactive and reactive.

 

2.   Big Data

 

As data sources and data volumes increase, the underlying data structures and storage methodologies within MSS platforms also need to scale.

Legacy database systems may no longer be sufficient to handle these large scale data sets. Traditional databases were designed to handle data that was predictable in terms of scale as well as volume and had a set structure to it.

Today’s data sources are unstructured and very dynamic. This is where big-data-friendly database systems which are NoSQL-based come in handy. A platform’s ability to ingest and process large amounts of data depends on the way it’s (big) data architecture is designed.

Big data technologies allow the platform to perform correlations and processing on very large unstructured data sets. Big data frameworks also allow for features such as cell-level security to enable highly-specialized use-cases such as GDPR policies and compliance enablers to be implemented within MSS platforms.

 

Big data technology stacks typically consist of the following layers:

 

  • Data Warehouse: This layer consists of the containers that hold big data, i.e. the NoSQL-based database structures.

Examples: Oracle Exadata, Teradata, Azure HDInsight, etc.

 

  • Storage: This layer consists of file systems that can handle unstructured data.

Examples: IBM GPFS, Hadoop HDFS, Apache HBASE, etc.

 

  • Compute: This layer is where the magic happens in terms of stream processing, correlation, and pattern detection as well as insights.

Examples: Hadoop MapReduce, Apache Drill, Spark, TEZ, etc.

 

  • Analytics/Visualization: The last layer is the presentation layer that helps us make sense of the data, insights, and its actionability.

Examples: IBM Cognos, Tableau, Power BI, etc.

 

3.   Artificial Intelligence and Machine Learning

 

The way to make sense of large amounts of unstructured data that comes from a variety of sources is to leverage artificial intelligence (AI) and machine learning (ML) technologies. That is because doing that manually would be next to impossible – only machines can help us handle that kind of scale and volume of data.

 

Risk Management Data Lifecycle

Figure 1: The Risk Management Data Lifecycle

 

The benefit that big data provides when it comes to AI and ML is that it allows for a larger accuracy when trying to find patterns and insights within the dataset. This leads to better correlation, which in turn leads to meaningful and actionable data. Data on its own has no value until it becomes actionable.

Machine Learning takes this one step further: it allows for these actionable pieces of data to become part of the AI analysis process – this would directly drive (near) real-time prediction of outcomes on the basis of those actionable patterns in data, elevating the MSS platform from being just a data processor to being a self-learning system that improves itself over time.

 

Making Sense of it All

 

So, where does all of this leave us? Circling back from where we began, we come to a realization that a true risk management platform is a combination of many things:

  1. Increasing the scope to go beyond traditional IT infrastructure.
  2. Ingesting data from a large variety of data sources.
  3. Using big data technologies to hold and process all of this data.
  4. Leveraging AI to discover patterns and correlations in this data to make it actionable, predictable, and proactive.
  5. Finally, utilizing ML to embed learnings from the AI pieces into the platform itself – to allow the system to learn, grow, and become more effective as time goes by.

 

No single piece of this is enough on its own to solve the challenges faced by organizations today when it comes to security. But together, these would help build the ultimate risk management strategy that would not only mitigate threats of the future but also predict and prevent those threats before they occur.

 

Hitachi Systems Security’s ArkAngel platform is evolving to continue being on the forefront of this new information security age.

Reach out to us so that we can help you realize an airtight risk management strategy by harnessing the true power of data to deliver accurate and proactive security for your assets.

Siddharth Mankad
About author:
Currently Product Manager at Hitachi Systems Security, Siddharth ("Sid") is a cross-functional, influential and transformative product manager with 7 years of experience delivering enterprise products and features that have been market leaders with multi-million-dollar revenues in the data protection space. As “best of three worlds” product manager, Sid has deep expertise in user experience design, software development, and a background in business – to lead and drive execution in the creation of market-leading products and features. He has superior technical and user-centric design skills, and worked hands-on as a designer and front-end developer to create a shared library of user interface components for use by developers across the organization.

Subscribe

Recent Videos

What is Penetration Testing?

What is a Vulnerability Assessment?

What is a Control Assessment?


More