Understanding The Transition of SIEM to Data Lake and the Emergence of SOAR?

The world and everything associated with is increasingly going virtual. The new decade has made many verticals from education to construction increasingly use technology. This has, in turn, increased dependency on devices, and the internet and hence has led to a spike in endpoint devices. The increasing data is directly proportional to the increasing number of endpoints, but the downside of this is the variety of cyber threats and threat actors have also increased exponentially.

To counter these attacks and neutralize them organizations need multiple cybersecurity measures, the real challenge however is spotting these attacks based on the data trend. This is where the need for Security Information and Event Management (SIEM), Data Lakes and Security Orchestration, Automation and Response (SOAR) come in handy. To understand the need for these technologies, one has to understand what they do.

Defining SIEM, Data Lake & SOAR

SIEM: Security Information and Event Management (SIEM) is a software solution that aggregates and analyzes activity from many different resources across your entire IT infrastructure. SIEM collects security data from network devices, servers, domain controllers, and more. SIEM stores, normalizes, aggregates, and applies analytics to that data to discover trends, detect threats, and enable organizations to investigate any alerts.

Data Lake: A data lake is a centralized repository that allows you to store all the structured and unstructured data at any scale. Companies can store data as-is, without having to first structure the data, and run different types of analytics from dashboards and visualizations to big data processing, real-time analytics and machine learning to guide better decisions.

SOAR: Security Orchestration, Automation, and Response (SOAR) refer to a collection of functions and capabilities that allow organizations to streamline security operations in three key areas: threat and vulnerability management, incident response and security operations automation. To break it down further, security automation is the intelligent automatic handling of security operations-related tasks. Security orchestration refers to a method of connecting security tools and integrating disparate security systems.

Do these tools of cybersecurity converge or lead to the next emerging technology?

Companies limit the alerts and logs that they feed into SIEM due to high storage and processing fees for traditional SIEM tools. This limitation on the alerts also limits the visibility for security teams and constrains the ability of modern artificial intelligence (AI) and machine learning (ML) tools to learn and recognize potentially malicious behaviour. To address the limitations of SIEM security data lakes (SDLs) emerged providing a solution that enables unfiltered visibility for security teams. To address the cost purpose many SIEM solutions now integrate with SDLs to try and deliver the best of both worlds. The SIEMs can continue to analyze a limited set of key logs to provide meaningful security alerts, and security teams can return to the SDL to investigate those alerts within the context provided by the SDL.

While SIEM and data lakes complement each other, SOAR takes a different approach towards security. While the two technologies share some common components, they serve different purposes. While SIEM ingests various log and event data from traditional infrastructure component sources, a SOAR pulls in information from external emerging threat intelligence feeds, endpoint security software and other third-party sources to get a better overall picture of the security landscape inside the network and out. After a SIEM provides an alert, it’s up to the administrator to determine the path of an investigation, in contrast, a SOAR automates investigation path workflows.

Authored by: Vivek Balaji A., Director - Technology, ANLYZ