The Evolution of Cortex: Building the Future of Security Operations

Introduction

Palo Alto Networks has made a splash in the industry with Cortex XSIAM promising AI and ML in the SOC to automate the tedious tasks that analysts spend too much time on. They can focus on threat hunting and more complex SOC tasks. But did you know Palo Alto Networks has been building towards this future for over a decade? Since 2009, I've been working with the Palo Alto Platform, witnessing this momentum to secure the future year over year. Through strategic acquisitions and a commitment to innovation, Palo Alto Networks built Cortex to effectively empower organizations to detect, respond, and protect against cyber threats. In this post, I'll dive into the history of these acquisitions and innovations that built Cortex, culminating in Cortex XSIAM, a revolutionary step toward the automated SOC of the future in this new AI future. 

Cortex Data Lake: the foundation

We're going to start a little out-of-order on the timeline. Around 2015, I remember a new compliance requirement that we keep a year's worth of firewall logs in our environment. I remember doing the math on the storage requirements for our NGFWs would require about 5TB of disk space to be added to our current Panorama environment. I can still remember the SAN administrator coming by my office the next day, knocking on the door, and laughing that I had just requested the entire amount of new disk space she had projected our environment to require of the next two years. Our operations team even let out a big sigh that they didn't have the space or throughput to backup this massive additional virtual disk.

This was the problem that most companies were experiencing at the time. The cost of running local Panorama Log Collectors or exporting the data to the SIEM wasn't scaling at the performance needed. In 2017 at the Ignite Conference, the Cortex Data Lake debuted as bandwidth and storage cost in the cloud were getting cheaper and cheaper. The thesis of the Data Lake was build it, and customer's will migrate to this model since they can reclaim the expensive storage onsite while Palo Alto will handle the availability of these large datasets. By moving these logs to the cloud, Cortex Data Lake was able to solve a fundamental problem on the horizon: data fragmentation in logs.

Later in 2017, Palo Alto acquired LightCyber's Magna platform that was used to perform analysis of network traffic. Magna was the first step in providing analysis based on data in the Cortex Data Lake from the NGFWs (and eventually the forthcoming XDR Agents.)

There has been some divergence in these names in the last year. The traditional Cortex Data Lake has been renamed to the Strata Logging Service to avoid the confusion around the Cortex XDR Data Lake. Consider the Strata Logging Service more network operations-centric to collect logs from the NGFWs and Prisma Access to build better rulesets with the IoT license. In contrast, the Cortex XDR Data Lake is more geared to the security operations center. 

This Data Lake concept serves as the foundational data repository that integrates all security data from Palo Alto Networks products, creating a centralized, and scalable environment with less administrative overhead. With the Cortex Data Lake, security operations teams can collect, store, and analyze vast amounts of data from multiple products in multiple regions. It has paved the way for deeper integration, powering Cortex's advanced threat detection and response capabilities as the backend is based on scalable cloud infrastructure to query logs quickly compared to flat storage. 

Cortex XDR: setting the TRAPS for malicious actors

Around 2011, the most common way to distribute endpoint security software updates was a DAT file of bad hashes the vendors were aware of, but change was coming fast with the debut of ransomware and polymorphic viruses where the hash could be quickly changed. I remember reviewing our network graphs, watching the endpoints grab this 60MB pattern file every hour from the central server, and seeing the peak traffic on our LAN environment. I also worried about our endpoints that were disconnected from the LAN and VPN would get these pattern files in time. Palo Alto introduced WildFire on their NGFW's to respond to these new threats at a network level, but the increase of SSL was impacting the firewall's ability to inspect traffic as MITM Decryption was still in the early days. 

In early 2014, Palo Alto Networks acquired Morta Security to supplement what they had built into WildFire. Morta Security provided advanced malware detection and threat intelligence. Morta's technology set the foundation for Palo Alto Networks' broader security strategy to provide advanced, integrated threat intelligence across the NGFWs and endpoints.

In the Spring of 2014, Palo Alto Networks acquired a startup named Cyvera that had built a product called TRAPS (Targeted Remote Attack Prevention System). This was Palo Alto's first step out of their traditional network security market segment. TRAPS was an inverse of what many endpoint security products were focusing on at the time. Instead of tracking harmful files with a pattern file, TRAPS focused on the most common exploit techniques, software logic flaws, and memory corruption. My explanation for TRAPS was always "anti-malware acting like malware to prevent malware." TRAPS would use an Exploit Prevention Module to load a DLL into the most exploited executables, watch for any attacks, and then quit the executable when detected. However, TRAPS lacked an anti-virus scanning engine. The previous acquisition of Morta, combined with WildFire, was integrated into TRAPS to combine this continuous feedback loop from the endpoints to the NGFWs with Palo Alto's Threat Cloud concept.

TRAPS was initially an on-premises application called the TRAPS Endpoint Security Manager that was built on Windows IIS and Microsoft SQL servers with the option to forward these logs into Panorama Log Collectors for a consolidated view of threats and activity. But as the threat of ransomware continued to rise, Palo Alto decided to move the management and distribution of content updates from potentially vulnerable on-premises applications to a cloud platform called TRAPS Managed Service, utilizing the new data lake, in 2018. 

Although TRAPS was excellent at stopping these new threats and ransomware attacks, it still wasn't the best at telling the analyst the "why and how" of the danger. In the Spring of 2018, Palo Alto made another acquisition of SecDo, which records everything on the endpoint and sends the data to an analytics engine to help the analyst understand the incident response's end-to-end. This was an early form of AI+ML that used the Cortex Data Lake (and the LightCyber Magna acquisition) to visualize threats as they occurred across the endpoints and network. I always like to remind people that Palo Alto paused after the SecDo acquisition and the development of TRAPS. During this pause, Palo Alto took the codebases from TRAPS and SecDo to create a new product on a new backend. 

In early 2019, Palo Alto announced Cortex XDR, which redefined extended detection and response by correlating data across an organization's environment—endpoints, networks, and beyond. Cortex XDR provided a single front end for analysts to respond to incidents and a single back end for additional capabilities such as UEBA, Host Insights, and Forensics. XDR has a record of innovation over the last few years, including a flexible query language called XQL for threat hunting, creating indicators of compromise, and implementing simplified automation. 

By combining the strengths of Cyvera, SecDo, and LightCyber, Cortex XDR became one of the first products capable of unifying threat detection and response across endpoints, networks, and cloud environments.

Cortex XSOAR: transforming the SOC with automation

Even with XDR, there was still a gap in the SOC as most enterprises relied on other toolsets and processes unrelated to the NGFW or endpoint. SOC analysts also dealt with multiple alerts from different product sets, with each analyst responding to the alert differently. In my early days, I often remember being called on a SEV1 incident and seeing the chaos of different personas across the enterprise joining the web conferencing meeting, sharing the screens of multiple consoles, and then dealing with the random notes taken down on paper or in notepad as we tried to reconcile our memories in a report for the C-Suites and board members detailing the outage and next steps. 

In early 2019, Palo Alto Networks acquired Demisto (renaming the product XSOAR). Demisto was the leading SOAR provider, offering playbooks for automated incident response, case management, and collaboration. The strength of Cortex XSOAR is the visual playbook editor, allowing SOCs to drag and drop existing playbooks and tasks or add their own PowerShell or Python scripts to respond to incidents. Cortex XSOAR also includes a virtual war room for teams to collaborate and document actions as the incident progresses from opening to closure. 

Cortex XSOAR allowed SOCs to move away from manual processes and adopt automated, playbook-driven responses. I often remind people that XSOAR is still platform agnostic and can integrate with any API or system, even from other vendors, including competing endpoint and firewall products. 

Unit 42: a human touch to incident response

Since 2014, Palo Alto's Unit 42, a nod to "The Hitchhiker's Guide to the Galaxy," has been a threat intelligence team created to provide their intelligence to the security community and product feedback to the entire Palo Alto Platform. In mid-2020, Palo Alto Networks acquired The Crypsis Group to enhance the capabilities of the existing Unit 42 team. This acquisition enabled Unit 42 to offer a comprehensive suite of services, including proactive cyber risk assessments, tabletop exercises, litigation support, and rapid incident response. These new services can provide direct feedback to the Palo Alto Platform about active breaches in the field. 

Over the years, Unit 42 has been recognized for its contributions to cybersecurity. Unit 42 is a founding member of the Cyber Threat Alliance, which provides validated, curated threat intelligence to other vendors and organizations that might not otherwise have, including files, domain names, addresses, and MITRE TTPs. Unit 42 also publishes their threat research on their website  and the Unit 42 public GitHub for everyone in the cybersecurity community to take advantage of and improve their security posture.

Unit 42 continues to serve as a trusted advisor to organizations worldwide of all sizes. It leverages its combined expertise in threat intelligence, incident response, and cyber risk management to help clients proactively manage cyber risks and respond swiftly to security incidents. 

Cortex Xpanse: expanding attack surface management externally

While most Cortex products focused on internal security operations, Cortex Xpanse addressed a new, growing need in cybersecurity: understanding and managing external attack surfaces. Modern organizations often have an expansive digital footprint, making tracking assets and identifying vulnerabilities difficult. From outsourced applications to departments using public cloud resources outside the purview of the organization's IT department, Expanse, acquired in late 2020, brought visibility into all public-facing assets and continuously scanned for vulnerabilities and misconfigurations.

Cortex Xpanse helps organizations discover unknown assets, prevent shadow IT, and reduce their exposure to external threats, whether in an on-premises DMZ environment, an upcoming merger and acquisition, or services hosted in the public cloud. With this addition, Cortex expanded beyond internal defenses, creating a holistic view of an organization's security posture.

Cortex XSIAM: the culmination of Cortex

After years of building an integrated product set from in-house tools to strategic acquisitions, Palo Alto Networks took a significant step forward with Cortex XSIAM—a fully automated, AI-driven SOC platform designed to finish the job the SIEM started almost 20 years ago—much like they did with PAN-OS and the Next Generation Firewall. Cortex XSIAM represents the culmination of XDR, XSOAR, and Xpanse to integrate logs, detection, automation, and response into one unified platform.

Cortex XSIAM combines the core capabilities from the Cortex Data Lake, Cortex XDR, Cortex XSOAR, and Xpanse by leveraging artificial intelligence to automate threat detection, response, and prioritization. By unifying these elements, XSIAM empowers SOC teams to:

• Detect threats in real-time by correlating data from across the organization's assets.
• Automate incident response consistently with playbooks to reduce the median time to resolution.
• Drive operational efficiency with AI, reducing the need for manual intervention so that SOC Analysts can focus on essential alerts.

As the ultimate realization of Palo Alto Networks' vision for security operations, Cortex XSIAM enables organizations to operate a genuinely modern SOC, where technology and automation work seamlessly to provide a robust defense against today's advanced cyber threats.

Conclusion

Cortex's journey from data lake to endpoint to XSIAM exemplifies Palo Alto Networks' commitment to pushing the boundaries of security operations. By acquiring pioneering companies and developing cutting-edge technology, Palo Alto Networks has built Cortex as a comprehensive, unified platform that adapts to an ever-changing threat landscape. Cortex XSIAM stands as a testament to the power of integrating datasets, automation, and innovation, offering organizations a SOC experience that's more efficient, resilient and prepared for the future.