PAN-OS and EDLs - Everything You Wanted to Know about External Dynamic Lists

Many feature sets generally get thrown around with next generation firewalls, but External Dynamic Lists (EDL) are one of my favorites. If you haven't heard the term before, External Dynamic Lists allow the Palo Alto Firewall to dynamically query a webpage of IP addresses, URLs, and domain names and use them as a single object in your security policy. 

You can use the objects in allow or block policies. The best part is that dynamic lists can lower the amount of change requests. The best example is setting up an EDL  in which the SOC can add IP addresses, domains, or URLs and the firewall will update that list at a set frequency. Now, the SOC can control its destiny by adding and removing malicious resources without making a firewall change request.

There are a few limitations for EDLs that Palo Alto spells out from their TechDocs to be mindful of when architecting EDLs in your environment:

• You can use up to 30 external dynamic lists with unique sources across all Security policy rules.
• The maximum number of entries supported for each list type varies based on the model (refer to the different limits for each external dynamic list type). List entries count toward the maximum limit only if the external dynamic list is used in a policy rule. If you exceed the maximum number of supported entries, a System log is generated and skips the entries that exceed the limit.
• The external dynamic lists are shown in the order they are evaluated from top to bottom. Use the directional controls at the bottom of the page to change the list order. This enables you to reorder the lists to make sure that the most important entries in an external dynamic list are committed before you reach capacity limits (remember to uncheck the box that says "Group by Type" to make these changes).

You can access the EDLs from Objects > External Dynamic Lists in the WebUI. EDLs can be configured to update every 5 minutes, hourly, daily, weekly and monthly.

The two most important buttons at the bottom toolbar are Import Now and List Capacities. Import Now will run the EDLFetch job for the list, and List Capacities will show you the current values of the list versus the platform capacity.

Built-in External Dynamic Lists

If you have an active Threat Prevention or Advanced Threat Prevention license on the device, you'll also receive a pre-defined feed of lists:

• Tor exit IP addresses - IP addresses supplied by multiple providers and validated with Palo Alto Networks threat intelligence data as active Tor exit nodes. Traffic from Tor exit nodes can serve a legitimate purpose; however, it is disproportionately associated with malicious activity, especially in enterprise environments.
• Bulletproof IP addresses - IP addresses that are provided by bulletproof hosting providers. Because bulletproof hosting providers place few, if any, restrictions on content, attackers can use these services to host and distribute malicious, illegal and unethical material.
• High-risk IP addresses - IP addresses recently featured in threat activity advisories distributed by high-trust organizations; however, Palo Alto Networks does not have direct evidence of maliciousness for these IP addresses.
• Known malicious IP addresses - IP addresses currently used almost exclusively by malicious actors for malware distribution, command and control, and for launching various attacks.
• Authentication Portal Exclude List - Domains and URLs to exclude from the authentication policy; this list is managed by Palo Alto Networks.

These lists are updated daily from the Unit42 Threat Research. It's important to note that you cannot delete, clone or edit the settings of the Palo Alto Networks malicious IP address feeds. You can click on the list and view the contents by clicking on the "List Entries and Exceptions" tab.

It's often tempting when building policy to loop these objects into the same ruleset. Still, I encourage you to create separate inbound and outbound block policies so when troubleshooting, you'll be able to identify blocks quickly from the Monitor > Traffic and allow you to make specific exceptions if ever needed.

Create these policies towards the top of your ruleset or in the Shared Device Group to deploy to all devices easily. 

SaaS External Dynamic Lists

Palo Alto also hosts some ubiquitous lists that you can use in your security policy. You can get a list of the more popular Software-As-a-Service providers such as Microsoft 365, Azure, GCP, Datadog, Microsoft Defender, SalesForce, Zoom, Github, WebEx, Microsoft InTune, Okta, Palo Alto Networks, Akamai and Google Workspace.

You can view the available lists here; some examples of these EDL usage could be:

• Excluding decryption for the Microsoft 365 services.
• Building QoS policies for Microsoft Teams, Cisco WebEx and Zoom endpoints.
• Limiting access from servers to specific GitHub endpoints with the application github-downloading.

Palo Alto Networks optimizes the address information received from the application providers to reduce the number of addresses published in each EDL by identifying and removing duplicate IP addresses and then aggregating the remaining IP addresses into a smaller number of contiguous address ranges.

Authentication and Encryption

External Dynamic Lists can also be configured to support HTTP basic authentication and TLS certificates. TLS certificates utilize the certificate; profiles of PAN-OS, allowing you to define trusted root certificates and CRL and OSCP functionality. Certificate profiles can also block expired certificates and unknown certificate status to ensure the EDL has not been compromised. 

If your EDL is hosted over HTTPS, and you do not provide a certificate profile, you will see this warning similar to the following in the commit status window:

External Dynamic List Palo Alto Networks – M365 Worldwide Any IPv4 is configured with no 
certificate profile. Please select a certificate profile for performing server
certificate validation.

The following steps are a quick how-to on copying the root and intermediate certificate authorities into the PAN-OS configuration:

• Download the certificates by exporting them from the browser. Be sure to download both the intermediate and root certificate root certificates.
• Import the intermediate and root certificate root certificates to the firewall under Device > Certificate Management > Certificates.
• Create a certificate profile with the root and intermediate certificates.
• Apply profile to the EDL.

You can also configure the firewall to provide a username and password to download the lists if your hosting EDL server supports HTTP basic authentication.

Troubleshooting

Sometimes, you can run into an issue with the EDLs, whether it's on the web server side or a problem from accessing the page from the management interface. Here are some of my favorite troubleshooting steps and commands. 

The External Dynamic Lists are updated using the EDLRefresh and EDLFetch jobs.

• Immediately after configuring an EDLObject in the XML configuration and committing the firewall change, the list is fetched using EDLFetch, and the list is committed using EDLRefresh.
• Upon the timer expiration, the firewall will poll for an update from the remote server using EDLFetch. If no change is identified in the newly downloaded list file, the EDLFetch timer is set to 5 minutes. Every 5 minutes, we will see an EDLFetch job until a change in the list is encountered.
• If a change occurs, a commit will be issued using EDLRefresh and the cycle continues.

The External Dynamic Lists traffic is sourced from the management interface of the firewall or can be configured to use a service route under Device > Setup > Services > Service Features > Service Route Configuration.

You can view the EDL log events in the System log by searching for "( description contains 'EDL' )."

You can refresh the lists using the CLI command "request system external-list refresh type <domain | ip | url> name <name>." This initiates the same job as the "Import Now" button in the WebUI.

You can view the lists using the CLI command "request system external-list show type <domain | ip | predefined-ip | predefined-url | url> name <name>."

You can view the next time a list will check for an update using the CLI command "request system external-list stats type <domain | ip | predefined-ip | predefined-url | url> name <name>."

admin@GW-440> request system external-list stats type url name XSOAR_TIM_Malicious_URLs

vsys1/XSOAR_TIM_Malicious_URLs:

        Next update at        : Wed Sep 13 15:25:42 2023

        Source                : https://#.#.#.#/instance/execute/EDL_Malicious-URLs

        Referenced            : Yes

        Valid                 : Yes

        Auth-Valid            : Yes

This command will also display the URL; if the EDL object is referenced in the configuration, the URL is valid and the server authentication is valid.

You can quickly test the EDL URL validity using the command "request system external-list url-test <URL>".

Now that you've read how external dynamic lists function in PAN-OS, best practices and troubleshooting steps, you can begin implementing this functionality in your security posture.