Thoughts on Backup Retention
In this blog
Introduction
The World Wide Technology Data Protection Practice performs dozens of Data Protection Workshops each year. As part of the discovery process, we ask organizations what their Recovery Point Objective (RPO), Recovery Time Objective (RTO) and Backup Retentions are. RPO and RTO are usually set as Service Level Objectives (SLOs) between the Data Protection team and other business units. Retention typically has a component of daily, weekly and monthly versions with durations ranging from days to years.
Our follow-up question for retention is always, "Who decided the required retention and how was it selected?" The most given answers are "We've always done it that way" and "That's what the application owners want." We rarely hear retention is dictated by a company's legal or compliance groups who are responsible for providing direction for data and backup retention.
In this article, we're examining the value of retaining backups for longer than an absolute minimum period. It is my contention that retaining backups beyond 15 days is unnecessary and only adds cost without providing value.
Backup fundamentals
The primary purpose of performing backups is to enable recovery in the event of hardware failure, accidental deletion, corruption, or other data loss events, including ransomware. While the fundamentals are well known, here are just a few considerations that must be addressed before we consider retention.
Immutable backups
With the ever-increasing threat of ransomware, performing immutable backups is fundamental to ensuring successful restores. Backups are frequently used as the last line of defense and ransomware intruders target them to eliminate that safety net and force a ransom payment.
3-2-1 rule
The 3-2-1 rule of backups is the gold standard for data protection. The rule dictates you keep at least three copies of your data (one primary and two backups), store the copies on two different types of media, and keep one backup copy offsite.
Backup frequency
Establishing a backup schedule to meet an organization's RPO is a core requirement of any backup strategy. Application dependency mapping can help determine what key applications need to be backed up frequently.
Testing and verification
Regular testing of backup and recovery processes ensures data can be restored effectively. Verifying backup integrity ensures that backups are not corrupted. Testing procedures should be detailed enough to determine if the RTO can be met in the event of a large-scale restore. It is also important to test to ensure that applications can be recovered/resumed from the restored data.
Data retention vs. backup retention
Let's examine the distinction between data retention and backup retention. Data retention is almost always dictated by legal and compliance requirements and defines how long data should be saved as well as rules for when and how it should be disposed of once it is no longer required. Legal requirements for data retention include the Sarbanes-Oxley Act (SOX), the European General Data Protection Regulation (GDPR), the Payment Card Industry Data Security Standard (PCI-DSS), and the Health Insurance Portability and Accountability Act (HIPAA). New legal requirements are always being introduced and will vary depending on jurisdiction.
Backup retention defines how long a secondary copy of data needs to be maintained when loss, damage or disaster occurs. Fundamentally, backups should only be retained long enough to provide the capability for an organization's data recovered to the point in time nearest to when a data loss event occurs. Regardless of retention time, the 3-2-1 rule for backups must always be implemented.
All of the compliance requirements listed above dictate retention for data and record keeping, but none of them provide guidance for backup retention. In fact, the GDPR right to erasure (also known as the "right to be forgotten") makes retaining backup data for longer periods a liability since backups frequently contain data on individuals that have been erased from production systems.
We frequently see backup retentions of five or more years due to a mistaken belief that there is a legal or regulatory requirement to do so. If the production systems already contain data in compliance with the data retention requirements, then the backup should only be retained long enough to protect and restore that data in the event that it is lost or damaged in production.
Backup vs. archive
Another consideration is data backup vs. data archiving. Data backup creates a copy of production data to ensure that it can be recovered in the event of loss or corruption. Data archiving is the process of moving data that is no longer actively in use from production to a separate (presumably less costly) location for long-term retention. The key distinction is that data backup makes copies of production data while archiving moves data to another location and deletes the original, often replacing the data with a 'stub' or shortcut that points to the archive copy.
Archived data consists of older information that is still important or must be retained for future reference or regulatory compliance. Archiving is devoted to freeing up space on primary storage and applications to improve performance and reduce costs, while still meeting legal and regulatory data retention requirements.
Many backup products support archiving by copying selected items to the destination storage and then deleting them from the source once the copy is verified as successful. The archived data has a retention assigned that is typically years or even decades. It's important to emphasize, that even though the backup software is performing the archive, the source data is deleted. Unlike a 3-2-1 backup, you should consider a 2-1 archive strategy where you use two different archive media and maintain one copy offsite.
If you choose to archive data from your primary storage and applications to another location, the rule for a 3-2-1 backup still applies to the archive data.
Legal discovery
Regulatory compliance requires an organization to be able to find and access relevant information whenever required within all data it retains, including backups. eDiscovery is the process of producing data from electronic media in response to a subpoena or equivalent legal request. Response times are often short, and failure to respond to these requests in a timely and complete manner can result in fines or additional legal penalties.
Since organizations can only be required to produce documents held according to policy, retaining backup data longer than necessary for operational or disaster recovery is not only costly but can put an organization in the position of having to bear the cost of restoring older backups as part of the discovery process. Therefore, it makes sense to retain a limited set of backups to ensure that the data retained can easily be searched for information at a reasonable cost to the business.
While keeping a short backup retention can reduce the cost of eDiscovery by reducing the sheer amount of backups being investigated, it's vitally important to understand the obligation to protect backups in the event of an impending legal hold. According to the eDiscovery firm Casepoint, "… the duty to preserve attaches once litigation or investigation is reasonably anticipated. The duty also attaches when litigation or a regulatory event is foreseeable, also known as a 'triggering event.'" Legal hold is essentially a "forever" retention since it can be years or decades before the hold is released.
Financial cost of retention
Every backup architecture is composed of three fundamental components: the control plane, the data movers and the target storage. How each vendor implements these components varies significantly with each having some advantages depending on the backup use case.
Regardless of the method of implementation, retaining more backups results in more cost to implement and maintain. The control plane needs to manage more metadata and restore points and the target storage needs to be larger to store more backups. If backups are retained long enough, moving older backups to secondary storage such as tape or object storage may be required, driving additional cost and complexity.
If we look at the value of a backup vs. its age, it is easy to conclude that the longer a backup is retained, the less valuable it becomes. Older backups are less valuable as the data in the backup set diverges further from what is in production. The value of the backup should also be tied to the RPO for restoring the data. For example, if your organization's RPO for a ransomware event is five days, keeping 30 days of retention shouldn't be an objective. We'll address ransomware next in this article.
The result of retaining older backups is the value of the backup decreases quickly with time, while the physical and administrative costs of maintaining the backup remain constant, resulting in an increasing cost to value as the backup ages.
Recovery from ransomware
One area to possibly advocate for longer retentions is for recovery from ransomware. Many cyber resilience plans require performing test recoveries, going in reverse order from newest to oldest, and evaluating each backup for malware until a non-infected backup set is found. Depending on the dwell time of the malware, days or even weeks' worth of backups may need to be restored before an uninfected backup can be found.
However, according to Mandiant's M-Trends 2024 Special Report, "The global median dwell time—dwell time is the number of days an attacker is on a system from compromise to detection—continued its downward trend in 2023, and is now 10 days (from 16 days in the previous year)."
The Secureworks 2023 State of the Threat report states, "Average dwell times between initial access and ransomware payload delivery have dropped significantly to a median figure of just 24 hours."
With the trend of toward near-zero dwell times, the argument to retain backups for longer periods loses much of its merit. When we combine this with the rapidly declining value of backups as they age, the need for retaining backups for extended periods becomes much less significant.
Summary
Data retention requirements are well documented by numerous laws and regulations that dictate how long data and records must be kept. When determining the retention period for data, an organization must take guidance from its legal and compliance teams to ensure proper data governance through the complete life cycle of the data.
Given that nearly all data an organization maintains for compliance purposes is either online or in an archive, there is little value in retaining backups beyond the minimum time it will take to safely restore the data from a data loss event. This minimum should be determined by an organization's Recovery Point Objectives, with a reasonable but limited buffer of additional copies to ensure a timely and useful recovery in the event that a backup becomes damaged or corrupted.