10 Steps You Can Take Now to Address Major Data Challenges

This article was originally published on January 21, 2021, and has been updated to reflect current trends. 

Data is an organization's most valuable asset and the heart of digital transformation. It is ever-changing and, seemingly, never-ending. IT and business leaders face major challenges in developing new approaches and strategies for storing and managing their data in a way that enables, empowers and protects their organizations while leveraging it to innovate and grow.

Here are some ways you can begin to address these challenges as they continue to unfold.

Challenge #1: Cyber resilience 

With today's evolving cyber threat landscape, cyber resilience has become a topic that continues to gain a lot of attention. The threat is growing exponentially. Experts from Cybersecurity Ventures estimate that ransomware is expected to attack a business, consumer or device every two seconds by 2031, up from every 11 seconds in 2021.

Here are a few steps to ensure your cybersecurity remains strong:

1. Make sure your environment is hardened and protected.

Reduce your organization's vulnerability to attacks by hardening your platform. This step includes implementing basic data security controls and best practices, such as encryption in-flight and at rest, network segmentation, and the industry concept of '3-2-1', where you have three copies of your data in two locations and one offline. And by layering in immutability and zero trust, you can take your data protection approach even further.

2. Plan your ransomware response now.

Most organizations have disaster recovery plans in place, which are intended to recover systems and IT infrastructure following a single event like a natural disaster and enable the company to continue operating with minimal downtime. However, during a natural disaster, your environment isn't intentionally infiltrated as it would be during a cyberattack. Your backup is your last defense, and sadly, sustaining a ransomware attack is a "when" not an "if" scenario. Invest in a response and recovery plan and be sure to test it. You may also want to work with a cyber resiliency expert to develop and/or strengthen your cyber response and recovery strategy. 

3. Be proactive with ransomware.

Don't wait to be attacked; leverage solutions that use artificial intelligence and machine learning to identify activities that indicate ransomware. Examples include drastic increases in change rates, file types that no longer match extensions and increases in data randomization (also called entropy). 

Finally, consider developing the capability to provide an offline copy of your data in an isolated environment so there are recoverable backups in the event of an attack. Fortunately, new solutions have been developed and continue to emerge to meet cyber resiliency objectives. 

Challenge #2: App modernization and data management in multicloud  

Leveraging public, hybrid and multicloud solutions most effectively requires application modernization and radically new approaches to managing data.

4. If you haven't already, consider moving from a hybrid cloud to a multicloud strategy.

Organizations' approach to cloud has evolved from simply moving workloads to the public cloud and managing a hybrid cloud environment. Today, many organizations focus more on maximizing their options -- including multiple on-prem, private or public cloud options -- so they can easily move data and workloads and leverage the right provider or tools for specific purposes. 

This means they are not just gathering data from the edge or other sites but also making it accessible to take advantage of services that are cost-effective and deliver optimal outcomes. Consider solutions that connect to all major public clouds and provide direct access to your data to maximize your choices and can be best leveraged to your organization's benefit. After all, according to Forbes, the pandemic's widespread disruption "highlighted the value of having as agile and adaptable a cloud infrastructure as you can."

5. Make sure you closely manage consumption and storage costs in the cloud.

Many organizations continue to be surprised by the higher-than-expected and seemingly unmanageable costs of storage and consumption in the cloud. Customers engaging consultants to figure out what they are paying for has also become a common occurrence. The fact is, cloud is great for highly elastic and/or predictable data, but not for data that needs to last for years and not for unlimited consumption by users, which leads to multiple copies and use of expensive storage for low-value data. 

In addition, IT leaders often mistakenly expect that services or upgrades to solutions used by cloud providers will be automatically provided to them. Be sure you leverage the full value from your cloud provider by understanding what services you are entitled to in your agreement and ask for any additional support you need. 

And when possible, reduce your data footprint before storing it within a hyper-scaler. Once your data is there, leverage intelligent tiering that aligns with the lowest cost and desired SLAs for your data.

6. Understand the shared responsibility model as it applies to your data in the cloud and know what data is protected in the cloud and to what level. 

IT professionals often assume their data in the cloud is protected from ransomware or other attacks but later find out that protection is not included in the agreement. Whether you are consuming infrastructure, platforms or Software as a Service (SaaS), you are responsible for your data in the shared responsibility model. Protecting your company's data from accidents, negligence or ransomware is never the responsibility of your cloud vendor. 

While some SaaS vendors have tools to help you protect your data, they're primarily responsible for the availability and durability of their offerings for all customers. This places their focus on the security of their systems and applications, leaving it up to you to safeguard your data from cyber threats.

Challenge #3: Managing exponential data growth

IT leaders are struggling to deal with a massive explosion of unstructured data — data that doesn't reside in traditional databases. Instead, much of it lands on file systems as files or object systems as objects. Increasingly, unstructured data includes streaming data, which requires extensive analysis and interpretation in real time or near real time to turn it into actionable insights. It includes data like genome sequences, movies and IoT sensor data from devices and vehicles. Lately, it also includes more video surveillance and data from thermal imaging sensors, which can be used to identify people with fevers moving through spaces.

What can you do?

7. Understand what you have now.

The first thing to do is to gain an in-depth, data-driven understanding of what you currently have. This accomplishes several things: First, it gives you a starting point to make changes and measure progress as you prepare for the future. Second, it enables you to identify gaps and opportunities for improvement. And finally, it helps you begin to rethink your strategy and approach to cloud, your data and your data center. 

Additionally, you should consider data governance and compliance requirements. This addresses how long the data should be retained and where it needs to reside. Excluding regulations, how a business handles data should be aligned with the value that data provides back to the business and its customers.  

8. Consider how unstructured and nontraditional data could help drive growth and innovation and how it could impact your current strategies, approaches and your organization's future plans.

New types of data represent a significant challenge but also an incredible opportunity to enable your users and organization to make better decisions, create new revenue streams, improve the client experience and drive growth. Consider meeting with different lines of business, marketing, finance or other areas to discuss what types of information or insights they would love to have but currently lack. 

9. Educate yourself on new technologies and determine how they could work with, complement or replace what you have.

As you continue to address this unfolding trend, developing an understanding of emerging technologies and new approaches and then testing the solutions can also inspire new ideas. For example, many vendors can approach large unstructured data growth in a more flexible way by introducing their technologies within the hyper-scalers, providing operational ease of use and seamless transition from managing infrastructure onsite to the same platform within a hyper-scaler's environment.

Challenge #4: Continuously track cost and time to value for your storage and data center investments

Your technology decisions have significant costs and impacts to your organization. It is crucial to continuously measure, monitor and communicate the cost and ROI of your investments and the vision those investments support back to the business stakeholders. Close alignment between business and technology teams creates an effective and efficient partnership, which helps identify deliverables and accelerate ROI. 

10. Plan ahead to maximize the utilization of your investments.

When investing in new technology, remember that the value and cost of your solution vary by utilization. The cost per terabyte of a storage array while not fully deployed is an order of magnitude higher than when it is fully populated. The most expensive terabytes on an array of storage are the ones you're paying for when you're migrating data in and out at the beginning and end of its lifecycle. By planning ahead and having a clean data migration strategy, you can accelerate the impact of your investments and optimize costs and asset usage. 

Most importantly in these complex times, consider your vendor's supply chain process and how disruption may impact your investment and business goals.