Dell PowerMax Performance During Upgrade

In the WWT Advanced Technology Center (ATC), we needed to do a code upgrade on a Dell PowerMax 2000 All-Flash Array. We decided to do some performance tests against the array during this code upgrade. Dell Support performed the upgrade while an ATC Lab Services storage architect checked the performance of the array during the upgrade process. We wanted to determine if any impact occurred to the array during a routine code upgrade. We had several customers who were interested in the outcome of this test for their own upgrade processes in their production environments.

ATC Insight

The goal of this document is to show the impact a code upgrade does on the performance of a Dell PowerMax 2000 dual-engine storage array. The code upgrade was conducted by Dell Support after a thorough health check was completed before the upgrade began. The code upgrade includes many fixes and minor application-level upgrades.

Code upgrade details:

PowerMax OS 5978 in a high availability configuration
PowerMax OS level before upgrade: 5978_221 Epack 7624
PowerMax OS level after upgrade: 5978_479 Epack 8870

Baseline Performance Testing during the upgrade process:

Performance test ran over the entire duration of the upgrade and was configured to run at 75K IOPS with 8K block sizes and 70% read.

Hardware and software consisted of the following components:

Cisco UCS 9706 Fiber Channel switches
Dell PowerMax 2000 dual-engine All-Flash Array
Vdbench cluster consisting of 8 virtual worker nodes each with 8 LUNS masked from the Dell Powermax array

Important time stamps during the upgrade (central time):

9:28am MMCS(Management Module Control Stations) re-image began with an estimated 45min for competition time
10:08am MMCS re-image completed successfully
10:30am code upgrade began
11:49am code upgrade completed

Performance testing results during the upgrade:

9:28am when the MMCS re-image began:

10:08am when MMCS re-image completed:

10:30am when code upgrade began:

Summary just before upgrade started until roughly 90 minutes after it completed at 11:49am

Upgrade Plan

Contact Dell support via support case to request a health check and upgrade date.
Work will support to schedule the upgrade and the health that needs to take place before the upgrade occurs.
Dell support will confirm remote access is working and complete health check on scheduled date.
Once health check passes the upgrade will commence on scheduled date
After upgrade completes a final check of the array will occur to confirm health of the array is "green"

Hardware

Dell PowerMax 2000 dual engine All Flash Array

Test Tools

VDBench

VDBench is an I/O workload generator for measuring storage performance and verifying the data integrity of direct-attached and network-connected storage. The software is known to run on several operating platforms. It is an open-source tool from Oracle. Visit VDBench Wiki for more information.

Graphite/Grafana

A Graphical User Interface (or GUI) that we use in the Advanced Technology Center (or ATC) to visually depict the results data that we derive in our compute and storage lab efforts. Visit Grafana for more information.

Final Thoughts

Here are my thoughts around the performance of the PowerMax 2000 All-Flash Array during an upgrade procedure: The PowerMax before, during, and after the upgrade continued to perform well at 75K IOPS with sub-millisecond latency throughout the upgrade process. The only impact in latency occurred when the code upgrade began. Throughput during the upgrade process continued as normal, therefore application impact would not be affected.