Memory Population Guidelines for Intel 3rd Gen Xeon Scalable Processors

Intel's 3rd Generation Xeon Scalable processor launched on April 6th, 2021, and their all-new architecture brings many advantages to the table. As new configurations and approaches are required here, we are taking this opportunity to dive deeper into each of these changes to ensure our customers maximize their investments. This article will focus on the new memory controller and proper DIMM population to ensure balanced configurations and optimal performance. Before we look at the new 3rd Generation Xeon Scalable processors, let's review how the 2nd Generation Xeon Scalable processors handled memory.

The 2nd Gen Intel Xeon Scalable Processor memory architecture 

Intel's 2nd Generation Xeon Scalable processors launched with two memory controllers that managed three memory channels per controller. Each channel could accept 2 DIMMs in the channel. To calculate the total DIMMs that a processor could support, multiply the number of controllers by the number of channels by the number of DIMMs per channel. For the 2nd Generation Xeon, this equaled a total of 12 DIMMs that a single processor could address.

This memory controller ran at DDR4-2933 speeds with 1 DIMM per channel populated. The speed reduced to DDR4-2666 when populated with 2 DIMMs per channel or Optane Persistent Memory Modules. The optimal configuration for maximizing memory bandwidth populated the same number of identical modules per channel. A two-socket server featured up to 24 DIMMs, allowing for two maximum performance configurations:

As this table illustrates, effectively utilizing six channels made for some interesting total memory numbers. Configurations with 2 DIMMs per channel will see a slight increase in effective bandwidth due to interleaving. (Interleaving is where the CPU balances requests between the memory modules on a channel to reduce the impact of read and write latencies.) It should be noted that servers featuring 16 DIMMs exist for use cases requiring a total of 1TB of memory, but effective memory bandwidth was reduced by 34 percent due to an unbalanced configuration.

The 3rd Gen Intel Xeon Scalable Processors memory architecture 

The Third Generation Xeon Scalable processors feature an all-new memory controller design. Each processor now has four memory controllers with two channels per controller. Each channel can be populated with two DIMMs. A 3rd Gen Xeon can address a total of 16 DIMMs per processor at full performance.

Each of the eight channels runs at DDR4-3200. This is regardless of the number of DIMMs per channel or if 200-series Optane Persistent Memory is populated. Improvements have also been made to the memory scheduler, lowering effective memory latency while increasing the overall bandwidth.

This new design results in a 2.6x increase in overall memory capacity and 1.5x increase in memory compared to 2nd Generation Xeon Scalable, without increasing read latencies at the local or remote socket layer. A two-socket server features up to 32 DIMMs and we still have two balanced memory configurations to choose from for maximum performance:

The Intel 3rd Generation Xeon Scalable allows for larger capacity memory configurations without sacrificing memory bandwidth. As with previous generations, some basic considerations should be followed when planning server memory:

• Memory modules must have identical specifications. Each DIMM must have an identical size, speed and rank.
• Channels are identically populated with either one or two DIMMs.
• Each CPU socket in the server must have an identical memory layout.

A security-focused architecture

That said, increased performance is only part of the story here. Regardless of the use case, security is a paramount concern that is top of mind for every IT organization today. Intel has seized the opportunity provided by this new design to embed advanced security features right into the CPU. Total Memory Encryption (TME) encrypts the entire system memory for added protection against physical attacks, and Software Guard Extensions (SGX), which provides fine-grained data protections via application isolations in memory, are included in all 3rd Generation Xeon SKUs. These new features will be covered in-depth in a future article, but it is important to note that SGX requires a balanced memory configuration.

Experience 3rd Gen Intel Xeon Scalable Processors in the ATC

Servers have an increasing demand to process more data, where it is created, in reduced timeframes. This demand drives the need for large amounts of high-performance DRAM and persistent memory directly attached to the processor. Intel 3rd Generation Xeon Scalable processors are built to accommodate demanding use cases without sacrificing performance or security. 

These processors are available now in DellEMC PowerEdge 15G, HPE ProLiant Gen10 Plus and Cisco UCS M6-series servers, with additional product lines being updated soon. WWT is pleased to demonstrate this technology in our ATC and we can accommodate virtually any use case or workload. We invite you to see them in action in the ATC or attend a dedicated workshop to learn more.