400G-ZR & ZR+: The Latest in Pluggable Coherent DWDM

With any networking device, product developers are always finding ways to make them faster, smaller and less expensive. This is abundantly clear in high-end routing platforms with systems exceeding 10Tbps in a single rack unit. 

Coherent DWDM products are poised to keep pace with the introduction of 400G coherent DWDM pluggable optics. It's been quite a journey thus far. 

In this article we will take a look at the evolution of coherent DWDM, a closer look at the details behind 400G QSFP-DD coherent DWDM pluggables and touch on how this can lead to the convergence of DWDM and routing technology.

Coherent DWDM, from then to now

The industry has come a long way in less than 10 years, with optics becoming smaller and faster. Figure 1 illustrates coherent DWDM module size evolution. As for speed, within the same time span it has increased 10-fold — from 40G in 2011 to 400G today, with 800G pluggables in the not so distant future.

One of the most significant innovations in DWDM system development was the introduction of coherent optical technology. At a very high level, coherent optical devices utilize advanced optics and digital signal processors (DSP) to transmit and receive complex light wave modulations, enabling high-speed data transmission. Coherent modulation continues to be the driving force behind high speed optical devices, including 400G and beyond. 

The first commercially available coherent DWDM systems were 40G followed by 100G soon after; the systems were line card and chassis-based, capable of supporting many line cards in each system. A major advancement in the same footprint as a 10G wave, we could now transport 100G waves and at longer distances to boot. Over time, line card speeds have increased to 200G and beyond, but the industry was approaching an inflection point as cloud providers began to emerge.

As cloud provider networks began to grow exponentially, the pressure for manufacturers to create even smaller, faster and cheaper network components only increased. This was that inflection point prompting the creation of the "pizza box" DWDM system.

The "pizza box" system did away with chassis and line cards. It's a physically small, self-contained system — picture a small data center switch, 1 or 2 RU (1.5"-3") in height. The engineering key to the pizza box form factor's feasibility was to decouple the two major components in coherent optical transmission: the optics (lasers, receivers, modulators, etc.) and the DSP (digital signal processor), which up until now both resided in a large module mounted on a line card.

Innovations on the optics side led to smaller components with lower power requirements. These innovations produced the pluggable CFP2-ACO (analog coherent optics), a pluggable DWDM module in the relatively small CFP2 form factor. DSP technology also evolved enabling one DSP chip to support multiple CFP2-ACO modules. 

By locating multiple DSPs within the "pizza box" where they serve multiple CFP2-ACOs, manufacturers like Cisco produced systems such as the NCS 1002, capable of transporting 2Tbps (20x 100G client connections) within 2 rack units (3 inches). In contrast, a chassis-based system would require 12 rack units. Along with space savings, they are much more power efficient as well. Learn more about Cisco's NCS 1000 platform.

You might be wondering, why is it called "analog" and asking, "aren't these systems digital, 1s and zeros??" That's the brilliance of coherent technology, it modulates those 1s and 0s into analog waves, enabling more data to be packed into each wave and then decoded accurately at the other end.    

Granted, that is a very simplistic explanation of coherent signal transmission but for our purposes, the point is that we need to convert digital to analog to transmit data and convert the analog back to digital at the other end. The CFP2-ACO is only equipped to handle the analog signal. It receives coherent analog signals from the DSP to be transmitted, or it passes a received coherent analog signal to the DSP to be converted to digital. Figure 2 illustrates this concept.

CFP2-ACO systems were, and still are, game changing in terms of shrinking the footprint, reducing power consumption and reducing the cost of optical networking equipment, specifically the transponder. These platforms have been widely adopted throughout the industry and have become the standard form of optical transport in almost every cloud provider's network.

Since the introduction of CFP2-ACO based systems vendors have introduced new, faster "pizza box" systems that do not rely on DWDM pluggables. The optics and the DSP reside on small field replaceable modules or mini line cards. These systems can support 600Gbps+ per wavelength; for example, the Cisco NCS 1004 can support 4.8Tbps in 2 rack units. Learn more about the Cisco NCS 1004 platform.

During the same time, advancements in pluggable coherent DWDM optics continued with the introduction of the CFP2-DCO. Notice something different in the name? That's right, ACO is now DCO. So what does that mean?

Well, the "D" stands for digital in digital coherent optics. Coherent optic developers like Acacia Communications have once again shrunk component size and power draw so that the optics and the DSP now reside within the CFP2. Learn more about Acacia's DCO products.

This eliminates the need for a chassis to house the DSP, enabling coherent DWDM transport directly from a router or switch (see Figure 3). This is the tipping point where a true convergence of DWDM and routing becomes reality.

Now this brings us to 400G-ZR and 400G-ZR+ QSFP-DD: the same technology as the CFP2-DCO but in the much smaller QSFP-DD form factor. There are other form factors, but as we discussed in my previous article, 400G Optics: Enabling Network Scale and Device Consolidation, the industry has widely settled on QSFP-DD. The availability of 400G coherent DWDM optics in such a compact package truly makes a viable case for the convergence of routing and DWDM.

400ZR, 400G-ZR+, 400G-OpenROADM, 400G-OpenZR+... what's the deal?

If you have read anything about 400G DWDM in a pluggable form factor, you have most likely come across one of these names and it may seem a bit confusing. The various names stemmed from multiple standards bodies going in slightly different directions. 

First was the Optical Internetworking Forum (OIF) who created the 400ZR standard. 400ZR is targeted towards edge and relatively short reach, up to 120km DCI applications. Around the same time, the OpenROADM Multi-Source Agreement (MSA) also defined a specification for a 400G DWDM pluggable. Their specification focused on what service provider networks would need, such as long optical reach (>120km), advanced forward error correction (known as oFEC) and selectable data rates (100G, 200G, 300G, or 400G). Though the additional capabilities were achievable, it would require more power than the 15W specified for ZR. Therefore, OpenROADM's specification became known as ZR+.

Ultimately, between the two organizations and the various optics manufacturers, they agreed to take the best of the OIF and OpenROADM standards, combine them and call it OpenZR+. By combing the features of each in the same form factor, it leaves us with one highly versatile coherent DWDM optic, as illustrated in Figure 4. 

With form factor, features and data rates sorted out we are left with optical reach, or how far apart your two locations can be. I will use a classic system engineer response: "that depends." Like any coherent DWDM module, it depends — the higher the data rate, the shorter the reach. That said, with the OpenZR+ standard the reach is quite impressive at 1400 km, more than 10x 400ZR's reach. Figure 5 shows data rate, modulation type and reach. Keep in mind that there may be specific vendor variances, but this is a reliable estimate based on vendor and standards data.

Considering the capabilities of 400G-OpenZR+ combined with high-density routing platforms, it makes a very strong case for the convergence of DWDM and routing. The most obvious application is data center interconnect (DCI), but there is a much bigger picture here. 

With high-density DWDM in the router, combined with the traffic engineering simplicity and path redundancy of Segment Routing, we are poised for a significant shift in transport network architecture.  Learn more about the benefits of Segment Routing. 

Conclusion

I hope this provided relevant insight into the development of 400G pluggable DWDM technology and illustrated some of the key benefits. Please stay tuned for my next article, where we will dive into converged architectures and how 400G-OpenZR+ combined with high density routers creates robust, carrier grade transport networks. 

Feel free to contact us today with any questions or to discuss your specific use case.