Pluggable I/O: The March to 400 Gb, 800 Gb Ethernet, and Beyond

By Robert Hult | May 18, 2021

Enhanced versions of pluggable I/O connectors are providing a clear migration path from current 40 and 100 Gb/s Ethernet and InfiniBand EDR specifications to 800 Gb and beyond.

A spectacular escalation of data rates has been a hallmark of computing equipment over the past 50+ years. The ability to create literally billions of transistors on a single chip enabled exponential growth of speed, processing power, and reduced cost per bit. Entering data and delivering results is the job of the input/output (I/O) connector. I/O connectors play a critical role in system performance and can create a serious bottleneck if they don’t keep up with the flow of data. Engineers must ensure the I/O port does not constrain that performance.

In addition to being capable of supporting each boost of data rates, components need to provide design flexibility to quickly reconfigure or upgrade a system. The ability to switch media from different lengths of passive and active copper cable as well as optical fiber is highly attractive.

I/O panel density is another key factor in system design. Standard rack-mounted equipment as small as 1RU (1.75” high) demands that I/O connectors consume as little space as possible to maximize the number of channels as well as provide space for cooling vents. The ability to unplug and reinstall an interface without shutting down the system (hot pluggability) is particularly important in network applications.

Standardization that encourages intermateability among multiple sources is assumed. Pluggable small form factor I/O connectors in standardized configurations offer a cost-effective solution to each of these issues.

Pluggable I/O interfaces including small form factor pluggable (SFP) and quad small form factor pluggable (QSFP) have experienced an ongoing progression of iterations with upgraded performance and panel density. They consist of a PCB mounted cage assembly with a high-speed, hot-pluggable connector located at the rear that mates with the incoming module. This modular concept enables engineers to interchangeably select direct attach copper cables, active optical cables, as well as optical transceivers. The cage provides mechanical guidance for the mating module as well as thermal dissipation and RF isolation for the module.

pluggable I/O connector form factors

These connectors have rapidly evolved into an alphabet soup of small form factor pluggable modules, from the original SFP to the most current QSFP Double Density and OSFP configurations.

The first electrical and mechanical specifications of the SFP interface was released by the Small Form Factor (SFF) Committee in 2001 and promoted by a multi-source agreement (MSA) consisting of industry users and connector manufacturers. It was designed to support network applications in SONET, Gigabit Ethernet, and Fibre Channel. This hot pluggable module enabled data rates of up to 1.0 Gb/s in copper and fiber media. The original SFP specification was updated to SFP+ with a bandwidth of 10 Gb/s while maintaining total backward compatibility. Subsequent upgrades bumped the bandwidth to 28 Gb/s. The most recent version of the SFP form factor is the SFP56 optical transceiver, which utilizes PAM4 modulation to provide 50 Gb/s Ethernet connectivity.

The QSFP connector goes one step further. Rather than providing a single channel, QSFP modules provide four high-speed copper or optic channels in a package somewhat larger than SFP. Stacked and ganged cages increase port density even further.

QSFP was originally rated at 1 Gb/s per channel but was also upgraded to QSFP+ with four channels at 10 Gb/s, each providing an aggregate bandwidth of 40 Gb/s. Fine-tuning of the internal signal path length, EMI cage design and PCB footprint layout pushed QSFP28 lane rates to as high as four channels at 25 Gb/s each. The QSFP56 iteration provides four channels at 50 Gb/s. The aggregated bandwidth of the four channels enable support of 40, 100, and 200 Gb Ethernet.

The next doubling of Ethernet bandwidth was documented with the release of the 400 GbE specification in 2017 with two new pluggable interfaces currently vying for leadership in supporting 200- and 400-Gb Ethernet applications.

This enables an eight-lane electrical interface with each channel operating at 25 Gb/s NRZ or 50 Gb/s PAM4 modulation. Each module can provide 200 or 400 Gb/s aggregated bandwidth per port. Up to 36 QSFP-DD connectors can be mounted on a standard 1RU switch panel to provide aggregate capacity of 14.4 Tb/s.

Molex’s QSFF-DD Interconnect

Molex’s QSFF-DD Interconnect System’s 8-lane electrical interface transmits up to 28 Gb/s NRZ or 56 Gb/s PAM-4, up to 200 or 400 Gb/s aggregate, with the same module form factor as QSFP Interconnects, making them backward compatible.

New QSFP-DD cages are backward compatible with all prior QSFP modules. Modules draw a maximum of 12 watts.

Heat sinks applied to the exterior of the cage are required to support optical modules.

QSFP-DD cables can reach from three meters of passive copper cables to 10 km over duplex single-mode fiber.

Octal small form factor pluggable (OSFP) connectors are the newest contender for high-speed applications in next-generation data centers.

Amphenol ICC OSFP connectors

Amphenol ICC’s OSFP connectors are compatible with both 25 Gb/s lane channel NRZ and 50G/lane channel PAM4 signaling protocols that allow the cables to deliver aggregate bandwidths of 200G and 400G per cable assembly.

They are similar in performance and profile to the QSFP-DD connectors but with a few differences. OSFP connectors are slightly wider and deeper than the QSFP profile. Up to 32 OSFP ports can be mounted on a line card. OSFP optic modules can support up to 15 watts, as opposed to 12 watts for QSFP-DD. OSFP modules feature integrated heat sinks on the module. Both interfaces provide eight channels at up to 50 Gb/s PAM4.

Several new interface specifications have sprung up to facilitate coherent optical 400 Gb transmission. The 400 ZR specification developed by the Optical Interoperability Forum (OIF) and released in 2020 defines a point-to-point 400 G link up to 120 km in length. The OpenZR+ MSA offers additional flexibility with 100G, 200G, 300G, and 400G optical line rates with reach of up to 480 km. Both specifications specify QSFP-DD or OSFP pluggable transceiver modules.

Inphi COLORZ II pluggable DCO transceiver

Inphi’s COLORZ II is a 400ZR QSFP-DD pluggable DCO transceiver for data center interconnects.

One would expect that 400 Gb channels would provide plenty of bandwidth headroom to support network requirements, but recent projections by the Ethernet Alliance indicated that demand will outstrip capacity within the next five years. As a result, IEEE has initiated an Ethernet study group to identify technical feasibility, market potential, and timing for the next iteration of Ethernet in applications that range from chip to chip to 40 km. Both 800 GbE and 1.6 TbE are under consideration. Lively debates among players including Google, Facebook, Microsoft, Intel, Cisco, and Broadcom are centering around the issues of modulation (PAM 4, 6, 8), number of and maximum bandwidth per lane, degree of error correction required, as well as power/thermal management.

Forecasts indicate that 51.2 Tb switches will begin to reach the market in 2022 which will require 800G optical interfaces.

A new optical 800G pluggable MSA is focused on data center connectivity to 2 km and will likely utilize PAM4-based signaling to create 8 X 100 Gb and 4 X 200 Gb optical channels using QSFP112-DD or OSFP32 pluggable form factors.

System designers are being challenged to balance the need for increased bandwidth, acceptable signal integrity, reduced power, thermal management, increased I/O panel density with flexible options of reach and, of course, cost. The utilization of coherent and optical dense wave division multiplexing (DWDM) transmission technology will enable this revolution.

We expect pluggable small form factor connectors to experience exceptional growth due to insatiable demand for higher bandwidth. Explosive growth of mobile electronic products, continuing adoption of cloud services, the IoT, Industry 4.0, AI, and emerging 5G networks will require constant upgrades of data center infrastructure.

Further into the future, emerging technologies such as co-packaged optics may represent the next step in high-speed data transmission. OIF recently announced the formation of a new project to investigate the use of co-packaged optics in intra-data center switching applications.

Enhanced versions of pluggable I/O connectors are providing a clear migration path from current 40 and 100 Gb/s Ethernet and InfiniBand EDR specifications to 800 Gb and beyond.

Like this article? Check out our other Connection Basics, I/O Connectors and Data Centers articles, our Datacom/Telecom Market Page, and our 2021 and 2020 Article Archives.

Robert Hult
Get the Latest News