High-Speed Solutions: Are Co-Packaged Optics Moving Toward Reality?
In high-speed data transmission, copper circuits introduce signal loss and distortion that increase with the length of the circuit. However, rapid advances in silicon photonics and a new generation of co-packaged optics are enabling designers to mount dissimilar chips directly on a common substrate, saving power and expanding bandwidth.
The pace of progress in high-speed data transmission has been fast and furious. Design engineers are determined to find ways to go beyond the apparent bandwidth limits of traditional copper interconnects while reducing power consumption. Evolving data-intense applications including artificial intelligence, machine learning, high-definition video, and future autonomous transportation are putting intense pressure on data center infrastructures to stay ahead of increasing demand. Billions of dollars are being invested in research as well as acquisition of companies with related resources.
High-performance switch and router silicone chips are at the heart of today’s hyperscale data centers. As serializer/deserializer (SerDes) speeds pushed from 28 Gb/s to 56 Gb/s and now 112 Gb/s, switching capacity increased dramatically. In March of 2023, Broadcom became the first to announce production availability of 51.2 Tb/s switch silicon.
Current switch packaging technology utilizes pluggable optical transceivers, a technology which has experienced near-exponential performance improvements over the past 20 years. The most advanced modules, including CFP2, QSFP-DD, and OSFP, can deliver up to 800 Gb/s data rates. Modules offer ease of upgrade and replacement, automated network management, multiple modes, selectable operating bands, and are available from a supply base of global manufacturers.
Modules mate with receptacles mounted on the faceplate of a switch or router.
The rich selection of pluggable form factor modules now on the market has given designers more I/O options and flexibility, both copper and fiber optic. New applications constantly push the limits of this technology.
How electrical high-speed signals are managed inside the box has become the new focus of attention.
The pluggable optical transceiver converts incoming optical signals into electrical pulses. Those signals are connected to copper traces imbedded in the switch motherboard and delivered to the switch ASIC. The problem with this arrangement is that as data rates increase, copper circuits introduce signal loss and distortion that increase with the length of the circuit.
To compensate for this loss, more power is required to drive the signal. When hundreds of signals are required, the extra power demand adds up to unacceptable levels.
The other problem is that the space available on a 1RU faceplate is limited and future switches may not have enough room to mount the required number of pluggable transceivers. Designers could choose a 2RU envelope, but that would limit the number of devices that can be mounted in a rack.
Co-Packaged Optics (CPO) is being proposed as a long-term solution to this problem. There are several interim steps between what is being done now and the ultimate form of CPO packaging, including on-board optics and near-package optics, but rapid advances in silicon photonics are enabling the ability to mount dissimilar chips directly on a common substrate.
Rather than perform the electro-optic conversion at the front panel, fiber is brought directly to a substrate that has a series of “chicklet” optical modules arranged around the switch ASIC.
Extremely short electrical paths connect between the optical module and the switch.
By packaging optical engines on the same substrate as the switch, the electrical signal path is reduced to short inter-package lengths, minimizing distortion and attenuation due to material loss as well as power demand.
Electrical signals from the switch ASIC are converted to optic signals by the photo integrated circuits (PICs) and coupled to optical fiber. These fibers are terminated to high-density optical connectors mounted on the I/O faceplate.
A significant issue has revolved around where the laser that drives the optical engines is located. One concept locates the laser on the substrate along with the optical engines which makes the module a self-contained assembly. A single laser could support multiple chicklets, or each chicklet could have its own laser. Redundant socketed lasers that are field-replaceable could allow the module to continue operating if one fails. Both concepts could generate thermal issues with high-power lasers mounted in close proximity to sensitive optical engines. Liquid cooling has been considered in this configuration.
An alternative approach is the use of an off-module laser source. An externally mounted laser could drive multiple on-board optical engines but adds connector reliability issues.
The Optical Internetworking Forum (OIF) has become a promoter of CPO technology and is in the process of actively creating much needed standards. In February 2022, OIF released an initial Co-Packaging Framework Implementation Agreement which had the objectives of identifying key CPO applications and defining their requirements which enabled the creation of interoperability standards. Issues addressed in this document include socketing, thermal management, laser type, laser safety, chip packaging options, electrical and optical budgets, reparability, and manufacturability. A list of potential applications was also included. The document introduced a proposed 3.2T module designed for 51.2+ Tb switch applications.
A unique aspect of this proposal is the recommendation of a pluggable external laser source that mates on the faceplate. The External Laser Small Form-factor Pluggable (ELSFP) is similar to a conventional small form factor pluggable. Locating the lasers external to the box allows resulting heat to be managed more efficiently, while offering a quick and simple method of hot swapping a failed laser array. Creating a common external laser module also permits flexibility to adapt to future high-performance iterations. The external laser source also addresses potential eye hazards.
This concept of a standardized optical module chicklet and external optical module offer a manufacturable and scalable solution to the design of co-packaged optical modules.
In March 2023, OIF announced the 1.0 standard for a 3.2 Tb module which defines the technical specifications for a 3.2 Tb/s Co-package Optical transceiver module and is intended to serve as a building block to enable reduced power consumption solutions for 51.2 Tb/s switches, with 16 modules arranged near the switch ASIC.
On August 8, 2023, OIF announced the External Laser Small Form-Factor Pluggable (ELSFP) Implementation Agreement which represents another major step forward in supporting CPO technology.
Broad industry adoption of CPO technology is far from a done deal. Implementation would be a major departure from existing system packaging technology. Questions about reliability of the ELSEP and recovery if this laser source were to fail are still unresolved. Should the transceiver modules be socketed, and does that intruduce another failure mode? Additional questions remain regarding the manufacturability of CPO architecture. Will CPO technology suffer implementation delay as engineers find less costly ways to support 51.2 and even 102.4 Tb switches using conventional pluggable transceivers?
Pluggable optical transceivers continue to evolve. A specification defining the OSFP-XP pluggable was released in March 2023 and is specifically targeting 51.2 Tb switches. Linear direct pluggable transceivers are gaining attention with the promise of reduced power, latency, and cost.
Given the immense investment being made in silicon photonics and expectation that switch speeds will continue to be driven by the demands of artificial intelligence and machine learning, we expect CPO will eventually become the most cost-efficient packaging technology in next-generation systems. That will happen only when traditional alternatives can no longer economically provide the required level of performance.
Like this article? Check out our other articles on Cable, and High Speed, our Wire and Cable Assemblies Market Page, and our 2023 Article Archive.
Subscribe to our weekly e-newsletters, follow us on LinkedIn, Twitter, and Facebook, and check out our eBook archives for more applicable, expert-informed connectivity content.
- Optics Outpace Copper at OFC 2024 - April 16, 2024
- Digital Lighting Enhances your Theatrical Experience - March 5, 2024
- DesignCon 2024 in Review - February 13, 2024
