Bandwidth Bottleneck at the I/O Panel
Processor and SerDes speeds continue to bump up, putting pressure on input/output connectors mounted on the faceplate. Like investors running to the exit when the market tanks, the I/O panel can quickly become a serious bottleneck that limits communication and slows the system. Computers and their users hate to sit idle while waiting for required data.
One solution has been the ongoing progression of pluggable I/O connectors, including SFP, QSFP, Micro QSFP, QSFP-DD, and OSFP. Each iteration delivered greater data rates than the last while consuming less panel space. The new OSFP form factor can provide up to 32 400Gb/s ports per 1RU line card to enable 12.8Tb/s per switch slot. The smaller profile of the QSFP double- density module allows mounting 36 connectors for a total throughput of 14.4Tb/s. These high-density connectors have enabled the industry to stay one step ahead of immediate demands, but next-generation servers may be looking for the ability to move 15+ Tb of data from a 1RU panel. Pushing that much data through multiple high-density cable assemblies creates at least two challenges.
Mounting all of these active copper or optical connectors in such close proximity can draw a lot of power and that results in significant heat build-up. In a fully configured panel, connectors are side stacked and even mounted belly-to-belly, which makes it difficult to ensure that cooling airflow is able to circulate adequately around each module cage. Adding cooling fins (fans) to the cage can reduce the problem, but can also limit faceplate connector density. Each specification has engineered their interface to minimize power consumption, but the challenge remains. System designers must carefully balance their priorities of total data throughput, panel density, power consumption, cooling strategy, reach and, of course, cost.
Secondly, as data rates continue to escalate, the gauge of the copper conductors in I/O cables must also increase. Longer cables also require larger conductors to minimize attenuation. Trying to terminate shielded cables with conductors as large as AWG 24 can create a manufacturing problem. Large conductors make the cable stiff and bulky; not desirable attributes when dealing with hundreds of cascading cable assemblies. The adoption of active connector technology such as “Gauge Changer™” equalization chips from Spectra7, which allow the use of conductors as small as 36 gauge, was evident at the recent DesignCon 2018 Expo. The use of active optical cables is another solution to shrink the size of external cable assemblies.
An alternative is the use of optical transceivers located on the PCB. They offer several significant advantages, including:
The transceiver module can be located immediately adjacent to a processor or SerDes, minimizing copper trace losses and distortion. Moving these high-speed signals off the PCB simplifies board design, improves signal integrity, and potentially minimizes the need for more expensive laminate materials.
Secondly, once converted to optical, signals are immune from EMI generated inside the box. The optical fiber can be terminated to a bulkhead via MT, MTP, or MXC type bulkhead connectors. The reduced profile of these connectors allows many more connectors to be mounted on the faceplate, enabling much higher total throughput per square mm.
Mounting the optical transceiver away from the PCB edge minimizes the concentration of heat-generating devices at the faceplate, reducing the thermal load in that area.
Samtec was one of the first connector manufacturers to develop optical transceiver modules for use within the box. Their FireFly Micro Flyover System offers options for both copper and fiber transceivers.
Several years ago, FCI Electronics (now absorbed into Amphenol ICC), Molex, and TE Connectivity introduced competitive 12 X 25Gb full duplex mid-board optical transceivers. Within about a year, both Molex and TE Connectivity quietly withdrew their transceivers, citing a lack of market and/or existence of governing standards.
Today, the Amphenol ICC Leap On-Board transceiver, along with the Finisar 25G Board-Mount Optical Assembly and Luxtera 8 X 26Gb transceiver, remain the few devices that currently participate (participating) in this market.
Yet, on-board optical transceivers remain the target of new development efforts. DustPhotonics, an Israeli startup, is expected to introduce an 8 X 56 Gb on-board transceiver terminating to a QSFP-DD connector at the OFC 2018 conference.
Each of these devices are proprietary designs, which remain a concern in the industry.
The issue of an industry standard for at least one segment of this market is being addressed by the Consortium for On Board Optics (COBO). This organization announced a specification for a mid-board optical module and related connectors at the OFC conference. The intent is to provide a protocol-agnostic open standard form factor and pluggable interface to bring order to a market that will likely grow as expansion of the cloud continues. Although capable of supporting any optical transceiver technology, from short-reach client optics to long-haul coherent optics, COBO appears to more focused on longer-reach applications.
On-board optical transceivers may be the answer when OEMs need to move to the next level of Tb per square mm of faceplate. In addition to offering significant advantages of signal density, power consumption per channel can be significantly reduced.
It is unclear at this point how new optical transceivers defined by the COBO standards will impact the use of existing long-haul pluggable alternatives such as CDFP and CFP8. With 400Gb ports being the prime target for next-generation equipment, the race is on for the industry to select the winning I/O technology of choice.