What good is going fast if you can’t get past the next rack?
In modern AI systems, using PCIe to stitch together accelerators is already too slow. Nvidia and AMD use specialized interconnects like NVLink and Infinity Fabric for this reason – but at the 900-plus GB/sec these links push, copper will only carry you so far.
According to Manish Mehta, VP of marketing and operations for Broadcom’s optical systems division, in high-end clusters, copper can only carry you about three to five meters before the signal starts to break down. And as serializer deserializers (SerDes) push beyond 200Gbit/sec, copper’s reach is only going to get shorter.
The answer, as you might expect, is to ditch copper in favor of optics – even though that comes at the expense of increased power consumption. Nvidia has estimated that using optics rather than copper for its NVL72 systems would have required another 20 kilowatts per rack – on top of the 120 kilowatts for which they’re already rated.
While the individual transceivers don’t pull that much power – just 13 to 15 watts each, according to Mehta – that adds up pretty quickly when you’re talking about multiple switches with 64 or 128 ports each. “If there is a need for scale-up to progress to higher reach and therefore optical connectivity, you’re going to need 10x of that bandwidth, and that will just be unachievable with this type of paradigm”, he explained during a speech at the Hot Chips conference this week.
Instead, Broadcom is now experimenting with co-packaging the optics directly into the GPUs themselves.
Co-packaged optics (CPO) is something Broadcom has explored for several years.
