641GB of total memory ought to be enough for anybody (and their LLM)
Less than three months after Nvidia’s Grace Hopper superchips went into full production, CEO and leather jacket aficionado Jensen Huang this week took to the stage at SIGGRAPH 2023 to unveil an even more capable version of the silicon.
The forthcoming processor shares the same basic DNA as the GH200 – the final form of the Grace Hopper superchip Nvidia announced in early 2022 and showed off at Computex this spring. The device’s 72-core Arm Neoverse V2 Grace CPU, its Hopper GPU, and its 900GB/sec NVLink-C2C interconnect all remain unchanged. And, much to our initial confusion, the new chip shares the same GH200 moniker.
Nvidia tells us this is because they’re different configurations of the same part – so it’s not unlike the 40GB and 80GB versions of the A100 from a few years back. In fact, the memory load out is the core difference here, at least for now. Instead of 96GB of HBM3 vRAM and 480GB of LPDDR5x DRAM on the model from this spring, the « next-generation » GH200 features 141GB of HBM3e and 500GB of slower 400GB/sec LPDDR5x. The previous-generation used 512GB/sec LPDDR5x DRAM.
