One of our favourite pieces from the past 12 months. Originally published 22 July, 2022.It takes a whole lot of GPU power to run today's most powerful AI models. Even the most extreme gaming PC b
Audio player loading…
One of our favourite pieces from the past 12 months. Originally published 22 July, 2022.
It takes a whole lot of GPU power to run today’s most powerful AI models. Even the most extreme gaming PC build (opens in new tab) wouldn’t be able to handle the kind of oomph needed to get the best AI Dungeon (opens in new tab) models up and running. In order for Latitude to power what’s essentially this less numbers-heavy, more free and creative online D&D campaign, you’d need something truly beastly.
According to Nick Walton, the CEO of Latitude and the software engineer who helped design the initial code for the D&D-inspired story generator (opens in new tab), „If you wanted to run the largest AI for AI dungeon, you would need not just a high-powered GPU, but a cluster of eight or 16 massive high-powered GPUs.“ Even those would need to be more powerful than the best graphics cards (opens in new tab) on the consumer market.
Language models such as AI Dungeon’s Dragon experience, based on the 178 billion parameter Jurrasic-1 Jumbo language model developed by AI21 (opens in new tab), need a huge level of computing power to recognise and recall important story elements, and utilise complex writing styles.
Nick pointed me toward the DGX Station A100 (opens in new tab) as reference for the kind of machines Latitude uses to power the game’s more intense AI models (opens in new tab). It’s a computing monster even with just four of Nvidia’s $10,000 A100 GPUs—the first and essentially the most powerful Ampere graphics cards to surface.
AI Dungeon … is the most demanding game in terms of graphics card requirementsNick Walton, Latitude CEO
„AI Dungeon, especially at launch and probably still even today, is the most demanding game in terms of graphics card requirements,“ says Nick, „and for a long time it had no graphics.“
For machines like the DGX Station with, say, eight top-of-the-line A100 GPUs jammed in, you’re looking at around $150,000 in up front tech costs, and that’s just to run the most advanced AI alone—the company has several other AI models it needs to run simultaneously for your dungeoning pleasure.
