NVIDIA’s massive Ampere-based A100 GPU is already in full production and shipping to customers as well.
According to NVIDIA, its Ampere-based A100 GPU is already in full production and shipping to customers, and it is claiming that the A100 represents the single largest generational uplift in GPU performance in its history. Given the specs, we have no reason to doubt the company.
The A100 features a whopping 54 billion transistors, which definitely allows it to take the crown as the world’s largest processor built on 7nm tech. There are a total of 6,912 FP32 CUDA cores, 432 Tensor cores, and 108 SMs. Onboard is 40GB of HBM2e memory with maximum memory bandwidth of 1.6TB/sec. FP32 compute comes in at a staggering 19.5 TLFLOPs, compared to 16.4 TFLOPs for Tesla V100s. FP64 compute stands at 9.7 TFLOPs, compared to 8.2 TFLOPs for Tesla V100s. In addition, its Tensor Cores employ FP32 precision that allows for a 20x uplift in AI performance. When it comes to FP64 performance, these Tensor Cores also provide a 2.5x performance boost, compared to its predecessor, Volta, with respect to HPC (High Performance Compute) applications.
