Microsoft Azure recently unveiled its latest Virtual Machines that are powered by NVIDIA's H100 GPUs, the high-end graphics processing units which are specifically designed to enhance the performance of generative AI applications like ChatGPT. These new Azure Virtual Machines are equipped with NVIDIA's cutting-edge H100 GPUs, enabling them to power the latest generative AI content, including ChatGPT.
Microsoft Azure introduces ChatGPT-accelerating NVIDIA H100 GPU VMs for Generative AI workloads
Microsoft is utilizing over a decade of experience in supercomputing and supporting large AI training workloads to create AI infrastructure that can deliver massive performance at scale. The Microsoft Azure cloud, powered by GPU-accelerated virtual machines (VMs), provides the foundation for many generative AI advancements from both Microsoft and its customers. This collaboration has been crucial in scaling demanding AI training needs, making research and alignment work on systems like ChatGPT possible.
To offer the most powerful and massively scalable AI virtual machines, Microsoft has introduced the ND H100 v5 VM. This virtual machine can be customized on-demand, ranging from eight to thousands of NVIDIA H100 GPUs interconnected by NVIDIA Quantum-2 InfiniBand networking. The innovative technologies used in this virtual machine, such as 8x NVIDIA H100 Tensor Core GPUs, next-gen NVSwitch and NVLink 4.0 interconnectivity, and 400 Gb/s NVIDIA Quantum-2 CX7 InfiniBand per GPU, provide significantly faster performance for AI models compared to the previous generation ND A100 v4 VMs.
The ND H100 v5 VM also boasts other advanced features, including NVSwitch and NVLink 4.0 with 3.6TB/s bisectional bandwidth between 8 local GPUs within each VM, 4th Gen Intel Xeon Scalable processors, PCIE Gen5 host to GPU interconnect with 64GB/s bandwidth per GPU, and 16 channels of 4800MHz DDR5 DIMMs. With these features, Microsoft Azure offers a powerful and scalable AI infrastructure to meet the most demanding AI training workloads.
Azure's most powerful and massively scalable AI virtual machine series
Today, Microsoft is introducing the ND H100 v5 VM which enables on-demand in sizes ranging from eight to thousands of NVIDIA H100 GPUs interconnected by NVIDIA Quantum-2 InfiniBand networking. Customers will see significantly faster performance for AI models over our last generation ND A100 v4 VMs with innovative technologies like:
- 8x NVIDIA H100 Tensor Core GPUs interconnected via next-gen NVSwitch and NVLink 4.0
- 400 Gb/s NVIDIA Quantum-2 CX7 InfiniBand per GPU with 3.2Tb/s per VM in a non-blocking fat-tree network
- NVSwitch and NVLink 4.0 with 3.6TB/s bisectional bandwidth between 8 local GPUs within each VM
- 4th Gen Intel Xeon Scalable processors
- PCIE Gen5 host to GPU interconnect with 64GB/s bandwidth per GPU
- 16 Channels of 4800MHz DDR5 DIMMs
Delivering exascale AI supercomputers to the cloud
Microsoft is leading the way in delivering exascale AI supercomputers to the cloud, enabling customers to achieve true supercomputer performance with its infrastructure. As generative AI applications continue to evolve, they are adding unique value across a wide range of industries, from AI-powered Microsoft Bing and Edge to AI-assisted Microsoft Dynamics 365.
"Our focus on conversational AI requires us to develop and train some of the most complex large language models. Azure's AI infrastructure provides us with the necessary performance to efficiently process these models reliably at a huge scale. We are thrilled about the new VMs on Azure and the increased performance they will bring to our AI development efforts."—Mustafa Suleyman, CEO, Inflection.
With Microsoft's experience in delivering multiple-ExaOP supercomputers to Azure customers worldwide, organizations can rely on its infrastructure to support large-scale AI deployments, enabling a new class of AI models. Azure's initial investments in large language model research, such as Turing, and engineering milestones, like building the first AI supercomputer in the cloud, have prepared the platform for the moment when generative AI became possible.
Azure's DNA is built around AI at scale, with services like Azure Machine Learning and Azure OpenAI Service providing access to AI supercomputers for model training and the power of large-scale generative AI models. Azure is bringing supercomputing capabilities to startups and businesses of all sizes, making it accessible without requiring the capital for massive physical hardware or software investments.
“NVIDIA and Microsoft Azure have collaborated through multiple generations of products to bring leading AI innovations to enterprises around the world. The NDv5 H100 virtual machines will help power a new era of generative AI applications and services.”—Ian Buck, Vice President of hyperscale and high-performance computing at NVIDIA.
Today, Microsoft is proud to announce that the ND H100 v5 is available for preview and will become a standard offering in the Azure portfolio, enabling anyone to unlock the potential of AI at Scale in the cloud. Customers can sign up to request access to the new VMs and join the revolution in generative AI.
