NVIDIA DGX A100

THE UNIVERSAL SYSTEM FOR

AI INFRASTRUCTURE

The Challenge of Scaling Enterprise AI SYSTEM SPECIFICATIONS

NVIDIA DGX A100 NVIDIA DGX A100
Every business needs to transform using artificial intelligence (AI), not only to 640GB 320GB

survive, but to thrive in challenging times. However, the enterprise requires a GPUs 8x NVIDIA A100 8x NVIDIA A100
80 GB GPUs 40 GB GPUs
platform for AI infrastructure that improves upon traditional approaches, which
GPU Memory 640 GB total 320 GB total
historically involved slow compute architectures that were siloed by analytics,
Performance 5 petaFLOPS AI
training, and inference workloads. The old approach created complexity,
10 petaOPS INT8
drove up costs, constrained speed of scale, and was not ready for modern AI.
NVIDIA 6
Enterprises, developers, data scientists, and researchers need a new platform NVSwitches
that unifies all AI workloads, simplifying infrastructure and accelerating ROI. System Power 6.5 kW max
Usage

The Universal System for Every AI Workload CPU Dual AMD Rome 7742, 128 cores total,
2.25 GHz (base), 3.4 GHz (max boost)

NVIDIA DGX ™ A100 is the universal system for all AI workloads—from analytics System Memory 2 TB 1 TB

to training to inference. DGX A100 sets a new bar for compute density, packing Networking 8x Single- 8x Single-
Port Mellanox Port Mellanox
5 petaFLOPS of AI performance into a 6U form factor, replacing legacy ConnectX-6 VPI ConnectX-6 VPI
compute infrastructure with a single, unified system. DGX A100 also offers 200Gb/s HDR 200Gb/s HDR
InfiniBand InfiniBand
the unprecedented ability to deliver fine-grained allocation of computing power,
2x Dual-Port 1x Dual-Port
using the Multi-Instance GPU (MIG) capability in the NVIDIA A100 Tensor Core Mellanox Mellanox
ConnectX-6 VPI ConnectX-6 VPI
GPU, which enables administrators to assign resources that are right-sized for
10/25/50/100/200 10/25/50/100/200
specific workloads. Available with up to 640 gigabytes (GB) of total GPU memory, Gb/s Ethernet Gb/s Ethernet

which increases performance in large-scale training jobs up to 3X and doubles Storage OS: 2x 1.92 TB M.2 OS: 2x 1.92TB M.2
NVME drives NVME drives
the size of MIG instances, DGX A100 can tackle the largest and most complex
Internal Storage: Internal Storage:
jobs, along with the simplest and smallest. Running the DGX software stack 30 TB (8x 3.84 TB) 15 TB (4x 3.84 TB)
U.2 NVMe drives U.2 NVMe drives
with optimized software from NGC, the combination of dense compute power
Software Ubuntu Linux OS
and complete workload flexibility make DGX A100 an ideal choice for both single
Also supports:
node deployments and large scale Slurm and Kubernetes clusters deployed Red Hat Enterprise Linux
with NVIDIA DeepOps. CentOS

System Weight 271.5 lbs (123.16 kgs) max

Direct Access to NVIDIA DGXperts Packaged System 359.7 lbs (163.16 kgs) max
Weight
NVIDIA DGX A100 is more than a server. It’s a complete hardware and software System Height: 10.4 in (264.0 mm)
platform built upon the knowledge gained from the world’s largest DGX proving Dimensions Width: 19.0 in (482.3 mm) max
ground—NVIDIA DGX SATURNV—and backed by thousands of DGXperts at Length: 35.3 in (897.1 mm) max

NVIDIA. DGXperts are AI-fluent practitioners who offer prescriptive guidance Operating 5–30 ºC (41–86 ºF)
Temperature
and design expertise to help fastrack AI transformation. They've built a wealth Range

NVIDIA DGX A100  |  DATA SHEET  |  Nov20

of know-how and experience over the last decade to help
Up to 3X Higher Throughput for AI Up to 1.25X Higher Throughput for
maximize the value of your DGX investment. DGXperts
Training on Largest Models AI Inference
help ensure that critical applications get up and running
DLRM Training RNN-T Inference: Single Stream
quickly, and stay running smoothly, for dramatically-
improved time to insights. 3X 2X 3X
3X

Fastest Time to Solution 2X

1.25X
1X
NVIDIA DGX A100 features eight NVIDIA A100 Tensor Core 1X
1X
GPUs, which deliver unmatched acceleration, and is 1X
fully optimized for NVIDIA CUDA-X ™ software and the 0.7X

end-to-end NVIDIA data center solution stack. NVIDIA 0 0

DGX-2 DGX A100 DGX A100 DGX A100 DGX A100
A100 GPUs bring a new precision, Tensor Float 32 (TF32), 320GB 640GB 320GB 640GB
FP16 FP16 FP16
which works just like FP32 but provides 20X higher Time Per 1,000 Iterations - Relative Performance
Sequences Per Second - Relative Performance

floating operations per second (FLOPS) for AI compared DLRM on HugeCTR framework, precision = FP16 | 1x DGX
MLPerf 0.7 RNN-T measured with (1/7) MIG slices. Frame-
work: TensorRT 7.2, dataset = LibriSpeech, precision = FP16.
to the previous generation. Best of all, no code changes A100 640GB batch size = 48 | 2x DGX A100 320GB batch
size = 32 | 1x DGX-2 (16x V100 32GB) batch size = 32.
Speedups Normalized to Number of GPUs.
are required to achieve this speedup. And when using
NVIDIA’s automatic mixed precision with FP16, A100
offers an additional 2X boost to performance with just one
additional line of code. Up to 83X Higher Throughput than CPU, 2X Higher Throughput than
DGX A100 320GB on Big Data Analytics Benchmark
The A100 80GB GPU doubles the high-bandwidth memory
from 40 GB (HBM) to 80GB (HBM2e) and increases GPU 90X

memory bandwidth 30 percent over the A100 40 GB GPU to 80X

83X
be the world's first with over 2 terabytes per second (TB/s). 70X
Up to 2X
DGX A100 also debuts the third generation of NVIDIA® 60X

NVLink®, which doubles the GPU-to-GPU direct bandwidth 50X

Up to 83X
to 600 gigabytes per second (GB/s), almost 10X higher 40X 44X
than PCIe Gen 4, and a new NVIDIA NVSwitch™ that’s 2X 30X

faster than the last generation. This unprecedented power 20X

delivers the fastest time to solution, allowing users to 10X

1X 11X
tackle challenges that weren't possible or practical before. 0
CPU Only DGX-1 DGX A100 DGX A100
320GB 640GB
The World’s Most Secure AI System Time to Solution - Relative Performance

for Enterprise Big data analytics benchmark | 30 analytical retail queries, ETL, ML, NLP on 10TB dataset | CPU: 19x Intel Xeon Gold
6252 2.10 GHz, Hadoop | 16x DGX-1 (8x V100 32GB each), RAPIDS/Dask | 12x DGX A100 320GB and 6x DGX A100
640GB, RAPIDS/Dask/BlazingSQL. Speedups Normalized to Number of GPUs.
NVIDIA DGX A100 delivers the most robust security posture
for your AI enterprise, with a multi-layered approach that
secures all major hardware and software components. InfiniBand adapters for clustering and up to two dual-port ConnectX-6
Stretching across the baseboard management controller VPI Ethernet adapters for storage and networking, all capable of 200 Gb/s.
(BMC), CPU board, GPU board, self-encrypted drives, The combination of massive GPU-accelerated compute with state-of-
and secure boot, DGX A100 has security built in, allowing the-art networking hardware and software optimizations means DGX
IT to focus on operationalizing AI rather than spending A100 can scale to hundreds or thousands of nodes to meet the biggest
time on threat assessment and mitigation. challenges, such as conversational AI and large-scale image classification.

Unmatched Data Center Scalability Proven Infrastructure Solutions Built with

with NVIDIA Mellanox Trusted Data Center Leaders
With the fastest I/O architecture of any DGX system, NVIDIA In combination with leading storage and networking technology providers,
DGX A100 is the foundational building block for large AI a portfolio of infrastructure solutions are available that incorporate the
clusters like NVIDIA DGX SuperPOD™, the enterprise best of the NVIDIA DGX POD™ reference architecture. Delivered as fully
blueprint for scalable AI infrastructure. DGX A100 features integrated, ready-to-deploy offerings through our NVIDIA Partner Network
eight single-port NVIDIA Mellanox® ConnectX®-6 VPI HDR (NPN), these solutions simplify and accelerate data center AI deployments.

To learn more about NVIDIA DGX A100, visit www.nvidia.com/DGXA100

© 2020 NVIDIA Corporation. All rights reserved. NVIDIA, the NVIDIA logo, CUDA-X, DGX A100, DGX POD, DGX SuperPOD,
Mellanox, NVLink, and NVSwitch are trademarks and/or registered trademarks of NVIDIA Corporation. All company and
product names are trademarks or registered trademarks of the respective owners with which they are associated. Features,
pricing, availability, and specifications are all subject to change without notice. Nov20