NVIDIA Hopper architecture

The Hopper H100 tensor core GPU will support the NVIDIA Grace Hopper CPU + GPU architecture, designed specifically for terabyte-scale accelerated computing, delivering 10x higher performance in AI and HPC at large models. The NVIDIA Grace CPU leverages the flexibility of the Arm® architecture to create a CPU and server architecture designed from the ground up for accelerated computing. H100 combines with Grace as well as NVIDIA's ultra-fast chip-to-chip interconnect to deliver 900 GB/s bandwidth, 7x faster than 5th generation PCIe. This innovative design provides up to 30 times the total bandwidth of the fastest servers currently available and up to 10 times the performance for multi-terabyte applications.

LEARN MORE ABOUT GRACE

NVIDIA Hopper architecture extends tensor core technology with the Transformer engine for accelerating AI model training. Hopper tensor compute units are capable of applying mixed FP8 and FP16 precision to significantly accelerate AI computations for Transformers. Hopper also triples floating point operations per second (FLOPS) for TF32, FP64, FP16, and INT8 precisions over the previous generation. Combined with the Transformer engine and fourth-generation NVIDIA® NVLink®, Hopper Tensor compute units enable massive acceleration of HPC and AI workloads.

Learn more about NVIDIA Tensor compute units

While data is encrypted in storage and in transit through the network, it is unprotected during processing. Confidential computing closes this gap by protecting data and applications as they are processed. The NVIDIA Hopper architecture is the world's first accelerated computing platform to support Confidential Computing.

Strong hardware-based security provides users running applications on-premises, in the cloud, or in the periphery with the assurance that unauthorized parties cannot view or modify application code and data while they are in use. This protects the confidentiality and integrity of data and applications while enabling users to leverage the unprecedented acceleration of H100 GPUs for AI training, AI inference, and HPC workloads.

Learn more about NVIDIA Confidential Computing

A multi-instance GPU (MIG) can be split into multiple smaller, fully isolated instances with their own memory, cache and compute units. The Hopper architecture further enhances MIG to support multi-tenant, multi-user configurations in virtualized environments for up to seven GPU instances, with each instance securely isolated at the hardware and hypervisor levels through Confidential Computing. Dedicated video decoders for each MIG instance enable high-throughput intelligent video analytics (IVA) on shared infrastructure. With Hopper's concurrent MIG profiling, administrators can monitor correctly sized GPU acceleration and optimize resource allocation for users.

Researchers with smaller workloads can use MIG instead of a full CSP instance to securely isolate a portion of a GPU, confident that their data is protected during storage, transfer and processing.

Learn more about MIG

Dynamic programming is an algorithmic technique for solving complex recursive problems by breaking them into simpler sub-problems. By storing the results of subproblems, which thus do not need to be recomputed later, the time and complexity of exponential problem solving are reduced. Dynamic programming is widely used in a variety of use cases. For example, Floyd-Warshall is a route optimization algorithm for planning the shortest routes for shipping and delivery fleets. The Smith-Waterman algorithm is used for DNA sequence alignment and protein folding applications.

Hopper's DPX instructions enable a 40-fold speedup of dynamic programming algorithms over traditional dual-socket CPU servers and a 7-fold speedup over Ampere architecture GPUs. As a result, disease diagnostics, route optimization, and even graph analysis can be achieved significantly faster.

Modern confidential computing solutions are CPU-based, which is too limiting for compute-intensive workloads like AI and HPC. NVIDIA Confidential Computing is a built-in security feature of the NVIDIA Hopper architecture, making NVIDIA H100 the world's first accelerator with confidential computing capabilities. Users can protect the confidentiality and integrity of their data and applications while benefiting from the unparalleled acceleration of H100 GPUs for AI workloads. A hardware-based trusted execution environment (TEE) is created that protects and isolates the entire workload. This is executed on a single H100 GPU, multiple H100 GPUs within a node, or individual MIG instances. GPU-accelerated applications can run unmodified in TEE and do not need to be partitioned. Users can combine the power of NVIDIA software for AI and HPC with the security of a hardware root-of-trust application offered by NVIDIA Confidential Computing.

LEARN MORE ABOUT CONFIDENTIAL COMPUTING FROM NVIDIA