Faster Scientific Simulations with New FP64 Algorithms in NVIDIA cuBLAS

NVIDIA CUDA Toolkit 13.4.2 introduces Ozaki-II and 2M algorithms that accelerate double-precision GEMM on integer Tensor Cores while preserving full FP64 accuracy. These methods enable significant speedups for scientific simulations without compromising numerical precision.

Audio reading is not available in this browser
Faster Scientific Simulations with New FP64 Algorithms in NVIDIA cuBLAS

Tags

Quick summary

NVIDIA CUDA Toolkit 13.4.2 introduces Ozaki-II and 2M algorithms that accelerate double-precision GEMM on integer Tensor Cores while preserving full FP64 accuracy. These methods enable significant speedups for scientific simulations without compromising numerical precision.

Faster Scientific Simulations with New FP64 Algorithms in NVIDIA cuBLAS

Many scientific simulations, ranging from molecular chemistry to materials science, spend the majority of their execution time computing high-precision dense general matrix multiplications (GEMM). These operations are critical for accuracy in fields where lower-precision approximations are insufficient. Traditionally, achieving full double-precision (FP64) accuracy required using native FP64 instructions, which are often slower than lower-precision operations on modern hardware.

With the release of CUDA Toolkit 13.4.2, NVIDIA cuBLAS introduces a new class of algorithms that accelerate double-precision GEMM on integer Tensor Cores while preserving full FP64 accuracy. This advancement allows cuBLAS to break each double-precision number into integer pieces, multiply them on the GPU's Tensor Cores, and reassemble the exact result. According to NVIDIA, this approach means cuBLAS no longer has to choose between accuracy and speed.

The Ozaki-II and 2M Algorithms

Two new algorithms make this acceleration more efficient: Ozaki-II and the 2M method. The Ozaki-II algorithm reduces the number of integer multiplications compared to the earlier Ozaki-I approach by utilizing the Chinese Remainder Theorem. The 2M method cuts complex matrix multiplication from four real multiplications to two.

These algorithms build on the Automatic Dynamic Precision (ADP) framework introduced previously in cuBLAS. The ADP framework delivers higher double-precision matrix-multiplication performance by mapping FP64 arithmetic onto integer Tensor Cores while preserving native FP64 accuracy. cuBLAS delivers automatic acceleration of DGEMM and ZGEMM with no application code changes. Because speed matters only when accuracy is not compromised, NVIDIA validates these gains on real-world applications.

How Ozaki-II Works

Ozaki-I and Ozaki-II schemes share a common first step: aligning the exponents of the input matrices and converting them to a shared-exponent, variable-length integer representation so that products accumulate exactly on integer Tensor Cores. cuBLAS uses heuristics to automatically choose between Ozaki-I and Ozaki-II for best performance. The integer width is chosen by ADP at runtime or can also be set explicitly by the user.

The key difference lies in how the big integer multiplication is performed. Ozaki-I breaks each big integer into 8-bit slices and multiplies slices of A against slices of B in a triangular pattern. Consequently, the number of Tensor Core GEMMs grows with the square of the slice count.

In contrast, Ozaki-II uses a faster big-integer multiplication algorithm based on the Chinese Remainder Theorem (CRT). It reduces each quantized operand modulo a set of small pairwise-coprime integers, performs exactly one integer GEMM per modulus, and reconstructs the exact product from the residues. With Ozaki-II, the GEMM count is linear in the number of moduli, whereas Ozaki-I grows with the square of the slice count. Therefore, the advantage widens as the accuracy target rises. At the accuracy required to match FP64, Ozaki-I needs roughly twice as many Tensor Core GEMMs as Ozaki-II.

Regarding memory usage, because Ozaki-II materializes one residue pair per modulus, a naive implementation would carry a much larger workspace than Ozaki-I. However, cuBLAS applies the memory-parsimonious tiling introduced for Ozaki-I in CUDA Toolkit 13.3 to the Ozaki-II scheme as well. This results in discarding each residue pair immediately after its GEMM, leading to a more manageable workspace size.

Performance Highlights

NVIDIA reports significant performance gains with the new algorithms across various hardware platforms. cuBLAS DGEMM with Ozaki-II runs 5.5x faster than native FP64 on NVIDIA VR NVL72 and 3.9x faster on NVIDIA GB200. It is also 2x faster than Ozaki-I on NVIDIA GB200.

For complex matrix multiplications, cuBLAS ZGEMM with the 2M algorithm achieves a 5x speedup over native Complex FP64 (CFP64) on NVIDIA GB200. In terms of peak performance, cuBLAS DGEMM with Ozaki-II achieves up to 175 TFLOPS on NVIDIA VR NVL72, 147 TFLOPS on NVIDIA GB200, and 37 TFLOPS on NVIDIA RTX PRO 6000.

Accuracy Guarantees

The cuBLAS ADP framework is designed to provide accuracy guarantees comparable to, or better than, standard floating-point GEMM algorithms. If at run-time, the number of mantissa bits required to preserve FP64-equivalent accuracy exceeds the library default or a user-supplied maximum, cuBLAS falls back to a code path that utilizes the native FP64 MMA instructions. This ensures that applications can safely use cuBLAS in production deployments without worrying about numerical accuracy.

Real-World Application Benchmarks

To demonstrate the potential application performance of these algorithms, NVIDIA evaluated two specific use cases: coupled cluster singles and doubles (CCSD) in quantum chemistry and BerkeleyGW in materials science.

CCSD in Quantum Chemistry

CCSD is a foundational post-DFT method in electronic structure that forms the "gold standard" of quantum chemistry. It is noteworthy for its very high cost of dense FP64 tensor contractions, which the Ozaki-II scheme in cuBLAS can significantly accelerate.

A small CCSD demonstration was implemented using cuBLAS in CUDA Toolkit 13.4.2. The results show that Ozaki-II DGEMM with an Error Scaling (ESC) guarantee of full FP64 precision causes performance to rise monotonically with generation, yielding a total speedup of 4.7x of Rubin over A100.

Another compelling possibility is to use an application-specific mixed precision mode with a fixed Ozaki-II mantissa bit budget, selected here to be 32 mantissa bits. This yields a total speedup of 11.5x native FP64 A100. With ESC at full FP64 quality, results are indistinguishable from native FP64 precision, with errors in the CCSD correlation energy of -14 E h.

Ozaki-II emulation also provides the opportunity to use lower fixed precision with 32 mantissa bits while still providing sufficient accuracy for the vast majority of practical applications. In this mode, errors in the energy are -10 E h, and the amplitude equations can be converged to a residual of 10^-9 or better. On Rubin for uracil dimer frozen-core CCSD/cc-pVTZ, fixed precision with 32 mantissa bits achieves an end-to-end throughput of 173 TFLOPS; the throughput of individual GEMMs is as high as 368 TFLOPS with an average GEMM throughput of 200 TFLOPS.

BerkeleyGW in Materials Science

BerkeleyGW is a materials science simulation tool. Using cuBLAS ZGEMM with the 2M algorithm, BerkeleyGW achieves a 6.6x speedup over native ZGEMM on NVIDIA VR NVL72 and a 3.9x speedup on NVIDIA GB200.

NVIDIA Rubin and TI16 Format

NVIDIA Rubin GPUs introduce the TI16 numeric format and the tcgen05.mma instruction, further accelerating Ozaki-I and Ozaki-II computations. The TI16 format is a 12-bit integer in sign-magnitude format with a range of [-2047, +2047], which is aptly suited for the acceleration of Ozaki scheme computations. cuBLAS 13.4 uses the TI16 type for acceleration of Ozaki-I and Ozaki-II computations through the tcgen05.mma instruction on NVIDIA Rubin GPUs (compute capability 10.7).

Getting Started

Developers can access these new algorithms by downloading and installing the latest NVIDIA CUDA Toolkit. The cuBLAS documentation includes a floating point emulation section with API details, including cublasSetEmulationStrategy and manual mantissa bit control.

To evaluate gains on their own workloads, users can explore the cuBLAS Emulation samples on GitHub. These samples allow developers to test the performance and accuracy of the new algorithms in their specific environments.

Conclusion

The introduction of Ozaki-II and the 2M algorithm in CUDA Toolkit 13.4.2 marks a significant step forward in high-performance computing. By leveraging integer Tensor Cores to perform double-precision operations, NVIDIA cuBLAS achieves substantial speedups over native FP64 implementations without sacrificing accuracy. Whether for quantum chemistry simulations or materials science modeling, these algorithms offer a powerful tool for accelerating scientific discovery.

Source: https://developer.nvidia.com/blog/faster-scientific-simulations-with-new-fp64-algorithms-in-nvidia-cublas

Sources