NVIDIA released CUDA 12.8, introducing full compiler, profiler, and library support for Blackwell GPUs and native FP4 / NVFP4 Tensor Core instructions across datacenter and workstation hardware.
CUDA
Also known as: CUDA Toolkit · cuBLAS · CUTLASS · CUDA Graphs
Plain English
NVIDIA's parallel computing platform and programming language that lets AI frameworks and inference engines run math directly across thousands of GPU cores at once.
Technical Overview
CUDA exposes NVIDIA GPU thread hierarchies, shared SRAM, warp-level matrix instructions (Tensor Cores), and low-overhead CUDA Graphs to C++/PTX developers and libraries like cuBLAS, cuDNN, and CUTLASS. Nearly every modern foundation-model kernel—including FlashAttention, Marlin INT4, FP8/NVFP4 GEMMs, and vLLM/SGLang/TensorRT-LLM runtimes—targets CUDA as its primary execution layer.
Chronological Entries (3)
NVIDIA shipped CUDA 12.0, adding full support for Hopper H100 features including Tensor Memory Accelerator (TMA), asynchronous thread-block clusters, and native FP8 Tensor Core instructions used by FlashAttention-3 and modern LLM clusters.
NVIDIA released the initial CUDA SDK, allowing general-purpose C programs to execute parallel kernels directly on GeForce 8800 GPUs without disguising math as graphics shaders—laying the software foundation for GPU-accelerated deep learning.
Direct Relations (4)
- Created byNVIDIA
- EnablesFlashAttention & GPU Kernels
- EnablesInference Engines
- EnablesPyTorch