CUDA
NVIDIA's parallel computing platform and programming model — launched 2006. The moat that made NVIDIA the world's most valuable company.
C/C++ extension (__global__ void kernel(...)), executes on GPU cores. Hierarchy: thread → warp (32 threads) → block → grid. Compiles to PTX (NVIDIA's intermediate representation) → SASS (specific to the GPU generation). Libraries on top: cuBLAS, cuDNN, NCCL, TensorRT.
PyTorch, TensorFlow, JAX and llama.cpp use CUDA via those libs. Competitor: AMD ROCm (trying to catch up), Intel oneAPI/SYCL, Apple Metal, OpenCL (lost). Mojo and Triton try to abstract CUDA away.