FIFoundationIndex
IndexMapAbout
Format / ProtocolLLMImage

GGUF

Also known as: GGML · .gguf

Plain English

A single-file model format built for running AI models locally on laptops, Apple Silicon Macs, and consumer GPUs—packaging compressed weights and tokenizer metadata together so you can run a model with zero complex setup.

Technical Overview

Introduced by Georgi Gerganov's llama.cpp project to replace GGML, GGUF is a self-contained binary format storing model metadata, tokenizer vocabulary, and block-wise quantized tensors (Q4_K_M, Q8_0, IQ4_XS). It supports memory-mapped partial GPU/CPU offloading and has expanded beyond LLMs to quantized DiT weights like FLUX in ComfyUI.

Chronological Entries (2)

#1Release
Curated

The llama.cpp team replaced GGML with GGUF, adding extensible key-value metadata so tools like Ollama, LM Studio, and llama.cpp could load any model architecture and tokenizer without brittle breaking changes.

Direct Relations (4)