IndexMapAbout
Format / ProtocolLLMCross-Modal

FP4

Also known as: 4-bit Floating Point · E2M1 · MXFP4 · OCP MXFP4

Plain English

A 4-bit floating-point number format that uses uneven spacing—finer near zero, coarser for large values—making 4-bit AI weights and activations more accurate than uniform 4-bit integers and executable directly on modern AI chips.

Technical Overview

FP4 typically uses the E2M1 layout (1 sign bit, 2 exponent bits, 1 mantissa bit), representing 16 non-uniform values that better match bell-curved neural weight distributions than INT4. Because 4 bits alone lack dynamic range, practical deployments rely on block-scaled formats such as the Open Compute Project's MXFP4 (32-element blocks sharing an E8M0 power-of-two scale) or NVIDIA's NVFP4.

Chronological Entries (3)

#1Paper
AI-Assisted

Researchers from the OCP Microscaling Alliance published Microscaling Data Formats for Deep Learning, evaluating the MXFP4 standard (E2M1 elements paired with a shared 8-bit E8M0 scale per 32-value block). Sharing one exponent scale across 32 elements let 4-bit floating-point dot products execute in hardware with only 0.25 bits of scale overhead per value.

Direct Relations (4)