Black Forest Labs released FLUX.1 Tools (Canny, Depth, Fill, and Redux), adapting structural and reference-image conditioning natively to DiT architectures alongside community ComfyUI ControlNet unions.
ControlNet & IP-Adapter
Also known as: ControlNet · IP-Adapter · Spatial Conditioning
Plain English
Add-on modules that give you precise visual steering over an image or video generator—letting you lock in a pose, architectural layout, depth map, sketch, or reference face/style instead of relying on text prompts alone.
Technical Overview
ControlNet clones trainable encoder/transformer blocks linked via zero-initialized convolutions to inject spatial conditioning (Canny edges, depth, OpenPose skeletons, segmentation maps) into frozen Diffusion Models. IP-Adapter injects decoupled image-prompt cross-attention from a reference image, complementing LoRA & QLoRA in production ComfyUI pipelines.
Chronological Entries (2)
Lvmin Zhang et al. published ControlNet, transforming text-to-image models from unpredictable slot machines into controllable design tools by conditioning Diffusion Models on depth maps, line art, normal maps, and human poses.
Direct Relations (3)
- ExtendsDiffusion Models
- Part ofComfyUI
- UsesFLUX