How Diffusion Models Make Images: From Pure Noise to a Picture

How AI image generators turn random static into a picture: the idea of destroying an image with noise and learning to undo it, how the model is trained to predict noise, what happens step by step when you generate, why latent diffusion made it cheap enough to run, and how a text prompt steers the result.

0:36 Destroy, then learn to undo

Diffusion models come from an idea borrowed from physics: slowly destroy the structure in data with noise, then learn the reverse process that restores it.

Sources: arXiv (original) · Hugging Face blog (original)

2:05 Training a noise detector

In the denoising diffusion approach of Ho and colleagues, the network learns to predict the noise that was added to an image, scored with a simple error measure.

Sources: arXiv (original) · Hugging Face blog (original)

3:41 Generating, step by step

Generation starts from random noise the size of the output; at each step the model predicts the noise and a scheduler removes part of it.

Sources: Hugging Face Diffusers docs (original)

4:56 Latent diffusion: working on a sketch

Latent diffusion runs the denoising in the compressed space of an autoencoder instead of on full pixels, which cuts the cost dramatically.

Sources: arXiv (original) · Hugging Face Diffusers docs (original)

6:19 How the prompt steers the picture

A text encoder turns the prompt into numbers the denoiser attends to, and classifier-free guidance controls how strongly the image follows the prompt.

Sources: arXiv (original) · arXiv (original) · Hugging Face Diffusers docs (original)

Follow the show in your podcast app to get each new episode as soon as it is out.

Produced by Juan Pablo.