3:11
A diffusion model is a type of generative machine learning model that creates new data, most commonly images, by learning to reverse a gradual noising process. Diffusion models are the technology behind most modern AI image generators, and are also increasingly used for audio, video, and even some scientific applications like molecule design.
The central idea behind diffusion models is deceptively simple: take a real image and gradually destroy it by adding random noise over many steps until it becomes pure static, then train a neural network to reverse that process, one small step at a time. Once the network has learned to reliably undo a single small step of noise, it can be applied repeatedly starting from pure random noise, gradually "denoising" it into a coherent image that the network has never actually seen before.
Training begins with the forward process: a real training image has a small, carefully calibrated amount of random (Gaussian) noise added to it, repeated over many steps (often on the order of hundreds or thousands), until the image is indistinguishable from random noise. This process itself requires no learning - it's a fixed mathematical procedure - but it produces exactly the paired data the model needs: for every step, a slightly-more-noisy version of the image and a slightly-less-noisy version, so the model has something concrete to learn to predict.
The neural network - typically a U-Net or, in more recent models, a transformer-based architecture - is trained to take a noisy image at a given step and predict either the noise that was added or the original clean image. By training on millions of images across all the different noise levels produced by the forward process, the model learns a general-purpose skill: given any noisy image and how much noise is roughly present, predict a slightly cleaner version of it.
noisy_image = add_noise(real_image, step)
predicted_noise = model(noisy_image, step)
loss = difference(predicted_noise, actual_noise_added)To generate a brand new image, the process starts from pure random noise and runs the reverse process: the model predicts and removes a small amount of noise, producing a slightly cleaner (but still very noisy) image; this is fed back into the model and the process repeats for many steps, with the image gradually becoming more coherent at each step until a fully denoised, realistic image emerges at the end. Because this starts from random noise, every generation run produces a different image, even with an identical prompt.
To turn a diffusion model into a text-to-image generator, the denoising network is additionally given a text embedding (usually produced by a separate language model) at every step, alongside the noisy image. The network learns to use this text embedding to guide which direction to denoise toward, so that a prompt like "a red bicycle on a beach" steers the many denoising steps toward an image matching that description, rather than an arbitrary realistic image.
Before diffusion models became dominant, generative adversarial networks (GANs) were the leading approach to image generation, using a generator network competing against a discriminator network trying to catch fakes. GANs can generate images in a single fast step, but are notoriously difficult and unstable to train. Diffusion models are generally more stable to train and tend to produce more diverse, higher-fidelity outputs, at the cost of being slower to generate from, since they require many sequential denoising steps rather than a single forward pass - though ongoing research has significantly reduced the number of steps needed in recent years.
By: Tomas Silny
Edited: 2026-08-13 06:50:01