A diffusion model makes an image or a video by starting from noise and taking it away a step at a time. When the engine running the model is your own, every one of those steps is open to you, and you can change what goes into the next one. That's what my video and image tools do to replace part of an image, replace part of a video, or extend a video, and there's a lot more that can be done with it.
How denoising works
The model is given a noisy image and how noisy it is, and predicts what the clean image would look like. The sampler uses that prediction to take away some of the noise, and the result goes back into the model for the next step. After twenty or thirty steps the noise is gone.
Replacing part of an image
At every step, the part of the image you want to keep is replaced with your own image, noised to the same level as the step. The model only has to invent the rest, and because it sees your image at every step, what it makes fits around it.
Extending with overlaps
To carry a video on, or anything else a diffusion model makes, the next block starts with an overlap: its first frames are the last frames of the block before, put back in at every step the same way. This works with any diffusion model.
Sliding windows
A sliding window goes further. Several blocks are denoised together, each at a different noise level, so the oldest comes out clean as a new block of pure noise joins at the far end. Each block is shaped by the ones next to it, which pushes the consistency further along than one overlap can.
Ideas to try
For consistency over longer stretches, the same denoising step could be repeated over and over, adding noise back and taking it away again, so the context has time to spread out further into the blocks that follow.
See also
- Writing my own AI inference engine - the engines that make this possible
- Making faces - a face generation tool
- My AI workflow in 2026 - how the video tools came about