Generative AIintermediate11 min · 8 steps
Video Generation: Diffusion Across Time
Video is many images that must agree. See how video diffusion denoises across space and time, and how the same machinery powers text-to-video, image-to-video, editing and prediction.
requestdataresponsesignal
ProblemStep 1 of 8
Generate each frame on its own and the video falls apart
A video is a sequence of images. The naive approach is to run an image model once per frame with the same prompt. Each frame looks fine on its own, but the fox changes colour, grows a second tail and jumps around.
The missing ingredient is temporal consistency: every frame has to agree with its neighbours, and motion has to look physically plausible.
Watch outVideo is not just “many images”. Coherence over time and realistic motion are what make it hard.
← Previous lessonImage Generation: Diffusion, Text-to-Image and EditingNext lesson →Audio and Speech Generation
Lesson 6 of 8 in Generative AI