Diffusion almost lives up to its hype. It is CGI for dummies, and will produce photorealistic video with less effort than hand-drawing a stick figure. LLMs might disappear entirely, with every aspect of their design replaced, but ‘remove all the pixels that don’t look like Hatsune Miku impregnating Goku’ actually works. God help us all.
The practical future is in “diffusion forcing,” where a human artist can draw however many frames they like, and the robot can only fill in the gaps. If the robot does something wrong… draw more frames between stuff. One frame per second and a say-what-you-see description will probably suffice.
We can presumably also expect variations that finish sketchy animatics, but that’s always going to be less art-driven than artists would like. Absolute maniacs like James Baxter can feed in a pencil version of a camera orbiting a ballroom dance, but the robot will emit a broadly similar motion in finished quality. It’s better-off being used to turn on-fours into on-ones.
Diffusion almost lives up to its hype. It is CGI for dummies, and will produce photorealistic video with less effort than hand-drawing a stick figure. LLMs might disappear entirely, with every aspect of their design replaced, but ‘remove all the pixels that don’t look like Hatsune Miku impregnating Goku’ actually works. God help us all.
The practical future is in “diffusion forcing,” where a human artist can draw however many frames they like, and the robot can only fill in the gaps. If the robot does something wrong… draw more frames between stuff. One frame per second and a say-what-you-see description will probably suffice.
We can presumably also expect variations that finish sketchy animatics, but that’s always going to be less art-driven than artists would like. Absolute maniacs like James Baxter can feed in a pencil version of a camera orbiting a ballroom dance, but the robot will emit a broadly similar motion in finished quality. It’s better-off being used to turn on-fours into on-ones.