Every AI filmmaker knows the shot: the performance landed, the light is right, the camera move works, and there is one wrong thing in frame. A logo. A sixth finger on a background hand. An object that should not exist. The reflex is to regenerate, and it is expensive, because a reroll replaces the whole shot, blocking, performance, and all, with a new sample that has new problems. Inpainting is the discipline of fixing the five percent and leaving the ninety-five percent untouched.
- A reroll resamples the whole shot; inpainting constrains generation to a masked region and preserves everything outside it, including the cut you already made.
- Count expected attempts, not single runs: several full-price rerolls chasing a shot you already had against one targeted patch.
- The workflow is five steps: segment, track, dilate, inpaint, composite. Per-frame filling flickers; the inpainting model must be temporally aware.
- Mask the object and its consequences: the shadow it casts and the reflection it leaves are the two orphans that give a fix away.
Why fixing beats rerolling
A regeneration resamples the entire latent: composition, performance, lighting, texture, everything moves at once. The odds that a new sample reproduces the specific take you liked, minus the one defect, are close to zero, so a reroll is the first of several attempts, each at full generation cost, each producing a shot your locked edit has never seen. Inpainting inverts the economics. Generation is constrained to the masked region; every pixel outside it ships exactly as it was. One pass, and the cut stays intact.
The second argument has nothing to do with money. Once an edit is assembled, a shot is load-bearing: its timing is cut against music, its eyelines match its neighbors. A reroll that changes blocking breaks the sequence around it. A patch does not. This is the same discipline the wider AI filmmaking guide teaches at every stage: protect what works, spend iteration only on what fails.
What does the workflow look like?
Hosted tools compress this into an erase brush and a prompt. Open pipelines expose every step. Either way, five things happen, and knowing them tells you where a bad result went wrong.
- Segment. Select the target on one frame, by click or text prompt, and get a pixel-accurate mask. Segmentation models of the SAM family are the standard tool.
- Track. Propagate that mask across every frame of the shot, following the object through motion and partial occlusion.
- Dilate. Grow the mask a few pixels in every direction, so edge halos and soft fringes of the object fall inside the fill region rather than surviving at its border.
- Inpaint. Fill the region with a temporally aware model, one that propagates appearance across frames instead of inventing a fresh answer per frame.
- Composite. Lay the patch back over the original so every untouched pixel stays bit-identical, then match grain so the clean patch does not float on the noisier plate.
What tools do this in 2026?
| Path | What it does |
|---|---|
| Runway | Prompt-driven shot surgery: erase an object and the background rebuilds, plus relighting and swaps, in the same editing family as its restyle tools. |
| Pika, Kling | Region edits: paint or select an area, prompt the replacement, keep the rest of the shot. |
| Premiere generative tools | Generative Extend lengthens a clip a beat when the edit needs it. Check current docs before planning around object removal; editor features ship gradually. |
| ComfyUI open path | SAM-family segmentation and tracking, flow-propagation inpainters of the ProPainter lineage, diffusion fills for larger regions. Free, local, inspectable. |
| Reframe and expand tools | Outpainting for aspect changes: Runway’s expand and Luma’s reframe generate the missing canvas when 16:9 becomes 9:16 or 21:9. |
Feature names churn quarterly; the five steps do not. Learn the workflow once in an open pipeline and every hosted erase button becomes legible, including its failures.
What does inpainting handle well, and what defeats it?
Inpainting fills a hole with what the surrounding frames say should be there. Its success tracks one question: does clean background ever show through, in some frame, behind the thing being removed?
- Good fits: static and slow-moving background objects, logos and watermarks, boom mics and rig gear, small generation artifacts, texture repairs on walls and sky. The camera or the object moves enough that the model has seen the pixels it needs.
- Bad fits: large fast-moving subjects that dominate the frame, since little clean background is ever visible behind them; objects that occlude your hero character mid-performance; and anything whose removal changes the physics of the scene.
That last category is the one that catches people. Remove a lamp and the model fills the wall convincingly, but the light the lamp cast on that wall stays. Remove a figure beside a window and their reflection keeps standing there. The object is gone; its consequences are orphaned, and an audience notices before it can say why. The fix is at the mask step, not the fill step: extend the mask to the shadow, the reflection, and any surface the object visibly affected, and let the model rebuild the region with consistent physics.
Outpainting is the same trick, inverted
Reformatting a 16:9 master to 9:16 without butchering the composition is the mask turned inside out: the hole is the empty canvas outside the original frame, and the model extends the scene into it, consistently across time. Everything above applies unchanged. The generated regions need the same review as any patch, and they fail in the same places, faces and text first. Since the invented canvas sits at the frame edges, watch what enters and leaves the shot: an object that exits through a generated region has to exit plausibly.
The pre-ship checklist
- Orphaned shadows and reflections. Scrub the fixed shot asking one question: what did the removed thing touch? Glass, water, glossy floors, and rim light are where survivors hide.
- Seams at the mask edge. A visible boundary or smeared ring means the dilation was too tight. Re-run with a wider, feathered mask.
- Shimmer inside the patch. The fill is re-deciding per frame: use a video-native inpainter or widen its temporal window, and scrub at half speed, where wobble shows before it shows at full speed.
- A patch that is too clean. Generated fill has no grain, and a noiseless rectangle floats visibly on footage. Regrain the patch to match the plate.
- Flip-test the frames. Toggle between original and fixed frames at the worst moment of the shot. If you can find the patch in one flip, so can an audience watching it move.
These checks are minutes, and they are the difference between a repair and a tell. Judge them the way the taste rubric judges everything: in motion, at full size, against the question of whether anyone would notice if you had not told them.
Questions creatives ask
How do I remove an object from an AI-generated video without regenerating it? Inpaint it: segment the object on one frame, track the mask across all frames, dilate the mask so edge pixels fall inside it, fill the region with a temporally aware inpainting model, then composite the patch back over the original so untouched pixels stay identical. Hosted tools bundle these steps behind an erase or edit control; open pipelines expose each one.
Is video inpainting cheaper than regenerating the shot? Almost always, once you count expected attempts rather than single runs. A reroll resamples everything, so matching a shot you already mostly liked usually takes several attempts, each at full generation cost, and each one changes the performance and blocking. An inpaint costs one targeted pass and preserves everything outside the mask, including the edit you may have already cut the shot into.
Why does my video inpaint flicker? The patch was filled frame by frame. An image inpainter run per frame invents a slightly different fill each time, and the disagreement reads as shimmer inside the patch. Use a video-native inpainting model that propagates the fill across time, or one with a long enough temporal window to cover your shot.
How do I convert a 16:9 video to 9:16 without cropping? Outpaint it. Reframing tools treat the empty canvas above and below the original frame as a mask and generate plausible image there, consistent across time. It is the same operation as inpainting with the mask inverted: instead of filling a hole inside the frame, the model extends the frame. Check the seams and any generated area near faces and text, the same places inpainting fails.