How Stable Diffusion Inpaint works

• VAE

The “function that redraws only the unmasked areas” is a crude explanation for the general public.

• It’s not a “don’t change anything that’s masked, just make the rest of it so that the boundaries add up.”

• If you zoom in, you can see the changing mesh pattern on the masked bench.

  • 

• It is natural that the masked area will also change, because there is not enough dimension in the latent space to hold that detailed information.

• Even if there were no mask and no noise, the image would still be shaky because it’s being restored from one-sixteenth of the information content.

---

This page is auto-translated from [/nishio/Stable Diffusion Inpaintの仕組み](https://scrapbox.io/nishio/Stable Diffusion Inpaintの仕組み) using DeepL. If you looks something interesting but the auto-translated English is not good enough to understand it, feel free to let me know at @nishio_en. I’m very happy to spread my thought to non-Japanese readers.