Inpainting is repainting one chosen part of an image so the new content matches everything around it. It's how photo editors erase a stranger from a vacation shot, swap a gray sofa for a green one, or fix a six-fingered hand while the rest of the picture stays put. Below is what inpainting is and where the idea came from, how AI inpainting and outpainting work, the Stable Diffusion settings that matter, the phone tools that do it for you, and how to fix a patch that doesn't blend.
What is inpainting?
Inpainting is a way of filling a missing or unwanted part of an image so the result looks whole. The word comes from art conservation, where a trained conservator fills in damaged, worn or missing parts of a painting or print so the work reads as complete again.
Computer scientists took the idea to digital photos in a 2000 SIGGRAPH paper titled "Image Inpainting" by Bertalmío, Sapiro, Caselles and Ballester. Their method copies the basic approach of professional restorers: you mark the region to fix, and the algorithm fills it from the surrounding picture without being told where the new detail should come from. Methods of that kind carry the lines and colors at the edge of the hole inward, so they can patch a scratch or a missing corner but can't invent anything new.
What is inpainting in AI?
AI inpainting uses a generative model to repaint the selected area from a text prompt, instead of only continuing the lines and colors at its edges. That is why it can add something new, like a dog on an empty lawn, rather than only erasing. Hugging Face's Diffusers guide lists both jobs: restoring a picture by removing defects, and swapping part of it for something new. In diffusion models it runs the same noise-removal process that makes whole images, limited to the masked area; the guide to how AI image generators work explains that process. The 2021 latent diffusion paper that led to Stable Diffusion set new records on inpainting benchmarks when it came out.
What an inpainting image needs
An AI inpainting job has three parts:
- The original image. The photo or generated picture you want to change.
- The mask. A black and white copy of the image marking the area to change. In Hugging Face's Diffusers library, white pixels are the area to repaint and black pixels are the area to keep.
- The prompt. A description of what should fill the white area, such as "a potted fern on the wooden floor".
Editors with a brush build the mask for you as you paint over the area. Editors that work from a typed instruction find the area from your words.
Inpainting and outpainting in AI, compared
Inpainting works inside the frame and outpainting works outside it. Both invent content from the surrounding image, which is why both need a careful look afterward.
| Feature | Inpainting | Outpainting |
|---|---|---|
| What changes | An area inside the picture | New area added beyond the edges |
| What you give it | The image, a mask or description, and a prompt | The image, the new canvas size, and a prompt |
| Typical uses | Removing objects, fixing hands, swapping one item | Turning a portrait photo into a wide scene, adding headroom or sky |
| Built-in phone example | Clean Up in Apple Photos | Extend in Apple Photos |
| Main risk | A patch that doesn't match the light or texture | Edges that repeat or invent odd details |
What is outpainting?
Outpainting is expanding an image past its original borders with new, matching content. OpenAI introduced it for DALL-E on August 31, 2022, describing it as continuing an image beyond its edges, in the same style, from a plain-language description. The same idea now ships on phones. On an iPhone running iOS 27 or later with Apple Intelligence, the Photos app's Extend tool pushes a photo past its original frame. It works on the iPhone 15 Pro, 15 Pro Max and iPhone 16 models or later. Apple Intelligence isn't available in every language or region, and Apple notes that usage limits may apply.
To extend a photo in Apple Photos:
- Open the photo in the Photos app and tap the Edit button.
- Tap Tools, then tap Extend.
- Pinch the photo, or drag it, to fill in more background around it. Dragging the frame's edge crops instead.
- Tap Extend, then tap the Done button, then Done again to close the photo. After saving, open Extend again and tap Reset to undo it.
What is inpainting in Stable Diffusion?
In Stable Diffusion, inpainting is a pipeline that takes an image, a mask and a prompt, and regenerates only the masked part. You can run it with a checkpoint trained specially for inpainting or with a regular one. The Diffusers documentation lists the settings that change the result most:
| Setting | What it does | When to change it |
|---|---|---|
| Strength | The amount of noise mixed into the masked area; higher values stray further from the original | Raise it to replace an object, lower it for a small touch-up |
| Guidance scale | How closely the new area follows the prompt | Raise it when the fill ignores your prompt |
| Negative prompt | Terms to steer away from, such as "blurry, poor details" | When the fill keeps adding something unwanted |
| Mask blur | Softens the mask's edge so the new area blends in | Raise it when you can see a hard outline |
| Padding mask crop | Crops around the mask, works on that area at higher resolution, then pastes it back | Small masks on large images that come out soft |
Diffusers points out one trade-off: inpainting-specific checkpoints blend the edge more naturally but are more likely to change the unmasked area too. If the rest of the image must stay exactly as it was, its apply_overlay option forces the unmasked area back to the original, at the cost of a less natural transition. For the terms that go in the negative prompt box, see the guide to negative prompts.
Mask-based vs describe-to-edit inpainting
There are two ways to tell a model which area to change: draw a mask, or describe the area in words. Both are inpainting, and each fits a different kind of edit.
- Mask-based. You paint over the area with a brush or selection tool. Midjourney's Vary Region, Apple's Clean Up and Stable Diffusion tools work this way.
- Describe-to-edit. You type what to change, and the model works out where. Google's image generation docs for developers call this semantic masking: you name the one element to change and say that everything else should stay exactly the same.
Even a drawn mask isn't always followed to the pixel. OpenAI's developer guide says its GPT Image models treat a mask as guidance and may not stick precisely to the outline you drew. Describe-to-edit works well on a phone and for objects that are easy to name ("the red car on the left"). A brush is better for an area that's hard to describe, like one patch of a busy crowd.
Inpainting on your phone
Your phone may already have inpainting built in. On an iPhone with Apple Intelligence, the Clean Up tool in Photos is inpainting with a finger for a brush: open a photo, tap the Edit button, then Tools, then Clean Up, and tap, brush or circle what should go. You can switch its processing between Auto, Fast and High Quality, and Reset undoes it after saving.
The step-by-step guide to removing someone from a photo covers Clean Up, Magic Eraser and Samsung's eraser, with device requirements. For signs, captions and date stamps, the guide to removing text from an image covers the extra problems text brings.
Tips for clean inpainting results
Each tip below targets a common reason a patch doesn't blend, usually a mask or a prompt that leaves the model guessing:
- Mask a little wider than the object. A tight mask leaves an outline of stray pixels. Include a small margin, and the object's shadow and reflection.
- Describe what belongs there, not what to remove. "Wet sand with small waves" gives the model something to paint. "No bucket" tells it nothing about the background.
- Match the light in your prompt. If the photo was taken at sunset, say "warm evening light" so the patch doesn't look flat.
- Work one area at a time. Two objects removed in one pass leave a bigger gap to invent. Remove one, check it, then do the next.
- Keep the resolution. Edit the original file, not a screenshot, and on large images use a setting like padding mask crop so the patch isn't softer than its surroundings.
- Watch straight lines and patterns. Tiles, fences and window frames must continue through the patched area. Follow each line across it.
Troubleshooting inpainting
| Problem | Likely cause | Fix |
|---|---|---|
| A visible outline around the patch | The mask edge is hard | Raise mask blur, or widen the mask slightly |
| The patch is blurrier than the photo | A small mask on a large image | Use padding mask crop, or edit at full resolution |
| The object comes back in a new form | The prompt describes the object | Describe the background that should replace it |
| Colors or light don't match | The prompt says nothing about the light | Add the time of day and light direction |
| Parts outside the mask changed | Inpainting checkpoints and prompt-based editors can redraw beyond it | Use apply_overlay in Diffusers, or add "keep everything else the same" |
| Outpainted edges repeat the same pattern | Too little context near the new edge | Extend in smaller steps and describe what the new area holds |
Remove or replace something without a mask
Toybox AI has no mask brush. Object Eraser and Smart Editor take a written instruction instead, so the describe-to-edit tips above apply.
- Open Object Eraser and sign in to your free account.
- Upload a JPG or PNG up to 10 MB, or take a photo with your phone's camera.
- In "What should we remove?", name the object, where it is, and anything attached to it, like a shadow, in 500 characters or fewer.
- Tap "Remove it" and give it about 30 seconds.
- Zoom in on the area and follow any straight lines through it.
These descriptions are written for the "What should we remove?" box:
The orange traffic cone on the sidewalk to the right of the front door, and its shadow on the pavement. Keep the door, the steps and the potted plants the same.The blue plastic bucket and shovel in the sand in the lower left corner. Fill the space with the same wet sand and small shells around it.Object Eraser currently costs 80 credits per photo and works on the Free plan. To replace something instead of removing it, use Smart Editor (Pro plan or higher, currently 85 credits per HD edit) and type the change in "What should change?", such as "Replace the gray sofa with a dark green velvet sofa. Keep the rug, walls and lighting the same." Smart Editor redraws the whole image with each edit, so small details outside your change can shift; check them against your original.
Neither tool has an outpainting feature. Picking a new shape in Smart Editor, such as Wide 16:9, redraws the whole picture in that shape, and the AI decides how to fit the scene, which may mean showing more at the edges, zooming in, or rearranging things. If you want more of the scene, say so in the instruction ("show more of the beach on both sides") and check the edges. For more on reshaping a photo, see changing an image's aspect ratio with AI, and if you need a true extension of the frame, Extend in Apple Photos is built for it.