How to Inpaint in Stable Diffusion A1111, A Detailed Guide with Inpainting Techniques to level up!

Summary

In this video, Keyboard Alchemist demonstrates advanced inpainting techniques using Stable Diffusion. He shows how to add elements like a parrot on a girl's shoulder and transform her arm into a cyborg arm. The tutorial covers installing and using the Photopia extension, refining inpainted images, and leveraging Lora models for specific effects. Detailed steps, parameter settings, and troubleshooting tips are provided to ensure successful image manipulation. Viewers are encouraged to leave comments for future tutorial ideas and to like and subscribe for more content.

Takeaways

• 😀 You can inpaint new objects, like a parrot on a girl's shoulder, using stable diffusion techniques.
• 🦜 To install Photopia, go to extensions, click on install from URL, paste the Photopia GitHub link, and install.
• 🖼️ Use the Photopia extension in automatic1111 for photo editing similar to Photoshop.
• 🔧 After installing Photopia, keep your extensions up to date by checking for updates and restarting the UI.
• 💻 To edit an image in Photopia, import it from PNG info into image-to-image, regenerate it with low denoising strength, and send it to Photopia.
• ✂️ In Photopia, use the magnet selection tool to cut out the background of the object you want to inpaint, like a parrot.
• 🎨 Use inpainting techniques to add new elements to an image, ensuring the mask covers the area with some padding for context.
• 📈 Adjust denoising strength values to ensure the new object blends well with the original image without distortions.
• 🎩 You can also inpaint completely new elements, like a pirate hat, by using fill mode and higher denoising strength values.
• 🤖 To inpaint specific parts, like changing an arm to a robotic arm, download and use a LoRA model, adjusting parameters like denoising strength and sampling steps to achieve the desired result.

Q & A

What is the purpose of the video?

The purpose of the video is to show viewers how to use inpainting techniques in Stable Diffusion to achieve various creative effects, such as adding a parrot to a girl's shoulder or changing a girl's arm into a cyborg arm.

What tool does the creator recommend for photo editing within the web UI?

The creator recommends using the extension 'photopia,' which is similar to Photoshop and allows for image editing directly within the web UI.

How do you install the photopia extension?

To install the photopia extension, go to extensions, click on the install from URL tab, copy the photopia GitHub link, paste it into the first field, click install, and wait for the installation to finish.

What are the key settings for inpainting a parrot on a girl's shoulder?

Key settings for inpainting a parrot include setting masked content to 'original,' using the whole picture setting for inpaint area, and adjusting the denoising strength to a suitable level to merge the parrot seamlessly into the image.

How can you improve the inpainting results for a specific area?

To improve inpainting results, you can use the XYZ plot script to test a range of denoising strength values and pick the best one, ensuring the new element integrates well without distorting the rest of the image.

What method does the creator use to add a pirate hat to the girl?

The creator uses the inpaint sketch method, drawing a rough shape of the pirate hat with a mouse, and setting the masked content to 'fill' with a higher denoising strength to generate the hat.

What is a 'lora' and how is it used in the video?

A 'lora' (low rank adaptation) is a training method to fine-tune a stable diffusion checkpoint for generating specific styles or objects. In the video, it is used to inpaint a robotic arm by downloading a 'real mechanical Parts lora' and applying it to the image.

How can you add a preview image to a lora for better organization?

To add a preview image to a lora, save an image representative of the lora in the stable diffusion lora folder, renaming it with the same name as the lora file but with a '.preview.png' extension, and refresh the web UI to see the preview.

What settings are important for inpainting a robotic arm using a lora?

Important settings for inpainting a robotic arm using a lora include setting masked content to 'fill,' inpaint area to 'whole picture,' resize mode to 'resize and fill,' and using a higher denoising strength for better integration.

How can you fix inconsistencies in an image after multiple inpainting steps?

You can fix inconsistencies in an image by using latent upscaling in the image to image interface, which helps correct minor issues and ensure elements like skin tones are consistent across the image.

Outlines

00:00

🎨 Introduction to Inpainting and Viewer Questions

The host, Keyboard Alchemist, introduces the video and addresses viewer comments about inpainting techniques using stable diffusion. He mentions the possibility of inpainting a parrot on a girl's shoulder and changing a girl's arm into a cyborg arm. The video will demonstrate these techniques in detail, starting with how to inpaint a parrot using the Photopia extension.

05:01

📸 Installing and Using the Photopia Extension

The host guides viewers on how to install the Photopia extension for automatic1111, comparing it to Photoshop for web UI. After installation, he demonstrates regenerating an image with minimal changes and using Photopia to cut out the background of a parrot image. The parrot is then placed on the girl's shoulder, ready for inpainting.

10:04

🦜 Inpainting the Parrot

The process of inpainting the parrot is explained, emphasizing the importance of masking with padding to provide context to the AI model. Different settings such as masked content, inpaint area, and denoising strength are discussed. The host generates several images to find the best seed and denoising strength for a realistic parrot inpainting result.

15:05

👒 Adding a Pirate Hat Using Inpaint Sketch

The host demonstrates adding a pirate hat to the image using inpaint sketch. He sketches the hat and adjusts settings like masked content and denoising strength. The video includes an XY plot to illustrate how these settings affect the inpainted hat's appearance. The host uses latent upscaling to fix any inconsistencies in the final image.

🦾 Using Loras for Specific Inpainting: Robotic Arm

The video explains how to use Loras (low rank adaptation) to inpaint a specific part of an image, such as turning a girl's arm into a robotic arm. The host details downloading, installing, and using the real mechanical parts Lora, along with tips for organizing Loras and adding preview images. Parameters for inpainting the robotic arm are covered, along with an example of refining the inpainting mask for better results.

🏝 Cleaning Up Background and Final Touches

To remove unwanted elements from the background, the host demonstrates careful masking and inpainting with specific prompts and settings. The video also covers using inpaint sketch to add back missing elements like a tree trunk and final upscaling to improve image quality. The host concludes by encouraging viewers to like and subscribe for more content.

Keywords

💡inpainting

Inpainting is a technique used to fill in or modify parts of an image. In the video, the creator demonstrates how to use inpainting to add a parrot to a girl's shoulder and change the girl's arm into a cyborg arm. This process involves using software tools to seamlessly integrate new elements into an existing image.

💡photopia

Photopia is an extension for automatic 1111 that provides photo editing capabilities similar to Photoshop within a web UI. The video explains how to install and use Photopia to edit images, such as selecting and cutting out the background of a parrot image before integrating it into another image.

💡denoising strength

Denoising strength is a parameter that controls the extent to which the original image is altered during the inpainting process. In the video, different denoising strengths are tested to find the optimal setting for merging the parrot into the image without distorting it. Higher values result in more significant changes.

💡mask

A mask in image editing is used to define the area of the image that should be modified. The video shows how to create and use masks to cover the area where the parrot will be placed, ensuring that the inpainting tool has enough context from the surrounding pixels to generate a cohesive image.

💡dream shaper 7 model

The Dream Shaper 7 model is an AI model used for generating images in the inpainting process. In the video, this model is employed to generate a parrot in the inpainted image, with specific prompts and settings to achieve the desired result.

💡batch generation

Batch generation refers to generating multiple versions of an image simultaneously using different random seeds. The video demonstrates this process to find the best image where the parrot is facing the right direction and integrates well with the background.

💡XYZ plot script

The XYZ plot script is a tool used to test various parameters, such as denoising strength and sampling steps, to find the best combination for image generation. The video uses this script to determine the optimal settings for integrating the parrot into the image.

💡latent upscaling

Latent upscaling is a technique to increase the resolution of an image while maintaining its quality. In the video, this method is used to fix minor inconsistencies and improve the overall cohesion of the inpainted image, ensuring that skin tones and other details match.

💡Lora

Lora (Low Rank Adaptation) is a training method to fine-tune stable diffusion checkpoints for generating specific styles or objects. The video explains how to use a Lora to change a girl's arm into a robotic arm by downloading and applying a specific Lora model designed for mechanical parts.

💡seed

A seed in image generation is a starting point for the random number generator that determines the output of the AI model. The video shows how different seeds can produce different versions of the inpainted image and how selecting the right seed is crucial for achieving the desired result.

Highlights

Introduction to inpainting techniques with Stable Diffusion, including user-submitted ideas such as adding a parrot to a girl's shoulder and changing a girl's arm into a cyborg arm.

Downloading and installing the Photopia extension for automatic1111, which integrates Photoshop-like capabilities into the web UI.

Detailed steps for installing Photopia: go to extensions, click install from URL, paste the GitHub link, and restart the UI.

Using the PNG info to image to image feature to regenerate the original image with minimal changes using a denoising strength of 0.1.

Transferring images between image to image and Photopia to manipulate and edit images more effectively.

Using the magnet selection tool in Photopia to cut out the background of the parrot image and clean up jagged edges with the eraser tool.

Positioning the parrot on the girl's shoulder in Photopia and sending it back to the web UI for inpainting.

Setting masked content to original and inpaint area to whole picture, and generating images with a higher denoising strength to blend the parrot into the image.

Using the XYZ plot script to test different denoising strength values and find the best one for merging the parrot into the image.

Switching to only masked mode in inpainting to improve the parrot's appearance, using the whole picture mode for better integration.

Demonstrating a method for adding new elements, such as a pirate hat, by drawing a sketch in inpaint sketch mode and using a high denoising strength.

Adjusting denoising strength and sampling steps for detailed inpainting results, and using latent upscaling to fix inconsistencies.

Introduction to using luras for specific part inpainting, like transforming an arm into a robotic arm, by downloading and installing a real mechanical parts lora.

Setting up and using a lora in the web UI, including adding keywords to prompts and organizing loras with preview images.

Fine-tuning inpainting results by adjusting mask size, denoising strength, and sampling steps, and addressing background inconsistencies with additional inpainting and sketching.

Final steps for refining and upscaling the image to achieve a cohesive and polished final result.

Transcripts

hello everyone keyboard Alchemist here

and welcome back to another stable

diffusion tutorial I always try to

review and answer any comments left by

my viewers and a lot of times people

will leave some fun ideas for me to try

such as I see that you can inpaint

something that already exists in the

image but what if I want to inpaint a

parrot on the girl's

shoulder the answer is yes of course you

can do that or can I use luras to

inpaint a specific part of an image for

example changing the girl's arm into a

cyborg arm the answer is yes you can do

that as well then you might ask okay but

how do you actually do it today instead

of explaining in words alone I'm going

to show you how to use some inpainting

techniques to achieve these fun ideas

now there are a lot of great ideas that

are left in the comments and I can't

possibly cover them all in one video but

if you like this type of content let me

know in the comments and I will I'll try

to cover more of them in due time now

let's go to the main course for today

first let's make the parrot on shoulder

picture we need to download an extension

called photopia I know it's spelled like

Photo NP but people call it photopia I

don't know why in any case this is a

great photo editing tool for automatic

1111 it's just like having Photoshop or

built directly in your web UI and

you can pass an image back and forth

from image to image to photop

or vice

versa thank you to viewer Steve Warner

for pointing me to the extension a few

months back here are the steps to

install photopia go to extensions then

click on the install from URL tab copy

the photopia GitHub link I'll leave it

in the video descriptions and paste it

into the first field here then click

install and wait for the installation to

finish once the installation is finished

go back to the installed tab

I like to keep my extensions up to date

at all times yes I know it's a disease

so I'll click check for updates then

apply and restart UI once the web UI

restarts you will see the photopia tab

at the top okay now let's import the

original image from PNG info into image

to image we are going to get rid of the

positive prompt and only keeping the

negative

prompts

here I'm just trying to regenerate the

image with original

settings setting the denoising strength

very low to 0.1 because I just want the

image to be regenerated again with

almost no change at

[Music]

all once the image is regenerated click

on this red bird icon to send the image

to

photopia you can do the following steps

in Photoshop or if you are more

familiar with those tools but I think

photopia is just as easy and when you

are done you can click this button to

transfer the image back to image to

image in photopia what I want to do is

to just select the parrot and cut out

the background in this image there's

more than one way to skin this cat but

I'm going to use the magnet selection

tool it's okay if the edges are a bit

rough right now we don't need to get it

perfect when you are done with SEL in

the parrot go to select then

inverse then do a cut or control X this

warning message is telling me that the

image needs to be rasterized first so

then we can go to layer then

rasterize now we can do edit cut again

to get rid of the background of the bird

image then let's clean this up by

erasing some of these Jagged edges

deselect or press contrl D then select

the Eraser tool you don't have to be

super careful or neat about it closeing

enough would be just

fine oh and don't worry about this piece

of wood here in painting we'll take care

of all of these little

imperfections once you are done with the

cleanup move the parrot to a location on

her

shoulder then we can send this image

back to image to

image after getting back to the web UI

click on inpaint to send it to the

inpainting interface let's cover up the

parrot with a

mask notice that I am leaving some

amount of padding on the edges of the

parrot meaning my mask is larger than

the parrot this is done such that we are

giving the AI model some context with

the surrounding pixels I am using the

dream shaper 7 model to generate the

parrot and I will add MAA parrot to the

positive prompt the inpaint settings are

mostly default settings but pay special

attention to the following settings

masked content should be set to original

because we want the inpainting to stick

fairly close to what's underneath the

mask we are going to use the whole

picture setting for inpaint area this

doesn't mean that we will inpaint the

whole picture it just means that the

model will use the entire picture as

reference when it is generating the new

image I will generate a batch of four

images with random seeds be sure to set

the denoising strength to something a

bit

higher I'm using an arbitrary number of

0. 75 here for the first batch but in a

little bit I will show you how to pick

the appropriate denoising strength value

when you are done with all your

parameters click

generate out of this first batch of

images I can tell that we don't have the

right seed for our image here because I

don't want the parrot to face away from

the viewer so let's roll

this out of this second batch the first

image at least have the parrot facing

the right direction so let's use this

seed by clicking on the recycle icon and

continue on to the next

steps I'm going to use the XYZ plot

script to test out a range of denoising

strength values and pick out the best

one for this image oops I forgot to

reset the batch count back to one so I

interrupted the image generation set

batch count to one and then restarted

the

generation now if you recall from my

previous videos a low denoising strength

means fewer changes from the original

image and the higher the denoising

strength the more changes you will see

in this case we can see that denoising

from 0.15 to 0.35 were not enough to

merge the parrot into the image we can

still see remnants of the wooden Branch

under the parrot's Talons but as the

denoising strength increased the wooden

branch is gone and the parrot started to

look like it is a part of the picture

and not just a simple cut and paste job

within Photoshop then as we increase the

denoising strength past 0.65 we are

starting to see distortions in the

parrot so this basically means that we

can pick a denoising strength value that

is somewhere around 0.45 to

0.55 in this case I picked

0.45 turned off the XYZ plot script then

generated another batch of four images

with random

seeds here I would say if you liked the

image that you had with the previous

seed you don't have to generate again

with a random seed you can just reuse

the previous seed and generate your

image with 0.45 denois strength but here

I just wanted to see what another seed

would look like and it turned out pretty

good as

well I'm going to show you what inpaint

area only masked setting looks

[Music]

like you can see that while in the only

mask mode the model is only referencing

a portion of the area around the mask to

generate the new images as opposed to

referencing the whole

picture sometimes this will have some

drawbacks but in this case because we

already have a solid image of the parrot

underneath the masked area the result

will not stray far from it in fact I see

that the parrot's tailin are a bit

better with the only masked mode so I'm

going to use one of these images for my

next

steps for this next part we are going to

use a different method to impt something

new into the picture here is our lady

with a pirate on her shoulder but let's

say that she is a pirate captain and a

pirate Captain will need a hat befitting

of her status

first send this image with the parrot to

inpaint sketch then I'll use a brown

color to draw the pirate hat or at least

a general shape of the Hat because I

suck at drawing with a mouse note the

area of your sketch will influence what

the inpainted Hat will look like also I

didn't do it here but you can totally

add some additional patterns or colors

to this sketch to Define it a bit

more for this method here are the

important notes and settings put in

pirate hat in your positive prompt for

the masked content setting choose the

fill option and not the original option

since we want to add something

completely new to the picture when using

the fill option your denoising strength

need to be set to a higher value

otherwise you will just get some noise

here is a XY plot of denoising strength

on the x-axis and sampling steps on the

Y AIS to illustrate my point we can see

that is the denoising strength is too

low you will just get this blur in the

shape of our sketched hat only when the

denoising rength is 055 or higher does

the Hat start to materialize we can also

see that the sampling steps should not

be too low at 20 sampling steps we are

not getting enough details in the

Hat I like the image at 80 steps and

0.75 denoising

strength so these were the values that I

went with in order to make my picture

more cohesive and fix some of the minor

imperfections in it I will usually do a

latent upscaling in image to image it's

fast and easy and it takes care of minor

issues in the image and this also

answers another question that I got

previously someone said after inpainting

the skin tone is different doing the

latent upscaling would fix any

inconsistencies that were introduced by

all the inpainting steps and will make

sure the skin tone of the body matches

the

face if you like my videos please click

the like And subscribe buttons to help

support this channel your likes and

subscriptions Are Much appreciated and

help me to grow this Channel and

continue making quality content thank

you okay now let's talk about how to

inpaint a specific part of an image like

changing the girl's arm into a robotic

arm using

luras a Laura is short for low rank

adaptation it is a Training Method to

fine-tune a stable diffusion checkpoint

it is a lot smaller in size compared to

a checkpoint point and it can be trained

relatively quickly ideal for generating

consistent Styles objects characters and

in our case robot

arms to imp paint the robot arm we are

going to download this real mechanical

Parts Laura from civit AI I will leave a

link in the

descriptions after downloading the Laura

go to your automatic 1111 install folder

then the models folder and click on the

Laura subfolder if you don't see the

Laura subfolder for some reason you can

just create a new folder and rename itur

Laura then put the real Mech safe tensor

file into the Laura

folder come back to your web UI and

click refresh to see the newly

Incorporated Laura we are going to do a

few quick things here to make using the

Laura a bit easier click on the hammer

and wrench icon here and let's add a

short description to the Laura

properties

page usually in order to use a Laura you

will need to type a keyword or

activation word into your prompt

this is not always the case but a lot of

lauras do need a keyword to

work for this Laura the keyword is real

Mech this was noted in the civit AI

download page we are adding the keyword

in this field so that we don't have to

remember to add it to our prompts every

time we want to use this Laura there are

a lot of lauras out there and this

little feature is such a timesaver now

click

save to use the Laura simply place your

cursor in the positive prompt field then

click on the icon for the real Mech

Laura as we can see the Laura and the

keyword we just saved are now both added

to our prompt now it is good to

go one additional tip before we move on

you will likely download a ton of

different lauras down the line so in

order to organize your collection better

I recommend adding a preview image to

your Laura here is one method to do it

right click on an image that is

representative of the Laura then choose

save image as in the save image window

navigate to your stable diffusion Laura

folder the same folder where you have

saved this Laura then rename the image

using the exact same name as your Laura

file here I just copied the name of the

original Laura file but changed the file

extension to. preview.png note the image

has to be saved as a PNG file otherwise

it won't work go back to your web UI and

click the refresh button to see the

preview

image okay now back to our main

objective I will upload the image of the

lady in the blue dress and send it to

image to image in paint get rid of the

previous positive prompts and add the

real Mech Laura and the words robot arm

draw a mask at the location where the

arm should be as you can see I am just

defining the general region so it

doesn't have to be very precise to begin

with you can always refine it later

which is what I did here I didn't like

the initial in painting results so I

enlarge the mask a bit keep in mind the

size and shape of the inpainting Mask

will impact how your inpainting results

will look the important parameters for

this task are as follows set the masked

content to fill and set inpaint area to

whole picture set resize mode to resize

and fill use the width and height of the

original image use a random seed and set

the denoising strength to a higher

value once your parameters are set click

generate it's quite interesting if we

look at the snapshots of the image

generation process here on the right

hand side we can see that the AI model

is generating a new image that is

similar to our original image and then

merging the portion of the new image

that we have defined with the inpainting

mask to our original image therefore if

the newly generated image has a

different posture or stance than our

original image then the inpainting will

look a bit off such as in the case of

image number two here but if we look at

image number three here we can see that

the new and the old images matched up

pretty well so in this case we want to

recycle this seed and try to tweak our

other parameters a

bit I generated a XY plot of denoising

strength versus sampling steps to see

which combination provided the best

results and in this case I think that 20

steps and a denoising strength of 0.85

or higher was

good taking a look at the other images

with higher sampling steps even though

the robotic arms looked good the robotic

hands were a bit

wacky so I went with with 20 sampling

steps and one denoising strength now

that we have a good robot arm how do we

get rid of all this random stuff that

was in the background well I must say I

was a tad too aggressive with the

masking initially so maybe a Slimmer

mask would have prevented having so much

junk in the background but never fear we

can get rid of it just as easily with

some more inpainting first we will

carefully create a mask to cover all the

stuff that we don't want to get to the

smaller areas in the image we can zoom

in by holding down the alt ke key and

scrolling with the mouse wheel thank you

to quite a few viewers for telling me

about this setting the default key is

alt but this can be changed in the

settings menu under the canvas hotkeys

category plus there are some other

hotkeys that are useful in there since

the background in this image is a white

sandy beach I'm going to change the

prompt to white sandy beach then change

resize mode to resize and fill batch

count to four and seed to random and

generate we can see that image number

four here did a pretty good job with the

beach background but while the in

painting did change the background it

also cut off a part of the palm tree so

I'm going to add that back using inpaint

sketch let's send our newly inpainted

image to inpaint sketch and sketch out

the tree

trunk then change the positive prompt to

Palm Tree make sure we are using a high

enough step count I think 60 steps

should do fine

make sure we are using the same width

and height as the original image and a

higher denoising strength such as

0.85 then generate a batch of four

images with the random

[Music]

seed we see that image number four is

doing a pretty good job of generating

the tree trunk so we will take this

image and do a final l upscaling to

finish the job send this image to image

to image and get rid of all positive

prompts we can reduce the number of

sampling steps to 40 then increase the

width and height of the image by about

1.2x we don't need the increase to be

very large then Den noising strength to

0.3 which should be good enough but you

can always try out different values with

an XY plot and that's it the final image

looks pretty good I hope the inpainting

techniques that we covered today were

helpful for you don't forget to leave a

like And subscribe thanks for watching

and I will see you in the next

video