How to Inpaint in Stable Diffusion A1111, A Detailed Guide with Inpainting Techniques to level up!
Summary
TLDRIn this video, Keyboard Alchemist demonstrates advanced inpainting techniques using Stable Diffusion. He shows how to add elements like a parrot on a girl's shoulder and transform her arm into a cyborg arm. The tutorial covers installing and using the Photopia extension, refining inpainted images, and leveraging Lora models for specific effects. Detailed steps, parameter settings, and troubleshooting tips are provided to ensure successful image manipulation. Viewers are encouraged to leave comments for future tutorial ideas and to like and subscribe for more content.
Takeaways
- 😀 You can inpaint new objects, like a parrot on a girl's shoulder, using stable diffusion techniques.
- 🦜 To install Photopia, go to extensions, click on install from URL, paste the Photopia GitHub link, and install.
- 🖼️ Use the Photopia extension in automatic1111 for photo editing similar to Photoshop.
- 🔧 After installing Photopia, keep your extensions up to date by checking for updates and restarting the UI.
- 💻 To edit an image in Photopia, import it from PNG info into image-to-image, regenerate it with low denoising strength, and send it to Photopia.
- ✂️ In Photopia, use the magnet selection tool to cut out the background of the object you want to inpaint, like a parrot.
- 🎨 Use inpainting techniques to add new elements to an image, ensuring the mask covers the area with some padding for context.
- 📈 Adjust denoising strength values to ensure the new object blends well with the original image without distortions.
- 🎩 You can also inpaint completely new elements, like a pirate hat, by using fill mode and higher denoising strength values.
- 🤖 To inpaint specific parts, like changing an arm to a robotic arm, download and use a LoRA model, adjusting parameters like denoising strength and sampling steps to achieve the desired result.
Q & A
What is the purpose of the video?
-The purpose of the video is to show viewers how to use inpainting techniques in Stable Diffusion to achieve various creative effects, such as adding a parrot to a girl's shoulder or changing a girl's arm into a cyborg arm.
What tool does the creator recommend for photo editing within the web UI?
-The creator recommends using the extension 'photopia,' which is similar to Photoshop and allows for image editing directly within the web UI.
How do you install the photopia extension?
-To install the photopia extension, go to extensions, click on the install from URL tab, copy the photopia GitHub link, paste it into the first field, click install, and wait for the installation to finish.
What are the key settings for inpainting a parrot on a girl's shoulder?
-Key settings for inpainting a parrot include setting masked content to 'original,' using the whole picture setting for inpaint area, and adjusting the denoising strength to a suitable level to merge the parrot seamlessly into the image.
How can you improve the inpainting results for a specific area?
-To improve inpainting results, you can use the XYZ plot script to test a range of denoising strength values and pick the best one, ensuring the new element integrates well without distorting the rest of the image.
What method does the creator use to add a pirate hat to the girl?
-The creator uses the inpaint sketch method, drawing a rough shape of the pirate hat with a mouse, and setting the masked content to 'fill' with a higher denoising strength to generate the hat.
What is a 'lora' and how is it used in the video?
-A 'lora' (low rank adaptation) is a training method to fine-tune a stable diffusion checkpoint for generating specific styles or objects. In the video, it is used to inpaint a robotic arm by downloading a 'real mechanical Parts lora' and applying it to the image.
How can you add a preview image to a lora for better organization?
-To add a preview image to a lora, save an image representative of the lora in the stable diffusion lora folder, renaming it with the same name as the lora file but with a '.preview.png' extension, and refresh the web UI to see the preview.
What settings are important for inpainting a robotic arm using a lora?
-Important settings for inpainting a robotic arm using a lora include setting masked content to 'fill,' inpaint area to 'whole picture,' resize mode to 'resize and fill,' and using a higher denoising strength for better integration.
How can you fix inconsistencies in an image after multiple inpainting steps?
-You can fix inconsistencies in an image by using latent upscaling in the image to image interface, which helps correct minor issues and ensure elements like skin tones are consistent across the image.
Outlines
🎨 Introduction to Inpainting and Viewer Questions
The host, Keyboard Alchemist, introduces the video and addresses viewer comments about inpainting techniques using stable diffusion. He mentions the possibility of inpainting a parrot on a girl's shoulder and changing a girl's arm into a cyborg arm. The video will demonstrate these techniques in detail, starting with how to inpaint a parrot using the Photopia extension.
📸 Installing and Using the Photopia Extension
The host guides viewers on how to install the Photopia extension for automatic1111, comparing it to Photoshop for web UI. After installation, he demonstrates regenerating an image with minimal changes and using Photopia to cut out the background of a parrot image. The parrot is then placed on the girl's shoulder, ready for inpainting.
🦜 Inpainting the Parrot
The process of inpainting the parrot is explained, emphasizing the importance of masking with padding to provide context to the AI model. Different settings such as masked content, inpaint area, and denoising strength are discussed. The host generates several images to find the best seed and denoising strength for a realistic parrot inpainting result.
👒 Adding a Pirate Hat Using Inpaint Sketch
The host demonstrates adding a pirate hat to the image using inpaint sketch. He sketches the hat and adjusts settings like masked content and denoising strength. The video includes an XY plot to illustrate how these settings affect the inpainted hat's appearance. The host uses latent upscaling to fix any inconsistencies in the final image.
🦾 Using Loras for Specific Inpainting: Robotic Arm
The video explains how to use Loras (low rank adaptation) to inpaint a specific part of an image, such as turning a girl's arm into a robotic arm. The host details downloading, installing, and using the real mechanical parts Lora, along with tips for organizing Loras and adding preview images. Parameters for inpainting the robotic arm are covered, along with an example of refining the inpainting mask for better results.
🏝 Cleaning Up Background and Final Touches
To remove unwanted elements from the background, the host demonstrates careful masking and inpainting with specific prompts and settings. The video also covers using inpaint sketch to add back missing elements like a tree trunk and final upscaling to improve image quality. The host concludes by encouraging viewers to like and subscribe for more content.
Mindmap
Keywords
💡inpainting
💡photopia
💡denoising strength
💡mask
💡dream shaper 7 model
💡batch generation
💡XYZ plot script
💡latent upscaling
💡Lora
💡seed
Highlights
Introduction to inpainting techniques with Stable Diffusion, including user-submitted ideas such as adding a parrot to a girl's shoulder and changing a girl's arm into a cyborg arm.
Downloading and installing the Photopia extension for automatic1111, which integrates Photoshop-like capabilities into the web UI.
Detailed steps for installing Photopia: go to extensions, click install from URL, paste the GitHub link, and restart the UI.
Using the PNG info to image to image feature to regenerate the original image with minimal changes using a denoising strength of 0.1.
Transferring images between image to image and Photopia to manipulate and edit images more effectively.
Using the magnet selection tool in Photopia to cut out the background of the parrot image and clean up jagged edges with the eraser tool.
Positioning the parrot on the girl's shoulder in Photopia and sending it back to the web UI for inpainting.
Setting masked content to original and inpaint area to whole picture, and generating images with a higher denoising strength to blend the parrot into the image.
Using the XYZ plot script to test different denoising strength values and find the best one for merging the parrot into the image.
Switching to only masked mode in inpainting to improve the parrot's appearance, using the whole picture mode for better integration.
Demonstrating a method for adding new elements, such as a pirate hat, by drawing a sketch in inpaint sketch mode and using a high denoising strength.
Adjusting denoising strength and sampling steps for detailed inpainting results, and using latent upscaling to fix inconsistencies.
Introduction to using luras for specific part inpainting, like transforming an arm into a robotic arm, by downloading and installing a real mechanical parts lora.
Setting up and using a lora in the web UI, including adding keywords to prompts and organizing loras with preview images.
Fine-tuning inpainting results by adjusting mask size, denoising strength, and sampling steps, and addressing background inconsistencies with additional inpainting and sketching.
Final steps for refining and upscaling the image to achieve a cohesive and polished final result.
Transcripts
hello everyone keyboard Alchemist here
and welcome back to another stable
diffusion tutorial I always try to
review and answer any comments left by
my viewers and a lot of times people
will leave some fun ideas for me to try
such as I see that you can inpaint
something that already exists in the
image but what if I want to inpaint a
parrot on the girl's
shoulder the answer is yes of course you
can do that or can I use luras to
inpaint a specific part of an image for
example changing the girl's arm into a
cyborg arm the answer is yes you can do
that as well then you might ask okay but
how do you actually do it today instead
of explaining in words alone I'm going
to show you how to use some inpainting
techniques to achieve these fun ideas
now there are a lot of great ideas that
are left in the comments and I can't
possibly cover them all in one video but
if you like this type of content let me
know in the comments and I will I'll try
to cover more of them in due time now
let's go to the main course for today
first let's make the parrot on shoulder
picture we need to download an extension
called photopia I know it's spelled like
Photo NP but people call it photopia I
don't know why in any case this is a
great photo editing tool for automatic
1111 it's just like having Photoshop or
built directly in your web UI and
you can pass an image back and forth
from image to image to photop
or vice
versa thank you to viewer Steve Warner
for pointing me to the extension a few
months back here are the steps to
install photopia go to extensions then
click on the install from URL tab copy
the photopia GitHub link I'll leave it
in the video descriptions and paste it
into the first field here then click
install and wait for the installation to
finish once the installation is finished
go back to the installed tab
I like to keep my extensions up to date
at all times yes I know it's a disease
so I'll click check for updates then
apply and restart UI once the web UI
restarts you will see the photopia tab
at the top okay now let's import the
original image from PNG info into image
to image we are going to get rid of the
positive prompt and only keeping the
negative
prompts
here I'm just trying to regenerate the
image with original
settings setting the denoising strength
very low to 0.1 because I just want the
image to be regenerated again with
almost no change at
[Music]
all once the image is regenerated click
on this red bird icon to send the image
to
photopia you can do the following steps
in Photoshop or if you are more
familiar with those tools but I think
photopia is just as easy and when you
are done you can click this button to
transfer the image back to image to
image in photopia what I want to do is
to just select the parrot and cut out
the background in this image there's
more than one way to skin this cat but
I'm going to use the magnet selection
tool it's okay if the edges are a bit
rough right now we don't need to get it
perfect when you are done with SEL in
the parrot go to select then
inverse then do a cut or control X this
warning message is telling me that the
image needs to be rasterized first so
then we can go to layer then
rasterize now we can do edit cut again
to get rid of the background of the bird
image then let's clean this up by
erasing some of these Jagged edges
deselect or press contrl D then select
the Eraser tool you don't have to be
super careful or neat about it closeing
enough would be just
fine oh and don't worry about this piece
of wood here in painting we'll take care
of all of these little
imperfections once you are done with the
cleanup move the parrot to a location on
her
shoulder then we can send this image
back to image to
image after getting back to the web UI
click on inpaint to send it to the
inpainting interface let's cover up the
parrot with a
mask notice that I am leaving some
amount of padding on the edges of the
parrot meaning my mask is larger than
the parrot this is done such that we are
giving the AI model some context with
the surrounding pixels I am using the
dream shaper 7 model to generate the
parrot and I will add MAA parrot to the
positive prompt the inpaint settings are
mostly default settings but pay special
attention to the following settings
masked content should be set to original
because we want the inpainting to stick
fairly close to what's underneath the
mask we are going to use the whole
picture setting for inpaint area this
doesn't mean that we will inpaint the
whole picture it just means that the
model will use the entire picture as
reference when it is generating the new
image I will generate a batch of four
images with random seeds be sure to set
the denoising strength to something a
bit
higher I'm using an arbitrary number of
0. 75 here for the first batch but in a
little bit I will show you how to pick
the appropriate denoising strength value
when you are done with all your
parameters click
generate out of this first batch of
images I can tell that we don't have the
right seed for our image here because I
don't want the parrot to face away from
the viewer so let's roll
this out of this second batch the first
image at least have the parrot facing
the right direction so let's use this
seed by clicking on the recycle icon and
continue on to the next
steps I'm going to use the XYZ plot
script to test out a range of denoising
strength values and pick out the best
one for this image oops I forgot to
reset the batch count back to one so I
interrupted the image generation set
batch count to one and then restarted
the
generation now if you recall from my
previous videos a low denoising strength
means fewer changes from the original
image and the higher the denoising
strength the more changes you will see
in this case we can see that denoising
from 0.15 to 0.35 were not enough to
merge the parrot into the image we can
still see remnants of the wooden Branch
under the parrot's Talons but as the
denoising strength increased the wooden
branch is gone and the parrot started to
look like it is a part of the picture
and not just a simple cut and paste job
within Photoshop then as we increase the
denoising strength past 0.65 we are
starting to see distortions in the
parrot so this basically means that we
can pick a denoising strength value that
is somewhere around 0.45 to
0.55 in this case I picked
0.45 turned off the XYZ plot script then
generated another batch of four images
with random
seeds here I would say if you liked the
image that you had with the previous
seed you don't have to generate again
with a random seed you can just reuse
the previous seed and generate your
image with 0.45 denois strength but here
I just wanted to see what another seed
would look like and it turned out pretty
good as
well I'm going to show you what inpaint
area only masked setting looks
[Music]
like you can see that while in the only
mask mode the model is only referencing
a portion of the area around the mask to
generate the new images as opposed to
referencing the whole
picture sometimes this will have some
drawbacks but in this case because we
already have a solid image of the parrot
underneath the masked area the result
will not stray far from it in fact I see
that the parrot's tailin are a bit
better with the only masked mode so I'm
going to use one of these images for my
next
steps for this next part we are going to
use a different method to impt something
new into the picture here is our lady
with a pirate on her shoulder but let's
say that she is a pirate captain and a
pirate Captain will need a hat befitting
of her status
first send this image with the parrot to
inpaint sketch then I'll use a brown
color to draw the pirate hat or at least
a general shape of the Hat because I
suck at drawing with a mouse note the
area of your sketch will influence what
the inpainted Hat will look like also I
didn't do it here but you can totally
add some additional patterns or colors
to this sketch to Define it a bit
more for this method here are the
important notes and settings put in
pirate hat in your positive prompt for
the masked content setting choose the
fill option and not the original option
since we want to add something
completely new to the picture when using
the fill option your denoising strength
need to be set to a higher value
otherwise you will just get some noise
here is a XY plot of denoising strength
on the x-axis and sampling steps on the
Y AIS to illustrate my point we can see
that is the denoising strength is too
low you will just get this blur in the
shape of our sketched hat only when the
denoising rength is 055 or higher does
the Hat start to materialize we can also
see that the sampling steps should not
be too low at 20 sampling steps we are
not getting enough details in the
Hat I like the image at 80 steps and
0.75 denoising
strength so these were the values that I
went with in order to make my picture
more cohesive and fix some of the minor
imperfections in it I will usually do a
latent upscaling in image to image it's
fast and easy and it takes care of minor
issues in the image and this also
answers another question that I got
previously someone said after inpainting
the skin tone is different doing the
latent upscaling would fix any
inconsistencies that were introduced by
all the inpainting steps and will make
sure the skin tone of the body matches
the
face if you like my videos please click
the like And subscribe buttons to help
support this channel your likes and
subscriptions Are Much appreciated and
help me to grow this Channel and
continue making quality content thank
you okay now let's talk about how to
inpaint a specific part of an image like
changing the girl's arm into a robotic
arm using
luras a Laura is short for low rank
adaptation it is a Training Method to
fine-tune a stable diffusion checkpoint
it is a lot smaller in size compared to
a checkpoint point and it can be trained
relatively quickly ideal for generating
consistent Styles objects characters and
in our case robot
arms to imp paint the robot arm we are
going to download this real mechanical
Parts Laura from civit AI I will leave a
link in the
descriptions after downloading the Laura
go to your automatic 1111 install folder
then the models folder and click on the
Laura subfolder if you don't see the
Laura subfolder for some reason you can
just create a new folder and rename itur
Laura then put the real Mech safe tensor
file into the Laura
folder come back to your web UI and
click refresh to see the newly
Incorporated Laura we are going to do a
few quick things here to make using the
Laura a bit easier click on the hammer
and wrench icon here and let's add a
short description to the Laura
properties
page usually in order to use a Laura you
will need to type a keyword or
activation word into your prompt
this is not always the case but a lot of
lauras do need a keyword to
work for this Laura the keyword is real
Mech this was noted in the civit AI
download page we are adding the keyword
in this field so that we don't have to
remember to add it to our prompts every
time we want to use this Laura there are
a lot of lauras out there and this
little feature is such a timesaver now
click
save to use the Laura simply place your
cursor in the positive prompt field then
click on the icon for the real Mech
Laura as we can see the Laura and the
keyword we just saved are now both added
to our prompt now it is good to
go one additional tip before we move on
you will likely download a ton of
different lauras down the line so in
order to organize your collection better
I recommend adding a preview image to
your Laura here is one method to do it
right click on an image that is
representative of the Laura then choose
save image as in the save image window
navigate to your stable diffusion Laura
folder the same folder where you have
saved this Laura then rename the image
using the exact same name as your Laura
file here I just copied the name of the
original Laura file but changed the file
extension to. preview.png note the image
has to be saved as a PNG file otherwise
it won't work go back to your web UI and
click the refresh button to see the
preview
image okay now back to our main
objective I will upload the image of the
lady in the blue dress and send it to
image to image in paint get rid of the
previous positive prompts and add the
real Mech Laura and the words robot arm
draw a mask at the location where the
arm should be as you can see I am just
defining the general region so it
doesn't have to be very precise to begin
with you can always refine it later
which is what I did here I didn't like
the initial in painting results so I
enlarge the mask a bit keep in mind the
size and shape of the inpainting Mask
will impact how your inpainting results
will look the important parameters for
this task are as follows set the masked
content to fill and set inpaint area to
whole picture set resize mode to resize
and fill use the width and height of the
original image use a random seed and set
the denoising strength to a higher
value once your parameters are set click
generate it's quite interesting if we
look at the snapshots of the image
generation process here on the right
hand side we can see that the AI model
is generating a new image that is
similar to our original image and then
merging the portion of the new image
that we have defined with the inpainting
mask to our original image therefore if
the newly generated image has a
different posture or stance than our
original image then the inpainting will
look a bit off such as in the case of
image number two here but if we look at
image number three here we can see that
the new and the old images matched up
pretty well so in this case we want to
recycle this seed and try to tweak our
other parameters a
bit I generated a XY plot of denoising
strength versus sampling steps to see
which combination provided the best
results and in this case I think that 20
steps and a denoising strength of 0.85
or higher was
good taking a look at the other images
with higher sampling steps even though
the robotic arms looked good the robotic
hands were a bit
wacky so I went with with 20 sampling
steps and one denoising strength now
that we have a good robot arm how do we
get rid of all this random stuff that
was in the background well I must say I
was a tad too aggressive with the
masking initially so maybe a Slimmer
mask would have prevented having so much
junk in the background but never fear we
can get rid of it just as easily with
some more inpainting first we will
carefully create a mask to cover all the
stuff that we don't want to get to the
smaller areas in the image we can zoom
in by holding down the alt ke key and
scrolling with the mouse wheel thank you
to quite a few viewers for telling me
about this setting the default key is
alt but this can be changed in the
settings menu under the canvas hotkeys
category plus there are some other
hotkeys that are useful in there since
the background in this image is a white
sandy beach I'm going to change the
prompt to white sandy beach then change
resize mode to resize and fill batch
count to four and seed to random and
generate we can see that image number
four here did a pretty good job with the
beach background but while the in
painting did change the background it
also cut off a part of the palm tree so
I'm going to add that back using inpaint
sketch let's send our newly inpainted
image to inpaint sketch and sketch out
the tree
trunk then change the positive prompt to
Palm Tree make sure we are using a high
enough step count I think 60 steps
should do fine
make sure we are using the same width
and height as the original image and a
higher denoising strength such as
0.85 then generate a batch of four
images with the random
[Music]
seed we see that image number four is
doing a pretty good job of generating
the tree trunk so we will take this
image and do a final l upscaling to
finish the job send this image to image
to image and get rid of all positive
prompts we can reduce the number of
sampling steps to 40 then increase the
width and height of the image by about
1.2x we don't need the increase to be
very large then Den noising strength to
0.3 which should be good enough but you
can always try out different values with
an XY plot and that's it the final image
looks pretty good I hope the inpainting
techniques that we covered today were
helpful for you don't forget to leave a
like And subscribe thanks for watching
and I will see you in the next
video
تصفح المزيد من مقاطع الفيديو ذات الصلة
SDXL Local LORA Training Guide: Unlimited AI Images of Yourself
BREAK Posing Limitations with Stable Diffusion!
We Create Realistic AI Influencer 2024 | Ai Consistent Characters & Clothes Change AI | DeFooocus AI
Text to Image generation using Stable Diffusion || HuggingFace Tutorial Diffusers Library
Create an Animated Scene with HTML and CSS: Adding a Tree and Moving Clouds
QR Code Makeup on ComfyUI using ControlNet Brightness Method. (MacBook Pro Intel i7/i9 )
5.0 / 5 (0 votes)