Civitai Beginners Guide To AI Art // #1 Core Concepts

Summary

This video serves as a beginner's guide to AI art, particularly focusing on Stable Diffusion. Hosted by Tyler, it covers essential AI art concepts, software installation, and terminology such as text-to-image, image-to-image, inpainting, upscaling, and more. The guide explains tools like ControlNet, LoRAs, and extensions like Automatic 1111, along with the importance of models and checkpoints. Additionally, it discusses the significance of prompts, upscaling, and training data. The video aims to equip viewers with the knowledge needed to create AI-generated images locally on their machines.

Takeaways

• 🎨 The video introduces the basics of AI art, focusing on generating images using Stable Diffusion software.
• 📜 Key AI art concepts like text-to-image, image-to-image, and batch image-to-image are explained, helping users understand how to generate images.
• 🖼️ The video introduces 'inpainting,' which involves using a painted mask area to add or remove objects from images, similar to Photoshop's generative fill.
• 🎥 Concepts like text-to-video (text to vid) and video-to-video (vid to vid) are introduced for generating videos using text prompts or transforming existing videos.
• 📝 'Prompt' refers to the text input that guides the AI in generating images, while 'negative prompt' tells the AI what to exclude from the image.
• 📈 'Upscaling' enhances low-resolution images or videos into higher resolution using built-in AI models or external software like Topaz Photo AI.
• 🛠️ Models, also called checkpoints, are critical in AI art generation. Different models produce different image styles, and users are advised to choose models wisely.
• 🔐 Safe tensor files are preferred over ckpt files for models, as they are more secure and less susceptible to malicious code.
• 🖌️ ControlNet, an important extension in Stable Diffusion, is used to process image structures, depth, and poses, essential for tasks like image-to-image generation.
• 🧠 Tools like Deorum and techniques like Enhanced Super-Resolution GAN (ESRGAN) and AnimateDiff are introduced for enhancing image resolution and adding motion to text-to-image generations.

Q & A

What is AI art, and how is it generated?

AI art refers to the creation of images, videos, or other media using artificial intelligence. It's typically generated by feeding a text prompt into a model like Stable Diffusion, which creates an image based on the input text.

What is Stable Diffusion, and what is its role in AI art generation?

Stable Diffusion is a machine learning model used for generating images based on text inputs. It can also handle tasks like transforming existing images, adding elements to images, and generating videos. It serves as the engine behind much of the AI art creation process.

What is the difference between 'text-to-image' and 'image-to-image' generation?

Text-to-image generation creates a visual output solely based on a text prompt, whereas image-to-image takes an existing image as input, using the text to alter or enhance the original image.

What is inpainting, and how is it used in AI art?

Inpainting involves masking specific areas of an image and using AI to add or remove elements from those parts. It's similar to Photoshop’s generative fill but is done locally using software like Stable Diffusion.

What are prompts and negative prompts?

Prompts are the text inputs that describe what you want the AI to generate, while negative prompts describe what you do not want to appear in the final image, helping refine the result.

What is upscaling, and why is it important in AI art?

Upscaling is the process of increasing the resolution of an image, making it larger and more detailed. It is often the final step before sharing AI-generated images to ensure they have high quality.

What are checkpoints and safe tensors in the context of Stable Diffusion?

Checkpoints, now often called models, are files that store the machine learning model used for generating images. Safe tensors are a safer version of checkpoints, as they are less prone to malicious code.

What is a 'LoRA,' and how is it used in Stable Diffusion?

LoRA stands for Low-Rank Adaptation, which is a model trained on a small, specific dataset to generate images with a particular character, style, or concept. LoRAs are used to fine-tune outputs in a targeted way.

What are 'textual inversions' and 'embeddings'?

Textual inversions and embeddings are smaller datasets used to fine-tune specific aspects of an image, such as fixing common issues like incorrect hand or eye shapes, or focusing on details like specific objects or faces.

What is a VAE (Variational Autoencoder), and how does it affect image generation?

A VAE is a file used to enhance the quality of generated images by improving details, colors, and sharpness. Some models have VAEs built-in, while others require you to add one manually to get the best results.

Outlines

00:00

🎨 Introduction to AI Art and Stable Diffusion

Tyler introduces the official beginner’s guide to AI art, covering key concepts and terminology related to AI art and stable diffusion. This video series aims to teach viewers how to generate AI images on their local machines, navigate the necessary software, and download resources from coty.com. Tyler outlines the core topics, including text-to-image and image-to-image generation, the importance of prompts, and commonly used software like automatic1111, Focus, Comfy UI, and Easy Diffusion.

05:01

🖼️ Types of AI Image Generation

This section breaks down the different types of AI image generation, beginning with text-to-image, which involves creating images from a textual prompt. Tyler also explains image-to-image and batch image-to-image, where an existing photo serves as a base for further modifications using AI. He introduces the concept of 'control net' for enhancing image generation and describes inpainting, text-to-video, video-to-video, and how prompts and negative prompts shape the final AI-generated outputs.

10:01

📈 Upscaling and Final Touches in AI Art

Tyler discusses the process of upscaling low-resolution images into higher resolution formats, commonly done with built-in AI models or external software like Topaz Photo AI. He emphasizes that upscaling is typically the last step before sharing AI-generated images. This segment also mentions that upscaling enhances the final quality and prepares the image for online sharing, ensuring the best possible resolution and visual appeal.

🧠 AI Models and Checkpoints

Here, Tyler introduces the concept of AI models and checkpoints, explaining their role in generating outputs. Checkpoints or models are built from millions of images and shape the style of the final image. He also explains the difference between checkpoint files (CKPT) and safe tensor files, advising the use of safe tensor files to avoid malicious code. The importance of training data and the evolution from Stable Diffusion 1.5 to Stable Diffusion XL 1.0 is also highlighted, showcasing how different datasets improve AI model quality.

🤖 Lora and Textual Inversions in AI Models

Tyler delves into specialized AI models like Lora, which is trained on a smaller dataset for specific purposes, such as generating images with a particular character or style. He also explains textual inversions and embeddings, which are used to fix specific image features like hands and eyes. Additionally, Tyler touches on VAEs (Variational Autoencoders) that enhance image detail and vibrancy, ensuring better final results in terms of color and sharpness.

🔧 Essential AI Extensions for Stable Diffusion

This section covers essential extensions for Stable Diffusion, starting with Control Nets, which are used to define image structures and positions. Tyler explains that Control Nets are vital for tasks like image-to-image and video-to-video generation. He also introduces Deorum, a community known for building AI tools, and explains the role of Deorum's popular automatic1111 extension for smooth video generation. Other notable tools, like the enhanced super-resolution GAN (ESRGAN) for upscaling and AnimateDiff for adding motion to images, are briefly discussed.

📚 Conclusion and Glossary Resources

Tyler wraps up the video by summarizing the core concepts and terminology covered throughout the guide. He encourages viewers to refer to the Stable Diffusion glossary on coty.com’s education hub for further learning and clarification on any terms they may encounter while using AI art tools. The video series aims to provide a strong foundation for beginners to confidently navigate AI art creation.

Keywords

💡Text to Image

Text to Image refers to generating an image from a descriptive text prompt using AI. In the video, it is explained as the most common method of AI art generation, where you input text that describes the desired image and the AI creates the image based on that input. This is the foundation of tools like Stable Diffusion.

💡Image to Image

Image to Image is the process of taking an existing image and using it as a base to generate a new image with AI, based on a text prompt. This technique allows users to modify or enhance existing images, often with the help of additional tools like ControlNet. The script explains how it’s used for refining a specific photo or batch of images.

💡Stable Diffusion

Stable Diffusion is an AI model used for generating images from text prompts. It is the core technology discussed in the video, and it operates by interpreting prompts and creating images through diffusion models. The script mentions versions like SD 1.5 and XL 1.0, highlighting how these models differ in training data and output quality.

💡ControlNet

ControlNet is an extension for Stable Diffusion that adds more control to the image generation process by reading and manipulating structures like depth and character positions in images. The video emphasizes how it is crucial for tasks like Image to Image and Video to Video generation, allowing more precise control over the outputs.

💡Prompt

A Prompt is the text input given to the AI model to instruct it on what kind of image to generate. In the context of the video, prompts are essential for all forms of AI image generation. The script describes how detailed prompts can shape the outcome and how there are positive and negative prompts, the latter telling the AI what to avoid in the image.

💡Upscaling

Upscaling is the process of increasing the resolution of an image, making a low-resolution image clearer and more detailed. The video describes this as one of the final steps in image generation, often using external software or AI tools, to ensure images are ready for sharing or printing. It is an essential step before publishing AI-generated images.

💡Checkpoints

Checkpoints, also called models, are files that contain the machine learning algorithms used in Stable Diffusion to generate images. The script explains that they are the result of training on large datasets of images, and the choice of checkpoint/model determines the style and quality of the generated output.

💡Safe Tensors

Safe Tensors are a newer file format for Stable Diffusion models, designed to be safer by minimizing the risk of malicious code compared to older checkpoint files (CKPT). The video suggests users opt for Safe Tensor files when downloading models to avoid security risks.

💡LORA

LORA, or Low-Rank Adaptation, is a technique for training smaller, more focused AI models that specialize in generating specific styles, characters, or concepts. The video explains that LORA models are often used to add very particular traits to image outputs, such as generating a specific anime character or replicating a certain artistic style.

💡Negative Prompt

A Negative Prompt is the opposite of a regular prompt—it tells the AI what to exclude from the image. The video highlights its importance in controlling the generation process by ensuring certain unwanted elements or styles don’t appear in the final image, refining the accuracy of outputs.

Highlights

Introduction to AI art and stable diffusion, guiding beginners from zero to generating their first AI images.

Overview of the core concepts and terminology of AI art, including text-to-image, image-to-image, and batch image generation.

Explanation of image generation methods such as text-to-image and image-to-image using reference photos or existing images.

Introduction to inpainting, the process of modifying parts of an image using a painted mask, similar to Photoshop's generative fill.

Text-to-video and video-to-video generation explained, using text prompts to create videos or alter existing video footage.

Understanding prompts and negative prompts: the core text inputs used to guide AI in generating images and excluding unwanted elements.

Overview of upscaling, transforming low-resolution images (e.g., 512x512) into higher resolution versions (e.g., 1080x1080).

Introduction to models and checkpoints, including different model types, and how they influence image style and quality.

Explanation of safe tensor files replacing checkpoints (ckpt) to reduce the risk of downloading malicious content.

Description of training data sets like LAION 5B and how they are used to train AI models for image generation.

Introduction to Stable Diffusion 1.5 and its continued popularity despite the release of Stable Diffusion XL 1.0.

Discussion of LoRA (Low-Rank Adaptation) and its use in fine-tuning models for specific styles, people, or characters.

Explanation of textual inversions and embeddings for fixing common image generation problems like bad hands or eyes.

Introduction to ControlNets for guiding image structure, depth, and character positions in image-to-image or video generation.

Overview of key tools and extensions like ControlNets, Deorum for smooth video generation, and ESRGAN for upscaling.

Transcripts

[Music]

welcome to cai.com official beginners

guide to AI art my name is Tyler and

throughout this series I will be your

guide as we go from zero to generating

our first AI images throughout these

videos you can expect to learn about the

Core Concepts and the terminology behind

AI art and stable diffusion we're going

to discuss and walk through how to

install the various pieces of software

and programs you will need to generate

AI images on your own local machine and

we're going to learn how to navigate

these programs as well as how to

properly download and store resources

from the coty.com resource library

before we get to installing anything

there are a lot of Core Concepts and

terminology used throughout AI art and

stable diffusion that if you're new to

all this really might be overwhelming or

not familiar so in this video we're

going to discuss some common terms

abbreviations and Concepts that you will

encounter as you're browsing websites

like civii and interacting with software

like automatic 1111 Focus comfy UI or

easy diffusion so let's get started by

discussing the various concept types of

image generation that you'll be doing

throughout your time making AI images

starting with our very first concept and

the most common which is text to image

you're going to to see this term a lot

and this refers to taking a text prompt

and generating an image out of nothing

using only the text and telling the AI

exactly what you would like to see in

your image then we have image to image

and batch image to image this is the

process of taking an existing image or a

reference photo for example a photo of

myself or a photo of a friend and using

that photo as the input for the AI to

then take your prompt reference the

photo and build the output image on top

of the already existing photo for this

you'll be using something called a

control net which we will talk about in

the extensions part of this video image

to image is doing so with only one

single image whereas a batch image to

image is taking a folder of images and

running them through the diffusion

process all at the same time next we

have in painting which is the practice

of using a painted mask area to add or

remove objects from an image think of

this as generative fill from Photoshop

except it lives locally in your stable

diffusion software and you get to paint

right on your image with a brush tool

punching The Prompt exactly what you

want to happen in the part of the image

that you painted next we have text to

video and video to video or as you'll

see them referred to text to vid or vid

to vid these are the processes of taking

a text prompt and getting a video output

with motion or taking an existing video

input and transforming that video

utilizing your prompt next we have the

most important part The Prompt and the

negative prompt The Prompt is the text

input that you give your stable

diffusion based software or any AI image

generation software in general to tell

it exactly what you would like it to

Output in your image the negative prompt

does the reverse this is where you take

your text input and tell stable

diffusion what you do not want in your

photo next we have upscaling upscaling

is the process of taking low resolution

media think an image that is a 512 x 512

small little square and converting it to

high resolution media think a square

that is 1080 x 1080 this is usually done

by enhancing the existing pixels and

most of the time we are now doing this

through either AI models that are built

into our stable diffusion software and

interfaces or we're using external

programs like topaz photo AI or topaz

video AI to upscale our images and

videos before we go and we share them on

the Internet or post them wherever we

want the upscaling process is usually

going to be the last part before you're

ready to share your images these are the

Core Concepts that you will be utilizing

anytime you sit down to generate

something with stable diffusion next

we're going to dive into the models

assets and resources that you're going

to come across on a regular basis so to

start off checkpoints checkpoints are

now more commonly referred to as models

but you will see these terms use

interchangeably as you go from site to

site and you're looking for different

models or checkpoints to use in your

Generations a model is the product of

the training millions of images scraped

from all over the web and this file

drives our results from text to image

image to image and text to video

Generations this is the heartbeat of

everything you will be doing in stable

diffusion typically your model will

dictate the overall style that you will

get out of your image some models are

really great all-arounders some are very

strictly trained on anime and some are

very strictly trained trained on

realistic images choosing the right

model is vital to getting the image that

you would like out of stable diffusion

all right let's move on to checkpoints

and safe tensors now checkpoints are a

file format created by pytorch lightning

it contains a machine learning model

which is used by stable diffusion to

generate our image outputs now this the

checkpoint or the ckpt file is

superseded and has mostly been replaced

by safe tensor files safe tensor files

are essentially the same thing except

they are less susceptible to having

malicious code put in them so whenever

possible you would want to look for the

safe tensor version of a model rather

than a ckpt this is also why it is good

to read reviews before you download any

models and install them into your hard

drive on your machine you want to make

sure that you're not downloading

anything malicious now anytime you hear

the term training data it's referring to

a set of many images that are used to

train a stable diffusion model Laura or

embedding Lon 5B this is a large scale

data set for research purposes that has

been trained on

5.85 billion clip filtered text to image

pairs this is the data set that stable

diffusion was trained on which brings us

to stable diffusion 1.5 or also referred

to all over the internet as SD 1.5 this

is a latent text to image model trained

on

595,000 steps at a resolution of 512x

512 images from the Layon 5B model this

has now been superseded by stability

ai's latest release stable diffusion XL

1.0 however a lot of the community still

uses stable diffusion 1.5 because of its

flexibility and the sheer amount of

resources that are available for SD 1.5

next up we have Laura L O R A which

stands for low rank adaption now Aura is

essentially a model but trained on a

much much much smaller data set geared

towards a very specific thing this thing

could be a person a style or a concept

so you will find many lauras trained on

specific anime characters so that when

you include the Laura in your image

generation process it is going to push

your image output to have that specific

character in the final image textual

inversions and embeddings well these are

similar to lauras but they're trained on

even smaller data sets and really geared

towards capturing Concepts such as

fixing bad hands fixing bad eyes objects

and specific faces next we have vaes or

vay vays are optional detail oriented

files that sometimes come built into

your models or more often you will have

to include a v next to your model for

your image generation you can think of

vays as the final touch to getting a

really crisp sharp colorful image some

models without the use of a vay the

colors will feel very dull and washed

out or they will have less details so

you either want to make sure that the

model that you are currently running has

a vay built into it or if not you want

to use your own vay alongside of it that

just about covers the model section now

let's jump into some of the most

important and common extensions that you

will encounter while you're using stable

diffusion all right so our first

extension and quite possibly one of the

most important things you will come

across while you're using stable the

fusion if you want to do anything

outside of just basic text to image

prompting is control Nets control Nets

consists of a bunch of different models

that are trained on specific data sets

to read different structures of an image

such as straight lines depth character

position where it will actually position

a dummy inside of the character in your

photo so that you can then take that

dummy and generate a whole new person on

top of the exact pose that that person

was in control Nets are essential if you

want to do anything involving image to

image or video to video next we have

deorum deorum is a community of AI image

synthesis developers enthusiasts and

artists that build a large set of

generative AI tools they are most

commonly known for their super popular

automatic 1111 extension that can take a

text prompt and generate a really really

smooth video output that you can also

key frame specific zooming panning and

turning motions into next we have estan

the enhanced super resolution generative

adversarial Network estan is a technique

that is used to generate high resolution

images from low resolution pixels think

upscaling a 720 image up to 1080 this

model is commonly found in a lot of

stable diffusion interfaces next we have

animate diff anim diff is a technique

used to inject motion into text to image

and even imageo image Generations these

are all of the Core Concepts and

terminology and terms that you will come

across during your time using stable

diffusion if at any point you get lost

or you need some extra help figuring out

what something means or you need

something to refer to feel free to visit

our stable diffusion glossery in the

coty.com education Hub we'll see you

guys in the next video