An AI artist explains his workflow

The transcript describes the workflow of an AI artist who uses Stable Diffusion technology to create unique art pieces. The artist, who views AI as a collaborative tool rather than a threat, uses a combination of traditional and digital art skills to craft his character, Stelfie, in various adventures. The process involves sketching, experimenting with random prompts, and manually adjusting poses and features in Photoshop. The artist emphasizes the importance of controlling the AI rather than being controlled by it, highlighting the ongoing interplay between human creativity and technology in the art world.

Takeaways

• 🎨 **Artistic Vision**: The artist uses AI to realize a creative vision, combining Stable Diffusion with traditional artistry to capture a scene with Stelfie and Muhammad Ali.
• 📝 **Sketching First**: The workflow begins with a sketch, which is crucial for maintaining the original idea amidst the AI's creative influence.
• 🧩 **Random Prompts**: Experimenting with various prompts helps find a suitable initial pose, showing the iterative nature of the creative process.
• 💻 **Photoshop for Posing**: When a pose is hard to find through AI, the artist manually recreates it in Photoshop, demonstrating the value of human intervention.
• 🔄 **Different Samplers**: The choice of sampler (like Euler or DPM) significantly affects the realism and details of the artwork, especially for replicating skin.
• 🔢 **Parameters Matter**: Parameters such as steps, inpaint, and outpaint are essential in guiding the AI to refine or expand the image as desired.
• ⏱️ **Time Investment**: The artist estimates that about half of the work is done using Stable Diffusion, with significant time spent in Photoshop and Procreate for fine-tuning.
• 📸 **Training Specific Models**: For character-specific features like Stelfie's face, the artist uses a model trained on numerous snapshots of the character's face from different angles.
• 🔧 **Noise Strength Control**: Adjusting noise strength allows for more control over the final image, which is particularly important for faces and popular figures.
• 🤽‍♂️ **Physical Authenticity**: The artist manually modifies features to achieve a realistic portrayal of the physique, such as Muhammad Ali's, rather than a generic athletic build.
• 🖌️ **Artistic Control**: The artist emphasizes the importance of driving the AI process rather than being driven by it, highlighting the ongoing role of human creativity.
• 👐 **Challenges with Hands**: Reproducing hands remains a challenge, leading the artist to use photographs of their own hand as a reference for accuracy.

Q & A

Who is Stelfie and what is his significance in the artist's work?

Stelfie is a character created by the artist who is portrayed as funny, clumsy, and a time traveler with incredible adventures. He serves as an alter ego for the artist, despite their physical differences. Stelfie is used to showcase the potential of Stable Diffusion technology combined with artistic skills.

What is the artist's initial step in creating a scene with Stelfie?

The artist's initial step is to draw a sketch. This helps to maintain control over the original idea and serves as a starting point before engaging with Stable Diffusion and other diffusion models.

How does the artist use Stable Diffusion and random prompts?

The artist uses Stable Diffusion and random prompts to find a good initial pose for the character. If a suitable pose is not found, the artist moves to Photoshop to recreate the pose manually.

What is the role of ControlNet in the artist's workflow?

ControlNet is an extension that the artist uses to reproduce poses. It is particularly useful for recreating poses quickly; the artist mentions it would take only about 15 minutes to reproduce a pose from two months prior using ControlNet.

Why is the choice of sampler important in the artist's process?

The choice of sampler is important because it affects the realism and details of the final image. Different samplers like Euler and DPM have different effects on the synthetic quality of elements like skin in the artwork.

What does the term 'steps' refer to in the context of Stable Diffusion?

Steps refer to the number of times the artist instructs Stable Diffusion to work on the prompt. It can be set to a low or high number, affecting the level of detail and refinement in the output.

How does the artist use the inpaint and outpaint features?

Inpaint is used to ask the machine to change specific parts of the image, while outpaint is used to imagine what's outside the boundaries of the image based on the content within it. These features help the artist to refine and extend the artwork as needed.

What is the approximate distribution of work between Stable Diffusion, Photoshop, and Procreate?

The artist estimates that 50% of the work is done with Stable Diffusion, about 40% in Photoshop, and the remaining 10% in Procreate.

How does the artist approach creating Stelfie's face?

The artist uses a model trained specifically on Stelfie's face. This was done by creating a 3D model of Stelfie and taking snapshots from different angles to train the model. A keyword is saved when training the model for future use.

What is the significance of noise strength in Stable Diffusion?

Noise strength is important as it provides more or less control over the image itself. It's particularly challenging for faces, especially when trying to replicate a well-known person like Muhammad Ali.

How does the artist achieve a realistic portrayal of Muhammad Ali?

The artist instructs Stable Diffusion to create a face resembling Muhammad Ali and then manually adjusts features like the nose, jaw, and eyes in Photoshop to achieve the desired likeness. The artist also ensures the pose is correct and modifies the body to reflect Ali's physique accurately.

What is the artist's perspective on the role of AI in the creative process?

The artist views the process as a joint effort with AI and does not feel threatened by it. Instead, they see it as an opportunity for new talent to explore a new branch of art, different from traditional digital art, and to open up new ways of being creative.

Why does the artist use their own hand for Stelfie's hands in the artwork?

The artist uses their own hand for Stelfie's hands because reproducing hands has always been challenging. They take a picture of their hand in the needed position, clean it up, and paste it onto the artwork.

Outlines

00:00

🎨 Artistic Fusion: Stelfie's Creation and AI Collaboration

The first paragraph introduces Stelfie, a humorous and clumsy character who embarks on time-traveling adventures. Stelfie is an alter ego of the artist, despite their physical differences. The project's inception was to demonstrate the synergy between Stable Diffusion technology and artistic skills, with the ultimate goal of depicting Stelfie in a boxing match with Muhammad Ali. The artist's process begins with sketching and then experimenting with various prompts to find an initial pose. When the desired pose for Stelfie was elusive, the artist resorted to recreating it manually in Photoshop. The importance of using different samplers for realism and detail is emphasized, with a preference for DPM when dealing with skin textures. Parameters such as steps, inpaint, and outpaint are crucial in guiding the AI's output. The artist iterates between Stable Diffusion and Photoshop, with a significant portion of the work done in each. Stelfie's face is created using a model trained on his 3D snapshots, and the artist manually adjusts facial features to resemble Muhammad Ali. The pose's accuracy is critical, and the artist aims for a realistic, non-buff look for Stelfie, modifying the result further in Photoshop.

05:02

🖌️ Refining the Art: The Artist's Role in AI-assisted Creation

The second paragraph delves into the artist's understanding of Muhammad Ali's appearance and the meticulous work required to achieve a realistic portrayal. This involves extensive manipulation in Photoshop, including cropping, cutting, pasting, warping, and painting to adjust details like arms, eyes, exposure, and skin tone. Despite the challenges, the artist can rely on Stable Diffusion to assist with specific elements like edges, lighting, or skin. The paragraph highlights the importance of the artist's role in driving the creative process, rather than being driven by the AI. With a background in traditional and digital art, the artist views AI as an opportunity for new artists to explore a different branch of art and expand creative possibilities. The hands in Stelfie's artwork are particularly challenging to reproduce, so the artist uses photographs of their own hand, cleaned up and integrated into the artwork, to achieve the desired effect.

Keywords

💡Stable Diffusion

Stable Diffusion is an AI model that generates images from textual descriptions. It is used by the artist to create visuals based on prompts. In the video, the artist combines Stable Diffusion with his artistic skills to produce unique art pieces, showcasing the potential of AI in the creative process.

💡Alter ego

An alter ego is a second self, often used in fiction and by artists to express different aspects of their personality. In the context of the video, Stelfie is the artist's alter ego, a character through which the artist explores different adventures and scenarios.

💡Sketch

A sketch is a rough drawing that serves as a preliminary representation of an idea or concept. The artist begins his creative process by drawing a sketch, which forms the basis for further development using AI and digital tools.

💡Photoshop

Photoshop is a widely used digital image editing software. The artist uses Photoshop to refine the poses and details of the characters in his artwork, particularly when the AI-generated output does not meet his expectations.

💡ControlNet

ControlNet is an extension or tool that aids in the reproduction of poses or elements in artwork. The artist mentions that if he were to reproduce a pose today, ControlNet could significantly reduce the time required for the task.

💡Samplers

In the context of AI image generation, samplers are algorithms that determine how the AI processes the input to create an image. Different samplers can affect the realism and level of detail in the output. The artist emphasizes their importance in achieving the desired look for elements like skin texture.

💡Steps

Steps refer to the number of iterations the AI performs on a given prompt to refine the image. A higher number of steps can lead to more detailed and refined outputs, but it also increases the processing time.

💡Inpaint and Outpaint

Inpaint is a technique where the AI is instructed to modify only certain parts of an image, while outpaint involves the AI generating content that extends beyond the original image boundaries based on the existing content. These techniques are crucial for the artist's workflow, allowing him to direct the AI to make specific changes or expansions to his artwork.

💡Procreate

Procreate is a digital illustration app used for creating and editing images on mobile devices. The artist mentions using Procreate for a portion of his work, highlighting the multifaceted approach to digital art that combines various tools and platforms.

💡Noise strength

Noise strength is a parameter in AI image generation that controls the level of detail and randomness in the output image. It is an important setting for artists looking to balance control over the AI's creativity with the level of detail they want to achieve in their final artwork.

💡Training a model

Training a model involves feeding an AI system a large number of examples so it can learn to recognize and replicate certain features or styles. The artist trained a model specifically on Stelfie's face using 3D snapshots, allowing the AI to generate images of the character's face more accurately.

💡Artist's part in creation

The artist emphasizes the collaborative nature of the creative process when working with AI. He sees his role as guiding and directing the AI, rather than being replaced by it. This perspective highlights the ongoing importance of human creativity and skill in the age of AI-assisted art.

Highlights

Stelfie is an alter ego of the artist, representing a character with time-traveling adventures.

The project's aim was to capture a scene of Stelfie boxing with Muhammad Ali using Stable Diffusion.

The artist begins with a sketch and uses Stable Diffusion to generate initial poses.

Diffusion models can be cheeky and deviate from the original idea, requiring artist intervention.

Photoshop is used to recreate poses when Stable Diffusion fails to generate satisfactory results.

ControlNet, an extension, can significantly reduce the time required to reproduce poses.

Different samplers are used throughout the process for varying levels of realism and detail.

Euler sampler is synthetic and fake, while DPM works well for replicating skin.

Parameters such as steps, inpaint, and outpaint are crucial for the Stable Diffusion process.

Inpaint focuses on changing specific parts of the image, while outpaint imagines what's outside the frame.

The workflow involves a balance between Stable Diffusion, Photoshop, and Procreate.

A model trained on Stelfie's face is used for consistency, leveraging 3D snapshots for training.

Noise strength in Stable Diffusion's web UI is essential for control over the final image.

Replicating faces, especially of popular individuals like Muhammad Ali, is challenging.

Artistic adjustments in Photoshop are made to traits like the nose, jaw, and eyes to achieve accuracy.

The artist aims for a realistic, yet not overly muscular portrayal of Muhammad Ali.

The artist's experience with traditional and digital art is seen as an opportunity, not a threat, in the age of AI.

The artist emphasizes the importance of driving the AI process rather than being driven by it.

Hands are particularly challenging to reproduce, often requiring the artist's own hand to be photographed and edited in.