Runway Gen-2 Ultimate Tutorial : Everything You Need To Know!

Welcome to the tutorial on AI-generated video with Gen 2, where you'll learn how to create compelling content using a minimal web UI. The video covers prompt writing with a formula that includes style, shot, subject, action, setting, and lighting. It demonstrates how to use controls like seed number and interpolate function for smooth transitions. Reference images are shown to enhance output. The tutorial also explores character and setting archetypes, and the creative process of working with Gen 2, comparing it to collaborating with a stubborn cinematographer. It concludes with upscaling results and a note on differences between the Discord and web versions of Gen 2. The host, Tim, also mentions a Patreon for a close-knit community to discuss projects and share insights.

Takeaways

• 🎬 **Introduction to Gen 2**: The tutorial introduces the Gen 2 AI video generation tool, focusing on the web UI version and providing an overview of its capabilities.
• 📝 **Prompt Writing Tips**: The script suggests a formula for writing prompts that includes style, shot, subject, action, setting, and lighting.
• 🔑 **Seed Number and Interpolation**: The seed number is important for consistency, and the interpolate function should be left on for smooth transitions between frames.
• 📈 **Upscaling Quality**: The tutorial demonstrates the difference between the free and beta upscaled versions of the generated video.
• 🚫 **Watermark Removal**: The free version has a watermark, but this can be removed in the paid version.
• 📷 **Reference Image Usage**: Users can upload a reference image to influence the AI's video generation.
• 🎭 **Character Descriptions**: Keeping character descriptions simple can lead to better results and consistency in the generated video.
• 🏞️ **Setting and Environment**: Gen 2 can classify and generate settings like cities and environments, although specific actions might not always be accurate.
• 🔍 **Image Prompting**: Using image prompts can help refine the character's appearance to match the user's vision more closely.
• 🤸 **Action and Movement**: Gen 2 can handle certain actions like walking and talking but may struggle with complex or specific actions like a skateboard kickflip.
• 📉 **Seed Locking**: Locking a seed ensures a consistent look throughout a sequence of generated videos.
• 🌐 **Discord vs. Web UI**: There are differences between the Discord and web UI versions of Gen 2, with the potential for features to be transported between the two.
• 📢 **Community and Support**: The creator is launching a Patreon to build a community for support and discussion on various projects.

Q & A

What is the main focus of the tutorial?

The main focus of the tutorial is to provide an overview and guidance on using AI-generated video via Gen 2, including prompt tips and general advice on what to expect from the system.

Which version of Gen 2 does the tutorial cover?

The tutorial covers the web UI version of Gen 2, with a mention of differences from the Discord UI version.

What is the purpose of the seed number and interpolate function in Gen 2?

The seed number is used to ensure consistency in the generated output, while the interpolate function controls the smoothness between frames, which is recommended to be left on at all times.

What does the term 'upscale' refer to in the context of Gen 2?

In the context of Gen 2, 'upscale' refers to the process of increasing the resolution and quality of the generated video, which is available in the beta version and for paying customers.

How does the speaker suggest writing prompts for Gen 2?

The speaker suggests a formula for writing prompts that includes style, shot, subject, action, setting, and lighting, and emphasizes the importance of keeping character descriptions simple.

What is the significance of the 'lock seed' feature when generating a sequence of outputs?

The 'lock seed' feature ensures that the generated sequence of outputs maintains a consistent look by using the same seed number throughout the sequence.

What happens when the speaker tries to generate a skateboarding action with Gen 2?

The initial attempt at generating a skateboarding action results in an unrealistic output with a skater having three legs and a wonky anatomy. After revising the prompt, the output is closer to the desired result but still lacks the specific action of a jump or kickflip.

How does the speaker describe the process of working with Gen 2?

The speaker describes working with Gen 2 as collaborating with a very stubborn cinematographer, where the system may not always produce the exact desired shot but can be influenced through experimentation and re-rolling.

What is the speaker's approach to creating characters and settings within Gen 2?

The speaker suggests creating characters and settings within mid-journey and using them as storyboards or a casting department for Gen 2, aiming for consistency by keeping descriptions simple.

What is the difference between the Discord version and the web-based version of Gen 2 mentioned in the tutorial?

The Discord version has certain commands like 'CFG_scale' that are not available in the web-based version, and the speaker expects these features to be implemented in future updates of the web-based version.

What additional tool is mentioned for further processing of Gen 2 footage?

The speaker mentions using an app called Reface for face swapping characters in the generated footage, and then further processing in Kyber.

What is the speaker's future plan for community engagement?

The speaker is soft launching a Patreon with the aim of creating a smaller, more intimate community where members can discuss various projects and help each other out, with the possibility of expanding in the future.

Outlines

00:00

🎬 Introduction to AI Generated Videos with Gen 2

The video script introduces viewers to the world of AI-generated videos using Gen 2. The presenter provides an overview and tutorial on using the web UI version of Gen 2, mentioning the minimalist interface and the ability to write prompts, control seed numbers, and adjust the interpolate function for smoother transitions between frames. The presenter also discusses the free version of the software and the upscale feature, which improves the quality of the output. The script outlines a formula for writing effective prompts, which includes style, shot, subject, action, setting, and lighting. Examples are given to illustrate how to apply this formula, and the presenter demonstrates the process by generating a sequence of images with a cinematic action sci-fi theme and horror film lighting.

05:01

📹 Exploring Prompts and Image Prompting in Gen 2

The script continues with a deeper dive into the process of generating images with Gen 2, focusing on the importance of specifying the shot and how the software interprets various elements of a prompt. The presenter discusses the limitations of Gen 2 when it comes to generating complex actions, such as a skateboarding kickflip, and how to work around these limitations by revising the prompt. The concept of using mid-journey images as a form of storyboard or casting reference for Gen 2 is introduced, and the presenter demonstrates this by generating a sequence of images that resemble a James Bond film scene. The output is then compared between the web UI and Discord UI versions of Gen 2, highlighting the differences in quality and resolution when upscaling is used.

10:02

📚 Final Thoughts and Future Collaborations

The video script concludes with the presenter sharing his thoughts on the current state of Gen 2 and its potential for future development. He mentions the possibility of new features being implemented in the web-based version of the software, such as a slider for adjusting the scale of the entire prompt and the green screen command. The presenter also announces a soft launch of a Patreon, which will provide supporters with access to a secret Discord community for collaborative discussions and project assistance. He encourages viewers to join this community for a more intimate and focused environment and to have a say in its development. The presenter, Tim, thanks the viewers for watching and invites any questions in the comments section.

Keywords

AI generated video

AI generated video refers to the creation of video content using artificial intelligence. In the context of the video, it is the main theme as the tutorial focuses on how to use Gen 2, an AI tool, to generate videos. The script discusses the process of generating videos with AI, including writing prompts and controlling parameters like seed numbers.

Gen 2

Gen 2 is the second generation of an AI tool for video generation. It is central to the video's content as the tutorial provides an overview and guidance on how to use this specific tool. Gen 2 is mentioned multiple times throughout the script, indicating its importance in the video generation process.

Prompt

In the context of AI video generation, a prompt is a text input that guides the AI in creating the desired video output. The script emphasizes the importance of crafting effective prompts, providing a formula that includes style, shot, subject, action, setting, and lighting to achieve the best results with Gen 2.

Seed number

A seed number in AI video generation is a unique identifier that ensures the consistency of the generated output. The script mentions controlling the seed number to maintain a consistent look across different frames or sequences in the generated video.

Interpolate function

The interpolate function in Gen 2 is a feature that controls the smoothness between frames in the generated video. The script suggests keeping this function on at all times to ensure smooth transitions in the video output.

Upscale

Upscaling in the context of video generation refers to increasing the resolution of the video output. The script discusses the difference between the free version and the upscaled version of Gen 2, noting that the upscaled version provides higher definition outputs.

Watermark

A watermark is a visible or invisible marker added to a video to indicate the source or ownership. The script mentions the option to remove the watermark from the generated video, which is a feature typically available in the paid version of such tools.

Reference image

A reference image is a visual input that can be used to guide the AI in generating a video that matches or is inspired by the provided image. The script discusses uploading a reference image as a way to influence the style and content of the generated video.

Character description

Character description in the context of video generation involves specifying the appearance and attributes of characters in the video. The script suggests keeping character descriptions simple for better results with Gen 2, such as 'woman with red hair in a black dress'.

Shot

In filmmaking, a shot refers to a continuous series of frames captured by a single, uninterrupted camera operation. The script discusses different types of shots like wide angle, medium shot, close-up, and extreme close-up, which are used to direct the AI in the framing of the generated video.

Action

Action in the context of video generation refers to the activities or movements performed by the characters in the video. The script notes that the action should be something that can be found in stock video footage to increase the likelihood of successful generation with Gen 2.

Highlights

Introduction to AI-generated video via Gen 2 with a focus on the web UI version.

Minimalistic interface allows for prompt writing and control over seed number and interpolate function.

Interpolate function enhances the smoothness between frames and is recommended to be left on at all times.

Free version of Runway allows for watermark removal and image upscaling through beta access.

Reference images can be uploaded to influence the AI generation process.

A formula for writing prompts is suggested: style, shot, subject, action, setting, and lighting.

Experimentation with keywords like 'cinematic action' and 'black and white film' can yield good results.

Character descriptions should be simple and straightforward to maintain consistency.

Action-oriented prompts should align with existing stock footage for better results.

Specific cities can be named to give an overall vibe of that city in the generated video.

Lighting can be described in broad terms like 'sunset' or 'horror film lighting'.

An example prompt is given: 'cinematic action sci-fi film, a marine walks down a spaceship hallway, horror film lighting'.

Locking a seed ensures a consistent look in the generated sequence.

Adding 'close up' to the prompt results in a more focused shot.

Gen 2's limitations are shown when it doesn't have a reference for an action, resulting in a generic image.

Image prompting allows for more specific character archetypes to be generated.

Using mid-journey images as references can guide the AI towards desired outputs.

Upscaling the output through the Gen 2 Discord results in a significant increase in quality.

Differences between the Discord and web-based versions of Gen 2 are expected to be reconciled in future updates.

A Patreon is being soft-launched for a more intimate community discussion and project collaboration.