How to Prompt for Text to Video | Runway Academy

In this Runway Academy tutorial, viewers are guided through the process of creating a text to video prompt in Gen 2. The video begins with a simple prompt, 'a dog,' and then demonstrates how adding specific details such as breed, color, location, and time of day can enhance the quality and specificity of the generated images and videos. The presenter emphasizes the importance of focusing on 'what it looks like' and 'what it is,' using examples to illustrate how detailed prompts can lead to more nuanced outputs. The video also explores the use of style presets and prompt modifiers to further refine the style and direction of the generated content. By combining clear prompts with relevant modifiers and avoiding contradictions, users can gain greater control over their creations. The tutorial concludes with an invitation to join the Runway Academy community on Discord for additional resources and support.

Takeaways

• 📝 Start with a simple prompt for text to video, like 'a dog', to generate initial images.
• 🔍 Use free preview to view and select from the generated images before proceeding.
• 🎨 Refine your prompt by specifying details such as dog breed, color, location, and time of day.
• 🐕 Enhance the specificity of your prompt for more controlled and detailed outputs, e.g., 'light colored French Bulldog laying on a couch during the day'.
• 📷 Include details about the type of shot desired, such as 'extreme close-up' or 'shallow depth of field'.
• 🖼️ Understand the difference between 'what it looks like' and 'what it is', using examples like a painting vs. a bowl of fruit.
• 🌾 Add texture and context to your prompts for a more nuanced result, like 'a textured, historical oil painting of a bowl of fruit'.
• 🎨 Utilize style presets and prompt modifiers to further influence the style and direction of your output.
• 🔑 Experiment with style presets and prompt modifiers to achieve the desired style, even with simplified prompts.
• 🚫 Avoid contradictions, run-on sentences, and unnecessary filler words in your prompts for clarity and control.
• 💡 Focus on relevant prompt modifiers and keep prompts as specific as possible for better control over generations.

Q & A

What is the main topic of the video from Runway Academy?

The main topic of the video is discussing the basics of prompting for text to video in Gen 2.

How does the Gen 2 system generate images from a simple prompt?

The Gen 2 system generates images by using the input prompt to create starting images that reflect the details provided in the prompt.

What is the significance of being specific in your prompt when using Gen 2?

Being specific in your prompt allows the images and resulting videos to reflect those details more meaningfully, giving the user more control over the outputs.

What are the two main things to focus on when creating a prompt in Gen 2?

The two main things to focus on are what it looks like and what it is, which helps in creating a more detailed and nuanced prompt.

How can you control the style of the output in Gen 2?

You can control the style of the output by using style presets and prompt modifiers, which can be added to the end of a prompt for further control.

What is the role of style presets in Gen 2?

Style presets in Gen 2 have a significant influence on the style of the output results and can be used even with the most simplified prompts.

What are prompt modifiers and how can they be used?

Prompt modifiers are descriptive words that can be added to the end of a prompt to help control the output further. They can be used to steer the generation into a more stylized and specific direction.

How can prompt modifiers be combined with style presets?

Prompt modifiers can be used with style presets to give the user even more control over the generation, allowing for a more refined and specific output.

What is the advice given for creating effective prompts in Gen 2?

To create effective prompts, focus on what it looks like and what it is, include relevant prompt modifiers, keep it as focused as possible, and avoid run-on sentences, unnecessary filler words, too many concepts in a single prompt, or contradictory prompt modifiers.

What additional resources are available for learning more about using Runway?

Users can join the Runway Academy community on Discord for more information, experimentation, and to find specific answers to their questions.

How can users find specific answers to their questions about using Runway?

Users can find specific answers to their questions by using the button on their dashboard at any time.

What is an example of a detailed prompt that includes both 'what it looks like' and 'what it is'?

An example of a detailed prompt is 'a textured, historical oil painting of a bowl of fruit on a table in a farmhouse kitchen.'

Outlines

00:00

📝 Introduction to Text-to-Video Prompting in Gen 2

The video begins with a welcome to Runway Academy and an introduction to the basics of creating prompts for text-to-video in Gen 2. The process is demonstrated by starting with a simple prompt like 'a dog' and then refining it by adding details such as the type, color, and breed of the dog, its location, and the time of day. As more details are included, the resulting images and videos become more tailored to the prompt, offering greater control over the final output. The video also emphasizes the importance of focusing on the visual appearance ('what it looks like') and the subject matter ('what it is').

🎨 Enhancing Prompts with Details and Style

The video continues by illustrating how to enhance prompts with more details to achieve a specific look. It uses the example of an 'extreme close-up shallow depth of field portrait photo of a light colored French Bulldog laying on a couch during the day' to show how adding descriptive words can influence the output. The presenter also discusses the use of style presets found at the bottom of the prompt box in Gen 2, which can significantly affect the style of the generated content. The video encourages experimentation with these presets, even with simple prompts, to achieve a desired style.

📜 Using Prompt Modifiers for Further Control

The video explains the concept of prompt modifiers, which are descriptive words that can be added to the end of a prompt to further control the output. An example prompt is given: 'a photo of a city skyline made out of paper,' followed by modifiers such as 'burnt texture,' 'black and gray color palette,' and 'flat composition.' These modifiers help guide the generation process towards a more stylized and specific direction. The video also mentions that prompt modifiers can be used in conjunction with style presets for even greater control over the generation process.

✅ Best Practices for Effective Prompting

The presenter concludes with best practices for creating effective prompts. It is advised to focus on the visual aspects and the essence of what is being depicted, to include relevant prompt modifiers, and to maintain focus by avoiding run-on sentences, unnecessary filler words, too many concepts in a single prompt, or contradictory modifiers. The video encourages viewers to join the community on Discord for more resources and experimentation, or to use the dashboard button for specific answers to their questions.

Keywords

Text to Video

Text to Video refers to the process of converting textual prompts into visual video content. In the context of the video, it is the main theme as it discusses how to create videos from textual descriptions using Gen 2 technology. The script provides a step-by-step guide on how to input prompts to generate videos, emphasizing the importance of specificity in the prompts to control the output.

Gen 2

Gen 2 is likely a reference to a second-generation model or version of a text-to-video system. It is the platform on which the video's instructions are based, and the script describes navigating and using features within Gen 2 to create videos from text prompts.

Prompt

A prompt, in this context, is an input of text that serves as an instruction to the Gen 2 system to generate specific video content. The script emphasizes the importance of crafting detailed prompts to achieve desired video outcomes, such as specifying the type of dog, its color, and the setting.

Free Preview

Free Preview is a feature within Gen 2 that allows users to view starting images based on their text prompts without any cost. It is used in the script to demonstrate the initial results after entering a simple prompt like 'a dog'.

Detail

Detail refers to the specific information included in a text prompt that helps refine the video output. The script illustrates how adding details such as the type of dog, its color, location, and time of day can make the generated images and videos more aligned with the user's vision.

Type of Shot

Type of Shot describes the camera perspective or framing that a user may want in their video. The script mentions 'extreme close-up' and 'shallow depth of field' as examples of specifying the type of shot to direct the video's aesthetic.

Style Presets

Style Presets are pre-defined settings in Gen 2 that influence the stylistic outcome of the generated videos. They can be used to apply a certain look or feel to the videos and are mentioned in the script as a tool for achieving specific styles even with simple prompts.

Prompt Modifiers

Prompt Modifiers are additional descriptive words that can be added to a prompt to further control the output of the video. The script provides examples such as 'burnt texture', 'black and gray color palette', and 'flat composition' to demonstrate how these modifiers can steer the generation process towards a more stylized direction.

Output

Output, in this context, refers to the final video content generated by the Gen 2 system based on the text prompt. The script discusses how the specificity and detail of the prompt directly affect the quality and relevance of the output.

Control

Control denotes the level of influence a user has over the final video generated by the system. The script emphasizes gaining more control over outputs by providing detailed prompts, using style presets, and applying prompt modifiers.

Discord Community

Discord Community refers to a group of users who communicate and share information using the Discord platform. In the script, it is mentioned as a resource for users to join for more information and experimentation with Runway or to get specific answers to their questions.

Dashboard

Dashboard, as used in the script, likely refers to the user interface of the Gen 2 system where users can manage their prompts, view outputs, and access additional features or resources. It is mentioned in the context of finding a specific answer to a question using a button on the dashboard.

Highlights

Discussion on the basics of prompting for text to video in Gen 2

Navigate to Gen 2 and select 'text to video' to start

Enter a simple prompt like 'a dog' to generate starting images

Adding specificity to prompts improves the quality of generated images and videos

Include details like type, color, breed, location, and time of day for more control

Example prompt: 'a light colored French Bulldog laying on a couch during the day'

Details about the shot type, such as extreme close-up or shallow depth of field, can be included

Focus on 'what it looks like' and 'what it is' for effective prompting

Use style presets found at the bottom of the prompt box for different output styles

Experiment with style presets even with simplified prompts

Prompt modifiers can be added to the end of a prompt for further control

Modifiers are descriptive words that steer the generation in a specific direction

Example prompt with modifiers: 'a photo of a city skyline made out of paper' with 'burnt texture, black and gray color palette, flat composition'

Prompt modifiers can be used alongside style presets for greater control

Keep prompts focused, avoiding run-on sentences or contradictory modifiers

Join the Runway Academy community on Discord for more resources and experimentation

Use the dashboard button for specific answers to questions about using Runway