AI Video Tools Are Exploding. These Are the Best

Futurepedia
24 Jul 202423:13

Summary

TLDRThis video script explores the exciting advancements in AI video generation tools, highlighting Runway Gen 3 and Luma Labs' Dream Machine for their text-to-video and image-to-video capabilities. The script covers various platforms, including LTX Studio for detailed control and Kaa for abstract animations, and showcases the potential of AI in creating unique content. It also touches on lip-syncing tools and open-source models, emphasizing the current fun and creative possibilities in AI video production.

Takeaways

  • πŸ˜€ The video discusses the exciting advancements in AI video generation tools and their real-world applications.
  • πŸŽ₯ The speaker's favorite AI video tool not in the headlines is Runway Gen 3, which is considered the best text-to-video model available.
  • 🌟 Runway Gen 3 excels at creating title sequences with dynamic movement and intricate neon circuitry patterns.
  • πŸ“ The video provides examples of how to structure prompts for Runway Gen 3 to achieve better results, emphasizing the importance of camera movement and scene details.
  • πŸ”„ Runway Gen 3 can transform between scenes effectively, but the video also notes the potential for misses and the need for rerolls to get desired outcomes.
  • πŸŒ‹ Dream Machine from Luma Labs is praised for being the best image-to-video tool, especially with the use of keyframes for logical transitions.
  • 🎨 LTX Studio offers the most control and speed, allowing users to create short films from a script or simple prompt with various styles and customization options.
  • πŸ”„ The video showcases the style reference feature in LTX Studio, which regenerates every shot in a new style based on an uploaded image.
  • 🎭 The platform Kaa is highlighted for its focus on abstract and trippy animations, offering a unique creative avenue for video generation.
  • πŸ€– Hedra and Live Portrait are mentioned as accessible platforms for lip-syncing, with Hedra noted for its expressive talking avatars and Live Portrait for its mapping capabilities.
  • 🌐 The open-source community is acknowledged for pioneering tools and workflows that form the foundation of many paid AI video generation platforms.

Q & A

  • What is the main theme of the video script?

    -The main theme of the video script is exploring AI video tools, particularly focusing on Runway Gen 3, Luma Labs' Dream Machine, LTX Studio, and Kaa, and showcasing their capabilities in creating various types of video content.

  • Why is Runway Gen 3 considered the best text-to-video model according to the script?

    -Runway Gen 3 is considered the best text-to-video model because of its ability to generate impressive title sequences and fluid simulation physics, which are particularly useful for creating dynamic video content.

  • What is the significance of using a prompt structure when using Runway Gen 3?

    -Using a prompt structure helps to guide the AI in generating content that aligns with the creator's vision, reducing the number of attempts needed to achieve satisfactory results.

  • How does Luma Labs' Dream Machine differ from Runway Gen 3 in terms of video generation?

    -Dream Machine from Luma Labs excels in image-to-video generation and keyframe-based animations, offering high-quality results that are often good with just one or two tries.

  • What is the advantage of using keyframes in Luma Labs' Dream Machine?

    -Keyframes allow for more control over the video generation process, enabling the creation of smooth transitions and complex animations between two defined frames.

  • How does LTX Studio differ from other platforms mentioned in the script?

    -LTX Studio offers the most control and fastest speed among the platforms, allowing users to build out an entire short film in a few minutes with a high level of customization and flexibility.

  • What is the purpose of the style reference feature in LTX Studio?

    -The style reference feature in LTX Studio allows users to upload an image that sets the visual style for the entire video, ensuring consistency across all scenes.

  • What kind of content is Kaa particularly good for creating?

    -Kaa is particularly good for creating abstract, trippy, and morphing type animations, which are ideal for music videos or opening sequences.

  • How does Kaa's video upscaler work differently from traditional upscalers?

    -Kaa's video upscaler performs a creative upscale, reimagining the content with AI while staying close to the original video, rather than just increasing the resolution.

  • What are the limitations of using non-human characters in Hedra's lip-syncing feature?

    -Hedra's lip-syncing feature can struggle with non-human characters, especially if they appear less human, making it harder to map expressions accurately.

  • What is the significance of the open-source community in the development of AI video tools mentioned in the script?

    -The open-source community has been foundational in developing tools and workflows that have influenced the creation of paid platforms, offering more customization and control at a higher complexity level.

Outlines

00:00

🎨 AI Video Tools and Runway Gen 3

The speaker discusses their experience with AI video tools, highlighting the current excitement in the field. They focus on Runway Gen 3, which they consider the best text-to-video model available, showcasing its capabilities in creating dynamic title sequences. The speaker shares examples of impressive title sequences generated by Runway, emphasizing the tool's ability to handle fluid simulations and physics. They also demonstrate how to use Runway by generating a title sequence for 'Future Pedia', discussing the importance of prompt structure and the use of keywords for better results. The paragraph concludes with a mention of the occasional misses with Runway 2, noting that the showcased results are cherry-picked and that Runway can perform exceptionally well with sufficient credits.

05:01

πŸŒ‹ Image-to-Video Transformations with Luma Labs

The speaker explores the capabilities of Luma Labs' Dream Machine for image-to-video transformations, particularly praising its use of keyframes. They demonstrate the process by uploading an image and adding prompts to generate short video clips, noting the tool's consistency and relative ease of use. The paragraph also touches on the potential for creating longer sequences by using ending frames as starting points for new generations, and the creative possibilities this presents. The speaker mentions the long wait times for free users and the benefits of paying for faster access and more generations, concluding with a brief mention of another similar tool, Cling, which has a significant waitlist and a complex sign-up process without a Chinese phone number.

10:01

🎬 LTX Studio's Comprehensive Video Creation

The speaker introduces LTX Studio, emphasizing its control and speed in creating short films. They describe the process of generating a video from a script or simple prompt, customizing styles, characters, and even voices. The speaker illustrates the platform's flexibility by showing how to change character details, utilize face-swapping, and adjust the title. They also highlight LTX Studio's unique features, such as the style reference for consistent character design and the ability to customize motion and clip lengths. The paragraph concludes with the speaker's enthusiasm for LTX Studio, mentioning their positive experience meeting the team and encouraging viewers to sign up for the waitlist.

15:02

🌈 Abstract Animations and Creative Upscaling with Kaa

The speaker discusses Kaa, a platform for creating abstract and trippy animations, which they find enjoyable for its ability to open new creative avenues. They demonstrate the process of generating videos by adding keyframes and text prompts for morphing transitions between images. The speaker also mentions the platform's creative upscaler, which reimagines videos with AI, showing an example of how it can fix warped faces and change video styles. They conclude by expressing their excitement about Kaa's potential for unique and fun video creations, especially for music videos or opening sequences.

20:03

πŸ—£οΈ Lip Syncing Technologies and Open Source Tools

The speaker explores recent advancements in lip syncing technologies, mentioning platforms like Hedra and Live Portrait that allow for expressive talking avatars. They demonstrate Hedra's capabilities with classic movie lines and discuss the platform's limitations with non-human characters. The paragraph also introduces Live Anime, which maps reference videos onto avatars for more control over expressiveness. The speaker acknowledges the open source community's contributions to these technologies, citing tools like Comfy UI and Animate Diff, and recognizes the paid platforms' advantages in user interface and ease of use. They conclude by highlighting the creative potential of these tools and the importance of the open source community in pioneering AI video advancements.

πŸš€ Real-World AI Video Applications and Futurepedia

In the final paragraph, the speaker reflects on the current state of AI video tools, noting their real-world applicability beyond memes and the fun and creative possibilities they offer. They mention the limitations of these tools but celebrate the progress made. The speaker encourages viewers to stay updated with AI innovations through Futurepedia, a resource for finding the best AI tools and learning how to use them with curated tutorials. They conclude by inviting viewers to subscribe for more content like the video they are watching, promising to continue exploring AI advancements in future videos.

Mindmap

Keywords

πŸ’‘AI Video

AI Video refers to the use of artificial intelligence to create or enhance video content. In the script, AI video is the central theme, with the speaker discussing various AI tools that are revolutionizing video creation, such as Runway Gen 3, Luma Labs, and others. The script highlights the advancements in AI video technology that make it an exciting and fun time for content creators, allowing them to produce professional-grade videos with relative ease.

πŸ’‘Runway Gen 3

Runway Gen 3 is a text-to-video model mentioned in the script as being the best available for creating video content from textual descriptions. The speaker uses it to generate title sequences and demonstrates its capabilities with examples, showcasing how it can interpret prompts to create dynamic and visually appealing video content, such as the title sequence for 'future pedia'.

πŸ’‘Luma Labs

Luma Labs is highlighted in the script as a key player in the AI video space, particularly for its 'dream machine' tool that excels in image-to-video transformations. The script discusses how Luma Labs' technology can take a static image and create a dynamic video sequence, with examples provided to illustrate the quality and creativity of the results.

πŸ’‘Text-to-Video

Text-to-video is a technology that converts textual descriptions into video content. The script discusses this technology in the context of Runway Gen 3 and Luma Labs, emphasizing how it simplifies the video creation process. The speaker demonstrates the effectiveness of text-to-video models by generating title sequences and transforming scenes based on textual prompts.

πŸ’‘Keyframes

Keyframes are specific frames in an animation or video that define the start and end points of a transition or motion. In the script, the speaker explains how Luma Labs uses keyframes to create smooth transitions between images, allowing for the generation of complex video sequences with logical progressions between scenes.

πŸ’‘LTX Studio

LTX Studio is an AI platform mentioned in the script for its ability to create short films quickly from scripts or prompts. The speaker describes how LTX Studio offers a high level of control and customization, allowing users to input scripts, select styles, and generate fully editable scenes, making it a powerful tool for rapid video production.

πŸ’‘Style Reference

A style reference in the context of AI video creation is a sample or template that defines the visual aesthetic of the generated content. The script mentions the 'style reference feature' of LTX Studio, which allows users to upload their own style references to influence the visual style of the generated video, providing a more personalized and consistent look across scenes.

πŸ’‘Morphing Animations

Morphing animations refer to a type of animation where one image or object gradually transforms into another. The script discusses Kaa, a platform that specializes in creating abstract, trippy morphing animations. The speaker demonstrates how Kaa can take keyframes of different images and generate smooth transitions between them, resulting in unique and visually striking animations.

πŸ’‘Lip Syncing

Lip syncing is the process of matching an audio track, especially speech, with the mouth movements of a character or avatar in a video. The script mentions tools like Hedra and Live Anime that facilitate lip syncing, allowing users to upload audio and generate videos where the avatar's mouth movements are synchronized with the audio, creating a more realistic and engaging viewing experience.

πŸ’‘Creative Upscaling

Creative upscaling is a process that enhances the resolution of a video while also reimagining its content, often adding artistic flair or altering the style. In the script, the speaker demonstrates Kaa's video upscaler, which not only increases the resolution but also applies a new artistic style to the video, such as cinematic or animated, offering a fresh take on the original footage.

πŸ’‘Open Source Models

Open source models refer to software or algorithms that are freely available for use, modification, and distribution. The script acknowledges the contributions of the open source community to the field of AI video, mentioning tools like Comfy UI and animate diff that have laid the groundwork for some of the paid platforms discussed. These models have enabled a wider range of creators to experiment with AI video technologies.

Highlights

AI video tools are currently in an exciting phase with advancements in text-to-video models.

Runway Gen 3 is considered the best text-to-video model available, excelling in creating title sequences.

Runway's Gen 3 is effective for fluid simulations and physics in video generation.

The video creator demonstrates how to use Runway Gen 3 for generating title sequences.

Dream Machine from Luma Labs is praised for its image-to-video capabilities, especially with keyframes.

Luma Labs' platform allows for easy image uploads and prompt additions for video generation.

LTX Studio offers the most control and speed, enabling short film creation from a script or prompt.

LTX Studio's style reference feature allows for consistent character generation throughout a story.

Kaa is highlighted for its unique focus on abstract and trippy animations, offering a different creative avenue.

Kaa's creative upscaler reimagines videos with AI, offering a new take on traditional upscaling.

Hedra and Live Portrait are platforms for lip-syncing avatars, offering expressiveness and customization.

Live Anime allows mapping a reference video onto an avatar for lip-syncing, providing control over expressiveness.

The open-source community's contributions to AI video tools are acknowledged for their pioneering efforts.

Cling is mentioned as a platform with quality comparable to Runway and Dream Machine but with a significant waitlist.

AI video tools have come a long way and are now usable for real-world applications beyond memes.

The video creator emphasizes the fun and creative potential of current AI video tools.

Futurepedia is recommended as a resource for staying updated with AI innovations and tutorials.

Transcripts

play00:00

I've been experimenting with AI video

play00:01

tools and covering them on this channel

play00:03

for a long time and right now is the

play00:06

most exciting and fun time in AI video

play00:09

yet Runway and Luma Labs have been in

play00:10

the spotlight and I'll cover some

play00:12

immediate real world use cases for them

play00:14

but my personal favorite tool is

play00:16

actually not one of those making the

play00:17

headlines I'll get to that a little

play00:18

later plus some amazing lip syncing

play00:20

tools and some open source models

play00:22

there's been a lot of advancements I've

play00:24

got to start with Runway gen 3 since

play00:26

it's been dominating my timeline it is

play00:28

the best textto video model available to

play00:30

use right now something that's

play00:31

particularly useful with Gen 3 is text

play00:33

like these title sequences it is amazing

play00:35

at this I'll show a few of my favorite

play00:37

examples I've

play00:39

[Music]

play00:40

seen this is such a perfect title

play00:43

sequence for a cooking

play00:45

channel this one also showcases how good

play00:48

it is at fluid simulation the physics in

play00:50

a lot of these is really good adding

play00:52

some sound design would take these to

play00:53

the next level for sure and be

play00:55

production ready title sequences so I'll

play00:57

try one for future pedia I'm logged in

play00:59

already and here's a whole Suite of

play01:00

image and video tools that Runway offers

play01:02

but for Gen 3 I'll just click right here

play01:05

then type in a prompt a title screen

play01:07

with Dynamic movement the scene starts

play01:09

with intricate neon circuitry patterns

play01:11

that light up and move on a dark

play01:13

background suddenly the circuits

play01:14

converge and form the word future pedia

play01:16

with a glowing pulsating effect then I

play01:18

can choose between 5 seconds and 10

play01:20

seconds I'll lower it to five then click

play01:22

generate that's all there is to it right

play01:24

now this is how that came

play01:27

back and that looks amazing since

play01:31

futurepedia is a longer word I have

play01:33

gotten some misspellings but it gets it

play01:34

right A lot of the time this prompt

play01:37

structure has worked really well so I

play01:38

used one of the sample prompts in the

play01:40

Gen 3 prompting guide and modified it I

play01:42

modified it a lot really just kept the

play01:43

basic structure so I'd recommend trying

play01:45

to use the prompt structure they give

play01:46

here when you're starting out camera

play01:48

movement establishing scene then

play01:50

additional details hopefully that will

play01:51

help you cut down on rolls they do also

play01:53

have a lot of solid keywords in this

play01:55

guide to help give you some inspiration

play02:00

another thing it's really good at is

play02:01

transforming between scenes this is

play02:03

another prompt you can modify for

play02:05

yourself that comes back with great

play02:06

results pretty

play02:08

[Music]

play02:12

consistently I will copy that and paste

play02:15

it in

play02:16

here I'll change this to a wormhole and

play02:19

into an alien civilization

play02:21

[Music]

play02:30

that's not bad with some rerolls and

play02:32

modifying The Prompt I'm sure I could

play02:33

get a better generation here's a couple

play02:35

other examples using this same prompt

play02:37

[Music]

play02:43

[Music]

play02:48

[Music]

play02:54

idea now I've also had a lot of Misses

play02:57

with runaway 2 that's the biggest caveat

play02:59

here the amazing results you see posted

play03:01

are definitely cherry-picked that's the

play03:03

case with all AI video models including

play03:05

Sora but Runway can do some amazing

play03:07

things especially if you have a lot of

play03:08

credit here's a few more of my favorite

play03:10

Runway examples this was really hard to

play03:12

narrow down

play03:13

[Music]

play03:33

stop

play03:36

[Music]

play04:11

the best image to video right now is

play04:13

dream machine from Luma Labs you can

play04:15

also do text to video which gets some

play04:16

good results but where Luma shines is

play04:18

image to video and even better than that

play04:20

is with key frames I'll show a couple

play04:21

straight image to video examples first

play04:23

it's easy to use upload an image then

play04:26

add a prompt volcano erupting contained

play04:29

within a drinking glass surrounded by

play04:31

peaceful

play04:33

Tranquility this was the original image

play04:35

then here's the result and that looks

play04:38

perfect I'll show a few more straight

play04:40

image to video examples then we'll move

play04:41

on to key

play04:53

frames these are all within one or two

play04:55

tries it does weird things on occasion

play04:57

like this astronaut grows in extra

play05:00

finger or here's a weirder

play05:05

one or how about a much weirder

play05:10

one most of the time they do look good

play05:12

without having to do too many rolls the

play05:15

next step is adding an ending frame and

play05:16

you can start doing some really cool

play05:17

stuff you upload a starting frame then

play05:20

also upload an ending frame now add a

play05:23

prompt for it to use to create the whole

play05:24

clip in between those frames this works

play05:26

most consistently between things that

play05:28

are logical to transition between if

play05:30

they're not then try to be extra

play05:31

descriptive of what you want to happen

play05:33

in between it works great for

play05:35

transformation shots that are in the

play05:36

same or a similar

play05:39

scene but you can also do more out there

play05:45

[Music]

play05:55

transitions this one looks like it could

play05:57

legitimately be a Pixar movie

play06:01

you can extend that out by just taking

play06:03

your ending frame and using that as the

play06:05

starting frame for a new generation then

play06:07

adding a new ending frame you can do

play06:09

really long sequences this way there's

play06:10

been these long mashups of memes that

play06:12

people have been posting which are fine

play06:15

I am definitely more interested in the

play06:16

creative stuff this with this girl and

play06:18

the Wolf is one of the coolest I've seen

play06:20

using that

play07:04

it's amazing to keep that all consistent

play07:05

and there's a ton of possibilities for

play07:07

this you can do this for free but the

play07:09

generations take a long time like

play07:11

multiple hours when I tried before

play07:12

signing up paying will get you to the

play07:14

front of the line and a lot more

play07:16

Generations there's one more out of

play07:17

China that's similar in quality to these

play07:19

two called cling however there is a huge

play07:21

weight list for it and the process to

play07:23

sign up is difficult without a Chinese

play07:25

phone number so I'm going to cover that

play07:26

one at the end moving on to LTX Studio

play07:29

they they have the most control out of

play07:31

any of these platforms and the fastest

play07:33

speed they can build out an entire short

play07:35

film in a few minutes how it works is

play07:37

you can start from scratch input an

play07:39

entire script or start from a simple

play07:40

prompt I'll go with the prompt option

play07:42

for this example I will paste this in

play07:45

it's about a futuristic city controlled

play07:46

by AI entities and a hacker that can

play07:49

communicate with them he joins a

play07:50

resistance they battle all that good

play07:52

stuff hit next and it starts

play07:55

working it will show the basic story and

play07:58

the cast then you can select a style so

play08:00

this is Cinematic but let's say I want

play08:02

an anime I can direct it more with words

play08:05

as well I'll add

play08:07

cyberpunk there's all sorts of other

play08:10

styles here to

play08:15

try I like comic book it will use these

play08:18

as consistent characters throughout the

play08:20

story which is amazing because that's

play08:22

usually a struggle and I can change

play08:24

anything about them that I want I think

play08:25

this will be better without a last name

play08:27

I could change the essence appearance

play08:29

the clothes I can test out different

play08:31

voices if I want as an organizer I start

play08:34

from where the world is as it is not as

play08:37

I would like it to be you can even face

play08:39

swap these with a face you upload I want

play08:42

to change the title to how about Neon

play08:44

Flux see how many people know that

play08:45

reference to the anime not the movie

play08:47

they made from it this looks good here

play08:49

so I'll click Start and just from that

play08:51

it builds out an entire short film with

play08:53

fully editable

play08:55

scenes this whole thing was generated in

play08:58

less time than it took either of the

play08:59

others to generate a single clip and I

play09:01

want to show the style reference feature

play09:02

again now that this is open because it's

play09:04

even cooler to do it in here I'll try

play09:06

uploading my own style reference so this

play09:07

is a mid-journey image when I upload

play09:10

that style every shot across every scene

play09:12

regenerates in the new style that's

play09:14

super helpful and doing it here will

play09:16

give you a much better idea of how it

play09:17

will look and that looks awesome I'm

play09:19

actually going to run with that style

play09:21

now these are just static images

play09:22

currently I could generate them all into

play09:24

videos at once or open each one to

play09:26

customize it that gives you the most

play09:27

flexibility so I'll do that I can

play09:29

regenerate the image or upload my own

play09:31

even generate or remove objects from the

play09:33

image I'll just get rid of this little

play09:35

blob

play09:36

[Music]

play09:38

here I can choose from four types of

play09:40

motion and adjust how much of it most of

play09:42

the options are pretty straightforward

play09:43

but this one is really amazing when you

play09:45

switch to custom then orbit then you can

play09:47

have total control over the start and

play09:49

end of the shot and it will generate the

play09:51

motion in

play09:52

[Music]

play09:56

between you can also adjust how long the

play09:59

clip is I want this to be like a movie

play10:00

trailer so 3 seconds will be

play10:05

plenty you can also add music sound

play10:08

effects it generates a voice over all of

play10:10

that can be customized as well if it

play10:11

hasn't hit home yet everything can be

play10:13

customized I'll actually change the

play10:14

voiceover a little because this ended up

play10:16

telling the entire story I didn't

play10:18

mention in the prompt that it was for a

play10:19

trailer I don't want to spoil the ending

play10:21

the track it used is awesome so I'll

play10:22

keep it that's a lot of the features

play10:24

probably enough for this showcase so I'm

play10:25

going to go through and generate each

play10:27

scene how I want it then come back all

play10:28

right finished all the scenes it's

play10:30

really nice that you can export all the

play10:31

files so I can bring it into premiere

play10:33

for more fine grained editing the other

play10:34

really cool option is to export it as a

play10:36

pitch deck I'll do that as well first

play10:39

here's the pitch deck it has the story

play10:41

all of the characters and their

play10:44

backgrounds the mood board and color

play10:47

palette then the full story board people

play10:49

have already made some amazing things

play10:51

this way where they storyboarded a short

play10:52

film with LTX then actually filmed and

play10:55

produced the entire thing they posted

play10:57

this one called winged on X where this

play10:59

turns into this like human head with

play11:01

bird wings it's pretty good you should

play11:03

go watch

play11:04

it then with this premere option it

play11:07

gives me the project file that I can

play11:08

edit further I already opened that and

play11:10

synced things up to the music a little

play11:11

more and made just a couple other simple

play11:13

edits here is the final result in the

play11:16

neon lit City I discovered a secret

play11:18

power the ability to communicate with AI

play11:23

Spirits joining the resistance I became

play11:26

a spirit Guardian fighting alongside

play11:28

comrades unique Tech

play11:32

Powers we Face a rogue AI threatening

play11:35

our City's balance blending ancient

play11:37

wisdom with cuttingedge technology as

play11:39

the battle intensifies the fate of our

play11:41

world hangs in the

play11:45

balance I've been having a ton of fun

play11:47

with LTX studio and side note I did

play11:49

actually meet some people from the team

play11:51

at the Imagine AI conference a while

play11:52

back at one point they were just giving

play11:53

a little demo at their booth and like

play11:55

everyone crowded around to watch they

play11:57

really stole the show make sure to sign

play12:00

up for the weit list they're letting

play12:01

lots of people in the link is in the

play12:02

description and thank you to LTX for

play12:04

sponsoring this video the platform I've

play12:06

been having the most fun with out of all

play12:08

of these is Korea and with this one you

play12:09

can do quite a bit for free it's a lot

play12:11

different than the other ones we've

play12:12

covered it's more for abstract stuff not

play12:14

so much for realism so it focuses more

play12:17

on these like trippy morphing type

play12:18

animations which I like a lot personally

play12:21

I like to use AI to create things that

play12:23

would be impossible to create any other

play12:25

way to open up entirely new creative

play12:27

Avenues rather than replace EX in ones

play12:29

so I have a ton of fun in Kaa I'll go up

play12:31

to generate then video they also have a

play12:34

creative upscaler that's really useful

play12:36

I'll show that one next these are the

play12:37

three images I want to use they're these

play12:39

translucent kind of bioluminescent

play12:41

flower jellyfish and dragon I think they

play12:44

look cool together so I'll click add key

play12:46

frame select the flower then I'll add

play12:49

another key frame for the

play12:51

jellyfish and another for the dragon you

play12:55

can make these longer or shorter I'll

play12:57

lengthen it a little bit

play12:59

then you can add a text prompt I just

play13:01

want these to morph into each other so I

play13:03

don't need any longer prompt I'll just

play13:05

say what they are then drag the length

play13:07

of the prompt and that's where the

play13:08

transitions will start at least that's

play13:10

where they're supposed to it's not

play13:11

perfect there's a couple other settings

play13:13

for the aspect ratio motion intensity I

play13:16

like it around 60 usually then you can

play13:18

switch the looping on or off it defaults

play13:20

to on where the end will morph back into

play13:22

the first frames so it can just

play13:23

endlessly Loop then you have four styles

play13:25

to choose from I'll start with film then

play13:27

click generate video and I'll generate

play13:29

one at each of those for comparison in a

play13:30

second this will do a first pass at

play13:32

generating then give a preview at a low

play13:34

quality that'll give you a good idea

play13:36

what it will end up looking like if it's

play13:38

not going in the direction you'd like

play13:39

you can go to where it says enhancing

play13:41

and select click to cancel that will

play13:43

save you some generation

play13:44

time now I actually think it's moving

play13:46

away from the jellyfish too fast so I

play13:48

will cancel I'll add another key frame

play13:50

and select the jellyfish again so it

play13:52

hangs out on that for longer then we'll

play13:54

generate again that cancel option is

play13:56

especially helpful if you're on the free

play13:57

plan since your Generations are more

play13:59

limited I am on the basic plan which

play14:01

gets plenty of time for me in about a

play14:02

minute it's all done and here's what I

play14:05

[Music]

play14:08

got I think that looks really cool now

play14:11

I'll generate with render animation and

play14:13

experimental so you can see the

play14:14

difference

play14:18

[Music]

play14:33

I think these are amazing really fun and

play14:35

unique here's another quick example I

play14:37

did with these

play14:42

[Music]

play14:47

images I am working on some longer stuff

play14:50

with tracks from suo but they're not

play14:51

finished yet so here's one more weird

play14:54

one I did

play15:02

I love this style I know it's not for

play15:03

everyone but it's really great for

play15:04

something like a music video or an

play15:06

opening sequence but these are a couple

play15:08

of my favorite examples I found on x

play15:14

[Music]

play15:26

[Music]

play15:29

oh

play15:34

[Music]

play15:41

[Music]

play16:00

now I also want to demo their video

play16:01

upscaler this is not like a traditional

play16:03

upscaler where it resembles the original

play16:05

video as much as possible just at a

play16:07

higher resolution it does a creative

play16:09

upscale similar to how magnific works

play16:11

but for video it stays really close but

play16:13

kind of reimagines everything with AI so

play16:16

I'll use a clip from LTX video I made

play16:18

this one has a face that was just

play16:19

completely warped we'll see if it can

play16:21

fix

play16:22

that I don't need it upscaled that much

play16:25

I'll leave that at 1.5 frame rate 30 FPS

play16:28

write out a prompt of what it thinks is

play16:30

in the video that looks good so I'll

play16:31

leave the strength and resemblance at

play16:33

the defaults first and start with

play16:35

cinematic turn Loop off and enhance this

play16:39

took around 30 seconds and here's the

play16:41

result you can see that there's a face

play16:42

on here now that's pretty amazing it was

play16:45

able to fix that I ran one at each of

play16:47

the different presets so you can see the

play16:48

difference cinematic and render are

play16:50

fairly similar then animated makes it

play16:52

more of an anime style I'll show that

play16:54

with a few clips from that LTX video

play17:03

[Music]

play17:05

I hope to see them expand that into more

play17:07

Styles because it is very useful overall

play17:09

I think Kaa is a ton of fun and there's

play17:11

some additional options to play around

play17:13

with in here as well lip syncing has

play17:15

made some big improvements recently

play17:16

there's tons of impressive demos that

play17:18

have come out that we don't have access

play17:19

to use but there's two platforms I want

play17:21

to show that we do have access to one

play17:23

that's completely free and one that's

play17:24

free for five uses per day it's hedra

play17:27

and live portrait hedra has some of the

play17:29

most expressive talking avatars I've

play17:31

seen it's pretty easy to use so either

play17:32

generate the audio or upload some of

play17:34

your own I'll use the classic Fight Club

play17:36

line now I'll upload a character you can

play17:38

also generate one here if you need now I

play17:40

just generate the video it works pretty

play17:42

quickly and here's what I

play17:45

got the first rule of Fight Club is you

play17:48

do not talk about Fight

play17:50

Club second rule of Fight Club is you do

play17:54

not talk about Fight

play17:57

Club that looks great super expressive

play18:00

let's do another one but with a

play18:01

non-human character this time I'll use

play18:03

the line from

play18:08

Taken what I do have are a very

play18:12

particular set of skills skills I have

play18:14

acquired over a very long career skills

play18:17

that make me a nightmare for people like

play18:21

you so that one looks great but it can

play18:23

definitely struggle with non-human

play18:25

characters and the less human they look

play18:27

the harder it is a dog just didn't work

play18:28

at all or even some of those gnomes

play18:30

especially the ones with a beard it was

play18:31

having a hard time with you do not talk

play18:34

about Fight Club Runway also has a lips

play18:37

syncing feature that they implemented it

play18:38

works really well very high quality but

play18:40

it's much less expressive the first rule

play18:42

of Fight Club is you do not talk about

play18:45

Fight

play18:45

Club second rule of Fight Club is you do

play18:49

not talk about Fight

play18:52

Club live anime is also amazing but

play18:55

takes a different route you upload a

play18:56

reference video and it will map that

play18:58

onto the Avatar this allows you to have

play19:00

more control over the expressiveness

play19:01

it's on hugging face so you can use it

play19:03

for free upload a source portrait I'll

play19:05

give my face a try then upload a driving

play19:08

video it works best with a straight on

play19:09

shot like the ones they have in these

play19:11

examples I'll just use one of those

play19:12

here's a pretty classic one although I

play19:14

won't be able to use the song that goes

play19:15

along with it on YouTube now click

play19:18

animate and in under 30 seconds here's

play19:21

the

play19:24

result that worked pretty perfectly I'll

play19:26

show an example that didn't work this is

play19:28

a CLI clip from Breaking Bad it shows

play19:30

his face the whole time but he does move

play19:32

a bit so with that it's just not able to

play19:34

map that onto my face at all so there's

play19:35

limitations here but you can still do

play19:37

some pretty great stuff with it things

play19:38

you would never be able to get through

play19:39

other methods currently and really quick

play19:41

here I want to shout out the open source

play19:43

community and the tools and workflows

play19:45

they've been developing and pioneering

play19:47

those have really been the foundation of

play19:48

some of these other tools that will

play19:49

mimic or outright take those workflows

play19:52

and Implement them into a paid model the

play19:53

things you can get from Korea and some

play19:55

of these other video models are possible

play19:57

with open source tools most notably

play19:59

comfy UI and animate diff that method is

play20:01

significantly more complicated although

play20:03

it offers more customization and control

play20:05

the paid platforms tend to offer a nice

play20:07

UI and ease of use so far all of these

play20:09

examples are from James G not sure if

play20:12

that's how you say his last name he has

play20:13

so many amazing Creations on Instagram

play20:15

I'm only showing short clips from them

play20:17

they're all much longer and have music

play20:19

to them as well so I have been following

play20:20

his work for a while and it is some of

play20:22

my

play20:24

favorite this marble statue dance was

play20:26

one of his really early ones to go viral

play20:28

his Instagram is full of amazing stuff

play20:30

but I'll show some other people too this

play20:32

dancing spider is

play20:35

[Music]

play20:38

awesome these have all been from driving

play20:40

these animations using real footage but

play20:42

that's not the only way to use this at

play20:44

all this one by Fab dream going through

play20:46

the evolution of money was

play20:52

[Music]

play20:53

great or these weird frog animations

play20:56

from boltron I love these

play21:00

[Music]

play21:03

to learn more about how to do this I am

play21:05

not the person to be teaching it but if

play21:06

you want to do a deep dive to install

play21:08

and run it locally and learn how to do

play21:09

all of that I'll have a couple resources

play21:11

in the

play21:13

description cling is the platform that's

play21:15

as good or better than Runway and dream

play21:17

machine with text to video and image to

play21:19

video but the weight lless is huge and

play21:21

the process is more complicated if you

play21:22

don't have a Chinese phone number so I'm

play21:24

going to link to a thread by tile AI he

play21:27

walks through the process of downloading

play21:28

a couple apps and getting a burner

play21:30

number to use I was able to sign up for

play21:31

the wait list using my own number on

play21:33

their mobile app but then the new

play21:34

website doesn't have an option for a

play21:36

non-chinese number to sign in so it is

play21:38

probably best to use his method and fair

play21:40

warning I signed up for the wait list

play21:42

about 2 weeks ago it said there were

play21:43

around 500,000 people on it I still

play21:46

don't have access and now the wait list

play21:48

says it has around a million people on

play21:50

it so it is free to use right now but I

play21:52

imagine that will change by the time

play21:53

people that are just now signing up for

play21:55

the wait list get access so it may not

play21:56

be worth the hassle that's why I didn't

play21:58

want to spend all time going over that

play21:59

here follow that thread if you want to

play22:00

try but the quality coming out of cing

play22:03

is amazing

play22:07

[Music]

play22:42

we are finally to a point where you can

play22:44

use AI to make video that's usable in

play22:46

the real world outside of memes not

play22:48

usable for everything of course there's

play22:49

still plenty of limitations these tools

play22:51

have come a very long way and are a ton

play22:53

of fun right now if you want to keep up

play22:54

with everything in AI make sure to visit

play22:56

futurepedia find the best AI tools stay

play22:59

up to date with Innovations learn how to

play23:01

use it all with curated tutorials

play23:03

there's tons of new features on the site

play23:05

so check it out if you haven't for a

play23:06

while then subscribe so more videos like

play23:08

this show up on your homepage thank you

play23:10

so much for watching I'll see you in the

play23:12

next one

Rate This
β˜…
β˜…
β˜…
β˜…
β˜…

5.0 / 5 (0 votes)

Related Tags
AI VideoRunwayLuma LabsLTX StudioLip SyncingOpen SourceText to VideoImage to VideoCreative ToolsTech Innovations