NEW Best AI Video Generator? Stable Video Diffusion vs Pika vs Runway ML

Summary

This video reviews the top AI video generators, comparing Stability AI's Stable Video Diffusion, Runway ML, and Pika Labs. It showcases the strengths and weaknesses of each tool, highlighting Runway's advanced motion brush, Pika Labs' creative features, and Stable Video Diffusion's open-source accessibility. The video also includes examples of AI-generated videos, from realistic animations to imaginative visual effects. Viewers learn how to use these tools for free and are encouraged to explore AI-generated videos. The video ends with a comparison of Runway and Stable Diffusion, noting their unique capabilities for different purposes.

Takeaways

• 🎥 Stability AI released Stable Video Diffusion, an open-source AI video generator.
• 🏆 Runway ML is currently the leading AI video generation tool but comes with a price tag.
• 💡 Runway ML offers three video generation methods: text-to-video, image-to-video, and video-to-video.
• 🔄 The 'motion brush' feature in Runway ML allows users to selectively animate parts of an image.
• 💸 Pika Labs is a completely free AI video generator with easy-to-use features and simple prompts.
• 🏅 Pika Labs excels in generating creative, animated videos from still images with features like Parallax movement.
• 🚀 Stable Video Diffusion is free and offers high-quality, realistic video results, with adjustable frame settings.
• 👾 Stable Video Diffusion can create impressive and realistic videos, especially in complex scenes like rain and cityscapes.
• 📊 Both Runway ML and Stable Video Diffusion offer distinct advantages—Runway provides more dynamic motion, while Stable Diffusion has a more 2D, stable look.
• 🌐 Stability AI's tool is open-source, meaning it can be integrated into other software, allowing broader use and customization.

Q & A

What is Stability AI's new release called?

Stability AI's new release is called Stable Video Diffusion, which is described as the first open-source AI video generator.

How does Stable Video Diffusion compare to Runway ML and Pika Labs?

Stable Video Diffusion offers a different approach, providing open-source AI-generated videos for free. In comparison, Runway ML provides advanced features like motion brush, but it's a paid service. Pika Labs is also free and focuses on ease of use with features like slash encrypt image, but requires more precise prompting for best results.

What is the main drawback of using Runway ML for AI video generation?

The main drawback of using Runway ML is that it's a paid service and can become expensive, though it provides powerful features like text-to-video, image-to-video, and video-to-video generation.

What are the three different ways of generating AI videos using Runway ML?

Runway ML allows users to generate AI videos through text-to-video, image-to-video, and video-to-video methods.

What is the preferred method for generating videos with Runway ML?

The preferred method for generating videos with Runway ML is image-to-video, where an image generated by tools like MidJourney or DALL-E is used to create a video.

What unique feature does Runway ML have that no other AI company currently offers?

Runway ML is the only AI company offering the motion brush feature, allowing users to selectively add motion to specific areas of an image or video.

What is the best approach for creating AI videos using Pika Labs?

For Pika Labs, it's better to input an image from AI tools like DALL-E or MidJourney and then describe how you want the camera or object to move, rather than relying on text-to-video.

How can users access Pika Labs for free AI video generation?

Users can access Pika Labs for free by joining their beta via the Pika Labs Discord server and generating videos in the designated rooms.

What are the key strengths of Stable Video Diffusion according to the examples shown?

Stable Video Diffusion excels in generating realistic videos, especially in terms of environmental details, camera shake movements, and handling complex elements like water and lighting, making it highly effective for dynamic shots.

How does the Viking woman image differ when processed by Runway ML and Stable Video Diffusion?

In the comparison, the Viking woman image in Runway ML has more spatial warping and dynamic camera movements, while the Stable Video Diffusion version offers a smoother, more 2D-like feel with less camera movement.

Outlines

00:00

🎥 Introduction to Stable Video Diffusion and AI Video Tools

This paragraph introduces Stability AI's release of Stable Video Diffusion, the first open-source AI video generator, and compares it with other tools like Runway ML and Pika Labs. The narrator provides a step-by-step guide on using these tools and offers insights into their strengths and weaknesses, emphasizing that Runway ML is currently the leading AI video generator but comes at a cost. The paragraph also shows examples of videos created with Runway ML, illustrating the potential of AI in video generation.

05:02

🦸 Runway ML in Action: Exploring AI Video Capabilities

The paragraph dives into Runway ML's advanced features, such as text-to-video, image-to-video, and video-to-video generation. The narrator shares examples of AI-generated videos from Runway, highlighting the platform's ability to create realistic b-roll footage. The narrator tests Runway's motion brush feature, which allows selective motion in videos, showcasing its impressive results, like a Viking woman on a mountain. The paragraph explores various examples to show how Runway ML adds camera motion and general motion to still images, making them look dynamic and real.

10:03

👾 Pika Labs: A Free and Simple AI Video Generator

The paragraph shifts focus to Pika Labs, an entirely free AI video tool that offers unique features like 'slash encrypt image.' The narrator explains how Pika Labs can be accessed through its Discord server and discusses the types of prompts that work best, such as guiding camera movements and object motions. The narrator also tests Pika Labs by generating videos of a Viking woman and emphasizes that while it may not be as polished as Runway ML, it’s a great option for free use. Examples include the narrator refining a prompt and creating more dynamic video effects.

🏞️ Stable Video Diffusion: An Open-Source AI Video Tool

In this paragraph, the narrator showcases examples created using Stable Video Diffusion, such as a man on a mountain, a realistic train, and a robot. The open-source nature of Stable Video Diffusion allows users to freely experiment with AI video generation. The narrator explains the process of inputting images, adjusting frames, and using motion bucket settings to enhance the video. The results, while impressive, are compared to Runway ML, with the narrator pointing out differences in camera movements and rendering quality, emphasizing that both tools are strong in their own right for different purposes.

Keywords

💡Runway ML

Runway ML is an AI video generation tool that allows users to create videos using text-to-video, image-to-video, and video-to-video methods. It is considered one of the best in the industry due to its high-quality results, especially in tasks like motion brush, which allows specific areas of the image to move. In the video, the speaker compares its features and limitations, particularly its paid nature, with other free tools.

💡Stable Video Diffusion

Stable Video Diffusion is introduced as the first open-source AI video generator from Stability AI. It's a free tool that allows users to create AI-generated videos from images. The speaker discusses its features and compares it with other AI tools like Runway and Pika Labs, noting that while it's free, its results are still in the development phase and aren't as polished as Runway ML's.

💡Pika Labs

Pika Labs is another AI video generation tool mentioned in the video, notable for its ease of use and free availability. It allows users to animate images generated from other platforms like DALL·E or MidJourney. The speaker shows examples of impressive video shots and emphasizes the tool’s simplicity and features like 'slash encrypt image,' which adds versatility to animations.

💡Motion Brush

Motion Brush is a unique feature in Runway ML that allows users to apply movement to specific parts of an image. In the video, it’s highlighted as an advanced tool for creating subtle, realistic animations, especially for shots like drone footage, where parts of the image can be animated while the rest remains static. It demonstrates the technical sophistication of Runway ML compared to other tools.

💡Text-to-video

Text-to-video is a method of generating videos by providing a text description that AI then interprets and visualizes. This feature is common in tools like Runway ML and Pika Labs. In the script, the speaker discusses the utility of this feature but points out that for more refined results, using image-to-video generation may produce higher quality videos.

💡Image-to-video

Image-to-video is a method where users upload an image, and AI software animates it. It’s a core feature in Runway ML, Pika Labs, and Stable Video Diffusion. The speaker notes that this method often yields the best results when using AI-generated images from tools like MidJourney or DALL·E, particularly in creating subtle movements like zooms or parallax shifts.

💡Prompt addiction

‘Prompt addiction’ is a playful term used by the speaker to describe the addictive nature of experimenting with different AI prompts to generate creative and diverse video outputs. Users get hooked on generating multiple versions of animations by tweaking text inputs and settings, especially in tools like Runway ML and Pika Labs.

💡Free AI video generation

This concept refers to the ability to generate videos using AI tools without paying for subscriptions or credits. Pika Labs and Stable Video Diffusion offer free versions, unlike Runway ML, which requires payment after a limited number of credits. The video focuses on the advantages of using these free tools for creative purposes.

💡Camera motion

Camera motion refers to the simulated movement of a camera within a scene created by AI. Users can control the direction of the motion (horizontal, vertical, zoom) to make their AI-generated video more dynamic. This feature is highlighted in Runway ML as a way to make videos more engaging by adding professional-looking camera movements.

💡Open-source AI

Open-source AI refers to AI software whose source code is publicly available, allowing developers to modify and integrate it into other applications. Stable Video Diffusion is highlighted as an open-source AI tool, making it accessible for further development and customization by the community, contrasting with proprietary tools like Runway ML.

Highlights

Stability AI just released Stable Video Diffusion, the first open-source AI video generator.

Runway ML is currently the leading AI video generator, offering text-to-video, image-to-video, and video-to-video generation.

Runway ML's unique feature is the motion brush, allowing specific parts of an image to be animated.

Runway ML offers a free plan with limited credits, with paid options for extended use.

Pika Labs is completely free and easy to use, with features like text-to-video generation and slash-encrypt image functions.

Pika Labs recently held a Halloween contest, showcasing impressive user-generated films.

Pika Labs allows you to batch images for video creation, adding a creative edge with various camera and object motion effects.

Stable Video Diffusion offers realistic and consistent video generation, even with complex textures and motions.

Stable Video Diffusion excels in rendering faces and architecture, which often appear more consistent compared to other generators.

Stable Video Diffusion's open-source nature allows integration into other software, making it highly accessible for creators.

Runway ML and Stable Video Diffusion offer different strengths: Runway excels in camera movement, while Stable Video Diffusion provides smoother, more static effects.

Pika Labs emphasizes user input, allowing prompts for camera angles and object movements that lead to dynamic video effects.

Stable Video Diffusion offers fast rendering times with impressive results straight out of the box.

Runway ML and Stable Video Diffusion provide different video styles—Runway often has more dynamic camera movements, while Stable Diffusion emphasizes steady and realistic transitions.

Both Runway ML and Stable Video Diffusion offer distinct advantages based on the type of project, making them useful for different creative applications.

Transcripts

step number one drag your image in here

step number two click on run step number

three enjoy your free AI generated video

stability AI just released stable video

diffusion the first opsource AI video

generator but how good is it really I'll

show you how it compares to Runway ML

and Pika Labs the 10 best examples I

found and how you can use it right now

for free first I want to start showing

you the current champion of AI video

generation called Runway so let's just

quickly look at a video the main setback

is that it's paid and it can get quite

expensive

[Music]

so that was a video of them showing

motion brush and they're the only AI

company that is doing this right now but

let me show you some more examples of

how crazy it actually is in the unspoken

world of wols things aren't always as

they seem this squirrel is right in

front of me oh my God oh my God okay

stay calm meet Charlie who's all about

welcome gifts under the guise of what's

yours is mine and what's mine is also

mine meet princess fluff in her world

every couch is a stage and every Sunny

Spot as a spotlight for her next big

snoozefest I'm not spoiled I'm just

selectively pampered meet Gus who about

every hour or so claims he has an just

look how insanely good this is right

here are more realistic images look at

this

I don't know about you but if this was a

b-roll shot in somebody's story mixed in

together with other camera footage I

probably couldn't tell with Runway ml

you can do text to video you can do

image to video or you can do video to

video three different ways of generating

AI videos what I found to be the best is

the image to video so gener an image in

mid Journey or Dolly and plug it into

this software but the price here is free

forever 125 credits which you're going

to run through really quickly $12 a

month

$625 $28 a month

2250 as you can see I'm now on the

inside I'm going to click on start with

image and I'm going to take one of these

files of a house from sort of a drone

angle and then all I'm going to do is

click on generate you could also use

image plus description

but usually it doesn't really work to

add anything here it ends up warping

into a different thing instead of

actually what you want which is just

some movement right generate 4 seconds

in the settings they have upscale remove

Watermark they also have General motion

so you can apply more motion to it they

also have camera motion that you can

change if you want it horizontal

vertical up down basically and now the

new motion brush where we can select an

area that we want to move more now I'm

just going to let this render and here

is the

result that's really good just slight

bit of movement it feels like it's from

a drone that's pretty cool let's

obviously try a viking woman at the top

of the mountain holding an axe and I'm

going to slide the general motion to 10

and this time actually let's try to do a

little Zoom generate and here is the

result I don't know about that wa where

do you want me to look Runway damn you

ain't fooling me also let's see what's

going to do with this I'll put the

general motion to 10 but I'm not going

to put any camera motion this time and

let's see what

happens oh no way there is no way that's

so cool I had no idea it would turn out

this well yeah the prompt addiction is

real guys kind of has this explosion in

the beginning as well boom you're now

seen the capabilities of Runway but we

have two more tools that is better let's

find out let's find out next up we have

P Labs the best part about this is that

it's completely free and very simple to

use here's the first

[Music]

example simple straight to the point

they recently had a Halloween contest

where this film

won government officials insist this an

isol those with severe symptoms May

remain infectious Beyond 10 days oh my

God these things are

everywhere we will Contin to keep you

this

[Music]

story evening this is Claude Bennett

reporting hey that's pretty impressive

what about this shot right here

[Music]

all right I'm not trying to scare

anybody but this first movement right

here I mean such a cool establishing

shot to get this in camera is so hard to

do let's check out this example there's

a tail that lies Beyond these

walls we've never known its true name

but many of us simply refer to it as

of long ago of had a dream within the

heart of nothingness a time before

Darkness and Light had not yet been

born it was a dream so profound that so

felt like a witness to I just continued

to be amazed by this this one they Tred

to be more

realistic

[Music]

these shots are just mindblowing hamster

on the

[Music]

water come

on Jesus this last shot here is just

mindblowing I'm speechless am I the only

one that's Amazed by this what makes

Pika difference is that they have some

features like slash encrypt image that

makes a weight of for example apply this

image to that image Anu tried this with

the X logo they just announced that it

will get Pika 1.0 coming out soon they

made this awesome 3D animation teaser

Pika also has text to video however from

my experience it actually is way better

to just put an image that you made from

dolly or mid journey and you just put it

into the software to make it animate but

how can you do that for free right now

just go to p. art and click on Join Beta

this will take you to the Pika Labs

Discord server now that you're on the

inside all you need to do is to go into

any of these generate rooms and you can

see what everybody's generating like

this gecko having some movement you can

see the insane prompts that some people

are using for this one and if we scroll

all the way down just right SLC create

I'll click on plus one and image and I'm

going to drag this image of the house we

already did for me what I found the best

way to prompt is to just say how you

want the camera to move or how you want

the object to move let's try slight

Parallax movement of house drone shot

you can also add these Dash motion - FPS

or GS to refine your prompt they have

some really good prompting tips in this

prompt tutorial room but this is our

video Let's see all right slight

movement for sure let's try Viking woman

up top of the mountain with an axe

camera zooming in woman getting close

closer and we'll add some motion forward

to this all right let's see what we got

almost feels like she's shap shifting

into a bear the movement is really cool

though I'd love to see if they have more

seconds and the best way to do that is

actually just take a screenshot of the

last frame then we can just drag that in

there and we'll just do the same prompt

and we got the second part yeah there is

no bear in sight okay we got one last

image how do we prompt this one though

let's see if this one works let's see

yeah

maybe definitely not as good as Runway

but the movement is really interesting

it seems a little bit like he's moving

his mouth talking and the apps are

actually moving around so Pika one of my

absolute favorites it does take a little

prompting to get it right but it's

completely free and you can just batch a

bunch of images back to back to back and

just wait for the result is it better

than the newest stable video diffusion

let me show you some examples then how

you you can use it for free here's the

first example as you can see an FPS

shooter that looks very real almost like

from the game if you didn't double look

you would kind of think it is here's

another example man on top of the

mountain train going through this dark

city ooh a nice looking car and this

robot I really like the robot one the

pillow blanket and the shine on the

robot all looking very realistic here we

got the Next Generation kind of like a

miniature house zoomed all in with this

blur in the background so nice here's an

example of a woman in the rain very

realistic and I love the camera Shake

movement here gives it way more life and

also very realistic faces usually turn

out really wonky in some of these

softwares but here it seems very

consistent take a look at this one

flying through the city flying through a

cable hey why you got to do me like that

look at these results they're so the

movement and the faces and the

architecture kind of stays the same even

when you're moving around whoa the

stylization on this is wild so is this a

contender for the

best D look at that there's no way this

could be straight out of camera here's

an example of the image that they have

with mid journey and then

the video on the right see the olive oil

actually seems like it's moving

downwards and pouring into the salad

this more realistic right here seems

like New York or

something who almost perfect waves which

is very hard to

do the

waterfall yeah this looks really really

good

guys what is this even and the last

sample here look at this little two

second render of some camera Shake I

really like how the camera shaking and

everything moves along with it well

thanks to Louis C on Twitter for making

a replicate page where we can just put

an image in there and we can actually

use it I'll leave a link to it in the

description as per tradition around here

let's put this house in here they have a

couple of different settings like 14

frames or 25 frames

they can maintain aspect ratio or change

it to different things you can have a

maximum of 30 frames you can also have

motion bucket which basically increase

the overall motion and some of these

other settings as well but we're just

going to click on run take a timer to

see how long it actually takes but I

checked it out yesterday and it didn't

take that long it's starting and it's

going pretty fast and that's it 33

seconds and this is the footage what can

I say straight out of the box no

prompting completely free 30 seconds but

I'm not fully convinced yet okay we got

Viking woman we got the guy with a

prompt addiction let's drag her in here

let's just keep the basic settings and

run and here is the result yeah she's

not a princess anymore that's for sure

this one would more likely be a horror

film last boss or something and let's

try our favorite image of the day Mr

prompt addict and this is the result man

I love this image and how it turned out

Let's cross check it with Runway all

right let's see them back to back Runway

is on the left and the new stable

diffusion is on the right they are

definitely different the runway one

warps in space and the right one kind of

has a 2d feel on it it doesn't move the

camera around that much but to tell you

which one is better I can't they're both

really good for different purposes but

you can test it out in the description

down below for free right now I highly

encourage it check all your cool images

as well as being completely open source

meaning people are going to put this

into softwares and I'm loving every

single second of it if you are too

subscribe to the channel if you want me

to make a full AI movie I actually made

one here so check that out and I'll see

you there peace