AI Video Is About To Explode!
Summary
TLDRThe video discusses several AI video developments including Google's new Lumiere video model which generates videos in a more temporally coherent way, speculation that fully AI-generated 30-60 minute films may be available within months, a sponsored segment on Meshi which is a text-to-3D generator that can create models which can then be animated, some interesting hints that Midjourney may be moving into more 3D capabilities possibly based on the Media Molecule game creation tool Dreams, and coverage of Sim Francisco, an AI simulation of an entire city populated by AI agents with wants, needs and lifespans aimed at developing artificial general intelligence.
Takeaways
- 👨💻 Google announced a new video AI model called Lumiere that uses a spacetime diffusion architecture for improved coherence
- 🎥 YouTube's Matt Wolf predicts 30-60 minute AI-generated films that are coherent and enjoyable will be available within the next 2 months
- 🖥️ Meshi is a free AI text-to-3D model generator that can help you get started with creating 3D assets even if you're not an expert
- 🤯 Nick Beller created an awesome mixed reality animation by sculpting a 3D character in VR and compositing it into video footage
- 👀 Ex-Media Molecule co-founder Alex Evans is now working at Midjourney as a principal research engineer, possibly on 3D capabilities
- 🔮 Midjourney is building hardware focused on managing tens of thousands of virtual 3D spaces
- 🚀 AI simulation company Anthropic created an AI city called S. Francisco populated by agents with wants, needs and lifespans
- 🕹️ The AI agents in S. Francisco relax by playing old school Nintendo games
- 🧠 Anthropic's goal with S. Francisco is to achieve artificial general intelligence
- 📺 The Simpsons predicted AI cities would be created according to the video narrator
Q & A
What new video model did Google recently announce?
-Google announced Lumiere, a new video generation model that uses a spacetime diffusion model to generate videos all at once instead of frame by frame.
What are some of the capabilities of Lumiere?
-Lumiere can generate videos from text prompts with coherent motions like walk cycles. It can also stylize videos, do video in-painting and out-painting, and generate videos from images.
What did YouTube's Matt Wolf recently predict about AI-generated films?
-Matt Wolf predicted that 30-60 minute AI-generated films that are coherent and enjoyable will be available within the next 2 months.
What does the tool Meshy allow you to do?
-Meshy is an AI text-to-3D model generator that allows you to generate 3D models and textures from text prompts across different art styles.
How can you animate models made in Meshy?
-You can download Meshy models and bring them into Adobe Mixamo to auto-rig and animate them.
Who is now working at Midjourney and what might they be doing?
-Alex Evans, co-founder of Media Molecule, is now at Midjourney. He may be working on 3D capabilities for Midjourney.
What is Sim Francisco?
-Sim Francisco is an AI-generated city populated with AI agents that have wants, needs, and lifespans. It's an experiment in artificial general intelligence.
What graphical style is Sim Francisco designed in?
-Sim Francisco uses a graphical style similar to the South Park animation style.
What are some interactive elements in Sim Francisco?
-The AI agents in Sim Francisco can fall in love, play NES games, and have lifespans where they die.
What is the ultimate goal of Sim Francisco?
-The goal of Sim Francisco is to achieve artificial general intelligence by simulating agents in a virtual city.
Outlines
🎥 Google's New Lumiere Video Model Generates High-Quality Videos
Google has announced Lumiere, a new video generation model that uses a Spacetime diffusion model to generate high-quality, coherent videos all at once instead of frame by frame. It performs well at text-to-video, image-to-video, stylized video generation, in-painting, and out-painting.
🎞️ AI-Generated 30-60 Minute Films Coming in the Next 2 Months
YouTube's Matt Wolf has seen AI-generated 30-60 minute coherent and enjoyable films that will be publicly available in the next 2 months. He cannot share details due to an NDA but says the tech will be available for everyone soon.
📹 Meshi - An AI Text-to-3D Model Generator That's Easy to Use
Meshi is a free AI text-to-3D model generator with 200 monthly credits. It offers various art styles and can generate 3D models from text prompts quickly. The models can be downloaded and used in other 3D software. A coupon code for 20% off is provided.
🌆 AI Simulates a Dynamic City (San Francisco) With Virtual Agents
The Simulation has created an AI-generated city called San Francisco populated by virtual agents with wants, needs and lifespans. It looks visually similar to South Park but focuses on simulating urban dynamics and interactions to study artificial general intelligence.
Mindmap
Keywords
💡video generation
💡diffusion models
💡feature-length films
💡3D generation
💡virtual worlds
💡simulation
💡text-to-video
💡text-to-3D
💡video stylization
💡deepfakes
Highlights
Google has a new video model called Lumiere.
Lumiere uses a spacetime diffusion model to generate video all at once.
Matt Wolf predicts 30-60 minute coherent AI films will be available in the next 2 months.
Meshi is an AI text-to-3D model generator that is easy to use for beginners.
Meshi works well for generating cartoon and animate styles but struggles with ultra-realistic models.
Generated Meshi models can be textured with AI and animated in Mixamo.
A Media Molecule co-founder is now working at Midjourney, possibly on 3D features.
Midjourney is building hardware focused on collecting data for 3D.
Simulation created an AI city called S Francisco with autonomous AI agents.
S Francisco agents have lifespans, fall in love, and play Nintendo games.
S Francisco's visual style resembles previous Simulation South Park episode.
S Francisco runs continuously to achieve AGI like a hyper Sims game.
AR character sculpted in Dreams and composited onto video for cool effect.
Lumiere shows promise for coherent video stylization and image-to-video.
Upcoming full AI films predicted within Joe Russo's 2 year timeline.
Transcripts
hey everyone well we've got some pretty
big Bombshells in the world of AI video
namely Google has yet another new video
model and 30 to 60 Minute consistent and
enjoyable AI films coming soon we'll
talk plus we're going to take a look at
a tool that is going to be really
helpful if you are terrible at 3D like I
am I mean obviously if you're good at 3D
I think that this tool will be helpful
as well and I've got an interesting mid
Journey dive well maybe less of a dive
and more of a splunking into an area
that I don't think anybody else has
covered yet and just might show the
direction that mid Journey's future is
headed in all that plus an AI city of
the future that is already here all
right let's dive in kicking off Google
have announced Lumiere a new video
generation model now you might be
thinking to yourself didn't Google just
do this like a month ago and yes yes
they did about 3 weeks ago Google
dropped video poet a zero shot video
generator does anyone know what is
happening at Google right now does
anyone at Google know what's happening
at Google right now well anyhow as of
today we have Lumiere which does
something kind of different than
previous video generators namely This Is
A Spacetime diffusion model so we are
going to rip a hole into the SpaceTime
Continuum and see what makes Lumiere
different but first let's take a look at
sort of like the standard features uh
text to video for example it does a
really great job with walk Cycles like
this is uh astronaut on planet Mars he
doesn't look like he's like Moon walking
or sliding all over the planet handheld
shot of a woman walking through an
Autumn forest and an adorable puppy that
isn't morphing into something horrific
two other shots that I thought were
pretty exceptional were the Jack Russell
Terrier on a snowboard they actually
called out GoPro shot in this and it
definitely does have that characteristic
GoPro fisheye look the Lamborghini is
also really nice there are some
incoherencies in terms of the road and
maybe the physics of the car turning but
the fact that it's actually holding the
model of the car together and not like
morphing into a fire truck is impressive
image to video also looks looks very
good uh the characteristics and the
smile on the girl with the Pearl Earring
looks very naturalistic and I mean those
are five fingers on Sir Isaac Newton as
he's waving hello so funny enough they
also animated the famous flag over eima
shot uh which Google poet also did but
on the video poet side we kind of ended
up with like this weird gopher kind of
appearing out of nowhere it also has
this stylized generation which is
something that I don't think I've seen
before this wherein you can give the
model a reference image and then it will
generate videos in the style of that
reference image for example here we have
like this vector image and if you prompt
a bare twirling with delight you get
this and a cute bunny nibbling on a
carrot so it's definitely taking the
elements of that you know Vector
illustrat kind of style and applying it
over to the video model we've also got
video stylization kind of that gen one
thing where you know you can take an
input video and change it into a variety
of styles at first I thought that it was
actually completely segmenting out the
back ground but actually in further
reflection it does look like it does
change out the backgrounds for each one
of these like in the made of flowers we
have kind of like the Eiffel Tower in a
bouquet look back there made of flowers
for some reason is also super disturbing
to me you would think like made of
flowers it's that's nice and pleasant
but it actually comes off as like like
one step away from the corep virus in
The Last of Us it also does video in
painting and out painting and it looks
like it does that pretty well
considering you've got like this balloon
here with the mask on this side and it's
completely making up you know these
other balloons and the remainder of the
sky and Horizon the pizza example is
actually really impressive as well
considering that it has to generate not
only the top half of that pizza but the
hand Crossing into the Mast area and
dropping basil onto it um yeah that's
pretty cool so how is Lumiere different
from other video models and remember
this is my caveman brain reading the big
words in the paper so this is going to
be really simplified basically it all
comes down to this archit iture called
SpaceTime unit which allows for the
video to be created all at once as
opposed to I guess other models which
begin with an input frame and then have
an output frame and then generate key
frames between those the problem with
other video models as you can see in
this example is like you have your input
frame here and your output frame here
and then if you look down here there's
kind of this this temporal break in the
chain right here whereas with the
SpaceTime unet um you know we have
unbroken chain also because the video is
generated all at once in SpaceTime as
opposed to you know creating individual
frames from an in and an out it frees
the model up for a lot of other tasks
like video stylization in andout
painting and image to video so all of
those will be a lot more temporally
coherent as well so it all looks really
cool and super promising and of course
that leads to the question do we get to
play with it the answer is I don't know
I don't know if they release another
video model in like 3 weeks I'm just
hoping that they name it something like
on the nose like Kafka moving on are we
going to be seeing 32 60 Minute long
completely AI generated films sometime
this year or even trimming that down
sometime in the next 2 months well in a
recent tweet by YouTube's own Matt wolf
uh congratulations Matt you have gone
from reporting on the news to becoming
the news anyways Matt says yes the Tweet
reads in my 2024 predictions video that
video will be linked below I made a
comment that I don't think AI is going
to be creating fulllength shows and
films this year 30 to 60 Minute stories
that were coherent and enjoyable won't
be available this year I was wrong the
next two months will be wild winky face
now Matt can't go into a lot of details
on this due to the fact that he signed a
non-disclosure agreement but he was able
to field a few questions for example
when Dan the Man asked if this was going
to be stock footage type mashups uh
something that we saw with in video but
no Matt says he's talking about fully
generated videos additionally this
technology will be available for
everyone to use it's not going to be
locked behind some like private
firewalled gate in some you know Studio
mogul's house and lastly in terms of a
time frame as to when we will all see
this Matt says that he thinks that next
month people will see what he saw now I
can't comment too much on Matt's post
namely because I may or may not have
signed a very similar non-disclosure
agreement uh that said now would be a
good time to hit the Subscribe button
what I can say is that I keep thinking
about a news story involving Avengers
director Joe Russo in which he said that
he fully expected full AI movies within
2 years that story came out in April of
2023 so you know Tick Tock moving on if
you're not great with 3D or just
downright suck at it like I do uh I
think now is a really good time to start
learning some very basic fundamentals
about it because I do think that a lot
of creative AI is going to be moving in
that direction as the year prog
progresses I'll have some news on that
in just a little bit now I don't think
that you need like a 70-hour tutorial on
blender unless you're interested in it
in which case go for it you can start
with something a little bit simpler
which brings us to Meshi who are
sponsoring today's video so Mesi is an
AI text2 3D model generator it is free
with 200 credits per month so that's
perfect if you're just getting started
although it does have paid tiers
obviously as you ramp up and I do have a
coupon code for you as well we'll get
that in just a minute Mesi is very easy
to use you just come down to this like
text to 3D button here um you know you
can describe your object here put in a
style this would be sort of the
descriptors that you want for your
object and then negative prompts as well
below that you can choose from a number
of different art styles ranging from
realistic uh voxel which is kind of that
Minecraft um you know Lego block look um
all the way down to like realistic
handdrawn and cartoon line art uh I
quickly generated a up a spaceship and
yeah there you go a 3D spaceship now I
didn't give it too many details uh the
style was just highly detailed sci-fi
Unreal Engine and I did not give it any
negative prompts you do also have
texture options over here for color or
PBR uh PBR as I learned has to do with
sort of reflectivity I will say that
Meshi really excels when you're
generating things like props or if
you're aiming for more of a like cartoon
or animated style when you aim for the
realistic stuff it gets a little bit on
the wonky side we'll take look at that
in one second but first let's generate
us a cute Dragon so with the very simple
prompt cute baby dragon you end up with
four kind of like low res versions of
dragon options like we could go in and
do a lot of other stuff in the prompt to
call out certain colors or whatnot um
but once you find one that you like if
you hit this refine button you will end
up with a refined version so you can see
definitely the model has improved on a
lot of details so circling back to the
the realistic side or at least quote
unquote realistic in this case I did try
generating up a superhero full body pose
in a comic book style and we did get our
guy although you will see that there are
some issues uh going on in the face
there um that's that's fine that's if
that's where the technology is right now
that's where it is if you are somebody
that is good with 3D I'm sure that that
would be a super easy fix but for
someone like me I mean honestly the
simplest solution is just to start
generating Up characters that are either
wearing helmets or Mas like this kind of
Warhammer inspired Space Marine so if
you do end up with a model that you like
you just don't like the overall kind of
look or Vibe of it you can always
download it uh and just download it as
an fbx uh and then kick back over to the
main menu where you can use this AI
texturing module um you simply come and
hit new project uh we'll describe this
as robot uh and then upload your fbx
file here once that's in you see that
our model is here but textureless uh and
then you can come through and prompt for
what textures you would like to see for
example here we take our untextured
model and run it with the prompt a
cyberpunk robot with black metal armor
uh and yeah we get this which looks
pretty cool now you might be thinking
well that's kind of cool but what can I
do with that it's just a character
that's just sort of standing there I am
not a 3D animator for that we're going
to download our model and take it over
to adobe's miimo so Adobe miimo is a
completely free uh Auto riger for 3D
characters it basically builds you know
the skeleton for 3D characters as well
as provides a number of animations for
it actually don't know why mixim Mo
isn't bigger than it is it seems to be
kind of one of those like lost forgotten
experiments of adobe they actually do
provide a number of different characters
here but what's really cool is that you
can upload your own characters so uh
we're going to take the zip file that we
downloaded of our cyber Punk Robot from
here you just basically Point these dots
you know chin wrists Etc and and once
you have everything lined up uh hit the
next button and in just a few minutes we
have our character completely rigged up
and we have total camera control over
everything it's really kind of a lot of
fun now you will notice that our
character is a little bit on the low res
side right now uh that's just a mixo
limitation to get those textures back
and into full res you would bring it
into something like blender um we're not
going to get into that cuz like I said
that is a rabbit hole that is 70 hours
deep so if you're just dipping your toes
into 3D like I am I think that Meshi
provides a really cool solution in that
you can generate up assets and then
bring them into a 3D software package
and start playing around with it if
you're on sort of a higher 3D level
Meshi does have a number of tutorials um
for you on incorporating it into blender
or Unity which they also do have a
plugin for yeah all of this is way above
my head though you can also apparently
generate your Textures in meshy up to 4K
so yeah that's that's pretty cool the
link to meshy is down below once again
they do do have a free tier but they
also have a pro and a Max tier as well
if you are interested in subscribing to
either of those if you use the coupon
code Theo t h EO uh you get 20% off my
thanks to meshy for sponsoring this
video Moving on but still kind of
staying within the realm of 3D I do love
a good magic trick and Nick Beller uh
posted this up which is just really
super cool check this out yeah yeah yeah
yeah that's really cool it's a 3D object
in front of of the TV that has that
background yeah it's really awesome so
to accomplish this Nick actually
sculpted the character in the
PlayStation software dreams and did so
in VR as well so uh yeah it's pretty
awesome it is a real shame that Sony
kind of gave up on dreams uh I do hope
that somebody ends up picking up the
technology especially as we're making
this push into to VR with things like
you know the Apple Vision Pro from there
Nick took the model out into AR software
I believe it was Luma Labs that he ended
up using um yeah and there you go and
finally using a still image export of
the environment on his TV kind of like
it's a duct tape version of Disney's the
volume and placing the AR character in
front of it we we end up with this which
is I don't it's so cool I just love
experiments like this speaking of Dreams
which was developed by a company called
media molecule which I do not believe
exists anymore uh one of the programmers
and co-founders of media molecule uh
Alex Evans is now working at Mid Journey
turns out you can learn a lot just by
stumbling around LinkedIn now I don't
necessarily know what work Alex is doing
at Mid Journey he is listed as a
principal research engineer and is doing
so remotely actually in the UK as well
but I don't think it's too much of a
stretch to think that he might be
working on some of the 3D aspects of mid
journey I do think that the general
consensus when it comes to all of this
is that you will be able to you know
prompt an image and then have camera
rotational tools and be able to sort of
move around in 3D space for your 2D
images but I mean honestly who knows
given the fact that Alex is working with
mid journey I mean who knows what this
is going to look like but it is
interesting to think that there is going
to be a little bit of that dreams DNA in
mid Journey also did you know that mid
Journey has a head of Hardware I mean I
had to go digging into that and as it
turns out yeah in an office hours that I
missed but luckily Nick St Pierre was at
um Mid journey is building Hardware in
office hours yesterday they mentioned a
newly formed Hardware team it's
currently focused on collecting data for
3D it's going to be an orb it's
described is a device that enables
anyone to organize and manage tens of
thousands of virtual 3D spaces I mean I
don't I still don't know what that means
but I don't know I kind of want one
rounding out we have the first AI City
uh you might remember a while back the
simulation created a fully AI generated
South Park episodes well they're back
this time with s Francisco props on that
name so s Francisco is populated by a
bunch of AI agents all of whom have been
prompted to have wants needs and desires
and interact and learn from one another
the agents have instructions to fall in
love and in fact the agents do have
lifespans so they actually die as well
additionally they relax by playing old
school Nintendo games up up down down
left right left right kids now I'll say
the overall graphical interface of s
Francisco is very much in line with that
South Park animation that they did as
you can see we're kind of scrolling
around through the city here zering in
on one of the characters and uh you know
we see her talking to and Alexa it very
much looks like the South Park style of
Animation so yes while the overall
visual presentation does look a lot like
something we've already seen uh I'm
actually a lot more interested in what's
happening under the hood because
apparently San Francisco is just running
all the time so it's kind of like a
hyperactive version of The Sims That
Never Ends the ultimate goal that the
simulation are trying to achieve with
Sim Francisco is Agi and I I mean I
don't know who knows maybe it'll work it
actually really reminds me of uh the old
Simpsons episode with Lisa's tooth if
there's one thing that we've learned if
the Simpsons did it it ends up coming
true on that note I thank you for
watching my name is Tim
Просмотреть больше связанных видео
![](/_next/static/media/default-video-cover.615af72e.png)
How to Make Faceless YouTube Videos on ChatGPT Store (Prompt to Video)
![](https://i.ytimg.com/vi/21n45UsLPNQ/hq720.jpg)
How to Use Sora Prediction (& Early Access Tips!)
![](https://i.ytimg.com/vi/vSW0lPSCTMs/hq720.jpg)
OpenAI's New Model Releases LEAKED | Sam Altman talks about AGI, UBI, GPT-5 and what Agents will be
![](https://i.ytimg.com/vi/liQfsdSzKzM/hq720.jpg)
New AI Tools Anyone Can Use Today
![](/_next/static/media/default-video-cover.615af72e.png)
ChatGPT Explained Completely.
![](https://i.ytimg.com/vi/xsK4uATpxLY/hq720.jpg)
Всё о новой нейросети GPT-4o за 7 минут!
![](/_next/static/media/default-video-cover.615af72e.png)
Is Adobe Firefly better than Midjourney and Stable Diffusion?
5.0 / 5 (0 votes)