Massive AI News from Stability, Adobe, Ideogram, and More!
Summary
TLDRThis week brought major AI developments, including Morph Studio's new AI filmmaking platform for easily generating and editing video; EmoTalker's impressive lip sync tool for animating still images; Adobe's Project Music AI for music generation and editing; an update to the free AI image generator Idiogram; insights into how YouTube's Sora video generator works via reverse engineering; and the first fully AI-generated remake of Terminator 2, created by 50 artists, which will premiere theatrically in LA before being live streamed.
Takeaways
- 😲 Morph Studio partners with Stability AI to create a new AI filmmaking platform
- 🎥 EmoTalker adds impressive lip sync to still images along with heavy eyeliner!
- 🎵 Adobe releases Project Music AI for music generation and extension
- 🔍 Researchers start reverse engineering Sora for similar video outputs
- 🎞️ The first fully AI-generated remake of Terminator 2 gets a theatrical release
- 🖼️ Idiogram 1.0 upgrades aesthetics and adds a handy prompt helper
- 👄 Pik also has a new lip sync feature for images and video
- 🌌 Sora looks amazing but likely won't be accessible for a while
- 💸 EmoTalker leverages Stable Diffusion, trained on lots of data
- ✏️ Morph's platform allows sharing templates and workflows
Q & A
What new video generation platform was announced?
-Morph Studio partnered with Stability AI to create a new AI filmmaking platform called Morph Cloud.
What does the new platform Morph Cloud aim to do?
-Morph Cloud aims to streamline the filmmaking process by allowing users to regenerate shots, share workflow templates, and have more control over the editing process using AI.
What is the name of the new lip sync tool featured?
-The new lip sync tool is called Emo Talker, created by Alibaba.
What makes the Emo Talker lip sync tool special?
-Emo Talker can add lifelike lip sync to any still image and also expresses emotion through facial expressions and eye movement.
What new AI music project did Adobe release?
-Adobe released an AI music project called Project Music, developed with UC Berkeley and Carnegie Mellon.
What does Project Music allow users to do?
-Project Music allows users to extend and vary musical compositions using AI.
What is the name of the free AI image generator covered?
-The free AI image generator is called Idiogram.
How does Idiogram compare to other generators?
-Idiogram specializes in coherent text generation within images and released a new 1.0 model with upgraded aesthetics.
What is the historical event related to AI mentioned?
-The first fully AI-generated remake of a full length film (Terminator 2) will be screened in theaters.
Who created the AI remake of Terminator 2?
-The remake was a collaborative project among 50 different artists using AI to generate the film.
Outlines
😲 New AI Film Production Platform
Paragraph 1 introduces a new AI film production platform called Morph Studio that partners with Stability AI. It has a node-based interface that connects different AI-generated video clips, allows style transfer, and gives control over the blend. The platform aims to create a community for sharing templates.
💋 Emo Talker Adds Lip Sync and Emotion
Paragraph 2 covers Emo Talker, a new lip sync tool from Alibaba that adds lifelike facial expressions and lip sync to still images. It was trained on extensive video and image data and leverages Stable Diffusion. The results are impressive but it currently only works with still images.
🎵 Adobe Project Music AI for Music Generation
Paragraph 3 discusses Adobe's new Project Music, an AI system for music generation and manipulation developed with UC Berkeley and Carnegie Mellon. It has capabilities like extending and changing music but there is no timeline yet for if/when it would be incorporated into Adobe products.
Mindmap
Keywords
💡Creative AI Tools
💡Video generation
💡Lip sync
💡Adobe
💡Idiogram
💡Sora
💡Terminator 2
💡Theatrical screening
💡Parity
💡Rogues Gallery
Highlights
Morph Studio and Stability AI partnered to create an AI filmmaking platform with a node-based workflow
EmoTalker adds lifelike lip sync to still images along with emotive facial expressions
Adobe released Project Music, an AI music generator developed with UC Berkeley and Carnegie Mellon
Idiogram, the free AI image generator, released version 1.0 with improved aesthetics and text rendering
YouTuber MKBHD got early access to test out Google's video generator Sora
Researchers reverse engineered Sora, finding it uses spacetime latent patches for continuity
50 artists created the first AI-generated remake of Terminator 2 to be screened in theaters
Morph Studio's filmmaking platform has a simple node-based workflow for chaining AI models
EmoTalker was trained on 250 hours of video and 150 million images to achieve accurate lip sync
Idiogram received $80 million in funding to keep improving the free AI image generator
Researchers believe Sora-like video generation will be possible within months despite no public release
The AI-generated Terminator remake required extensive post work to meet theatrical standards
Morph aims to foster a community for sharing workflow templates on their filmmaking platform
Adobe's Project Music can extend and vary AI-generated music samples
Idiogram's new magic prompt feature suggests text to accompany images
Transcripts
so it's been a pretty crazy week for
Creative AI Tools in my last video I
went over LTX Studio as well as the
updates to Pika and Runway So today
we're hitting all the other stuff well I
mean at least as much as I can pack in
today we've got news from stability. a
on a new video generation platform a
really impressive new lip sync tool a
big release from Adobe in the music
space details on how Sora actually works
and the first fully AI generated film to
be screened in movie theater all right
grab a cup of coffee and buckle up
kicking off morph Studio have partnered
with stability AI to create a AI film
making platform that has a really kind
of cool workflow taking a look at this
shot from a video that they've released
you can see that it sort of has a
vaguely comfy UI node-based structure to
it but I do assure you this is much more
simple than comfy UI if you've never use
node-based workflows it's it takes a
minute to get your head wrapped around
but once you do it actually makes a lot
of sense since you can see everything
visually laid out here we have three
different videos um with the style
transfer on the third one and then as
you connect them together you can export
them out having control over the amount
of influence each one gives not calling
anybody out here but I did catch a typo
in that first prompt a grill is looking
at the camera uh again I'm not one to
judge you guys catch me misspelling
stuff all the time it does look like
this video generator will be able to
spell uh for example in that first video
morph Cloud we laid see the prompt come
up that says a cloud that spells morph
Billows out morph's co-founder XII I I
hope I pronounced that correctly uh said
filming editing and post- production
used to be separate steps in traditional
film making but AI blurs the boundaries
of these stages and turns them into one
continuous process if you aren't happy
with the shot you can regenerate it on
our canvas AI has introduced a new
workflow to Film Production the platform
aims to create a Vibrant Community by
allowing users to to share their
workflow templates with one another via
the gallery this one does fall under
weight list alert I just signed up
myself so once I get access I will
definitely be bringing you a full look
and I know any new video model that
comes out in the back of everyone's head
is like Sora Sora sore I I know we'll
talk about that in just a second next up
we have Emo talker which will not only
add lip syncing to any still image but
it also add heavy eyeliner to all of
your
characters
seriously no matter what kind of music
you're into go listen to the Black
Parade it is a 1010 album that
transcends any genre good is good anyhow
emo talker which is actually emote
portrait alive which is not an anacronym
for emo is brought To Us by Alibaba
let's take a quick look at it in action
here crying is the most beautiful thing
you can do I encourage people to cry I
cry all the time and I think it's the
most healthy expression of how you're
feeling there are a number of other
examples that you can check check out at
the link down below most of them have
music on them though and like in this
case this is Eminem's rap God uh which
is a very quick way for you know uh
Marshall's lawyers to show up on the
channel doorstep with a copyright strike
so I can't play it here overall the
thing that I'm actually super impressed
with is kind of the emotive aspects of
emo talker's performance there is still
some issues I feel with like the lip
flap but this is also an Eminem track in
which he is rapping extremely fast uh
but there is some issues with kind of
like the lip movement tracking but where
I think it really flies is with animated
or you know kind of CG characters as we
see in this example from the sleeper
game Detroit Being Human game is really
great when I was a kid I feel like you
heard the thing you heard the term don't
cry you don't need to cry digging into
the paper emo talker was apparently
trained on 250 hours of video and more
than 150 million images it can also
speak essentially in multiple languages
it does apparently also use stable
diffusion as its foundational framework
although that said the results are very
impressive but there are limitations for
one you can only lips sync to still
images so you can't like resync to video
the paper also notes that they did not
use any explicit control signals to
control character movement which as they
note uh may result in the inadvertent
generation of other body parts such as
hands leading to artifacts in the video
so you know basically weird AI video my
favorite emo talker has not been
released yet but Pik did release their
lip sync feature I did cover that in the
last video link is down below moving on
Adobe have released a new AI music
project called Project Music gen AI
Control I mean Adobe come on you got to
step it up with the names here Photoshop
Symphony InDesign Rhymes Lightroom
lullabies I mean I'm not giving you any
more for free call me Project Music was
developed in collaboration with
researchers at the University of
California and Carnegie melon they
released a promo video explaining some
of the things that you can do with
Project Music uh we'll take a look at
the section on extending
music all right here's the the
lengthened
one project music is just a research
project we may see it in the future but
not yet sliding back over to imagery
idiogram the free AI image generator
that spells better than I do has
released a 1.0 update yeah this one is
really cool idiogram always kind of sits
in my back pocket when I'm trying to
generate up something with text uh you
know mid Journey claims that it's doing
text but usually it's still kind of a
garbled mess idiogram has always really
had an edge on that front the new 1.0
model has really upped idiograms
aesthetic game and it actually now has a
magic prompt button that you can turn on
that kind of fills out your text if you
want to use it the idiogram Community
Feed is one of my favorites amongst all
of the image generation Community feeds
uh this one is actually really cool this
is from Devil's tuna or this awesome
penguin with the text cool people play
base and use After Effects I don't know
who made this but whoever did you are a
cool person the best part about idiogram
is that it is free it allows for 25
Generations per day obviously if you
want more you can then move into one of
their subscription tiers idiogram also
just secured $80 million in additional
funding so that's great news hopefully
it keeps that free tier free moving over
to some sore news YouTube's own mares
brownley got to play with it uh these
are some of the generations that he got
out of it I guess when you have 18.5
million subscribers on YouTube you get
to play with Sora and yes that is my
subtle way of asking for like 18.4
million of YouTube kindly hit the
Subscribe button anyhow as great as Sora
looks I still do not think that we will
be getting it anytime soon although in
my my last video I did mention that
someone asked Chrystal ball valenzula
Runway ml's CEO if there would be sore
like outputs coming out of Runway
anytime soon and he did say better on
the heels of that a paper was recently
released with a group of researchers who
were basically reverse engineering what
they saw in Sora I'm still going through
the paper it is obviously very dense and
it makes my head hurt quite a bit the
paper does indicate that Sora utilizes
SpaceTime latent patches which basically
Break Down the video into smaller
controllable pieces that it can
understand in both space and time which
allows for that sort of smooth
continuity again the paper is super
dense and while I have read a number of
white papers at this point you know I'm
still pretty much I'm like a caveman
that's sitting in at a meeting at JPL
you know I I can I can nod but you know
at the same time do I really understand
what's happening here but my overall
point is that very smart people have
already started to pull it apart and
they are in the process of developing
their own Sora like models even if open
AI does not release Sora in say the next
6 months I do think that we're going to
see something that looks like Sora
appearing within that time frame or
maybe a little bit later basically
anytime between tomorrow 6 months from
now or some point in the future I've
stopped predicting things because I'm
always wrong I'm always wrong rounding
out a historical event to my knowledge
at least the first fulllength AI
generated film to be screened out of
theater so a little while back I was on
the nerdy novelist podcast where I said
this I think a full feature movie is
kind of out of a fion currently with
with the way the technology is right now
although there are some lunatics right
now that are creating um their own
version of Terminator 2 wh generated by
Ai and just to be clear when I say
lunatics I do mean that as a term of
endearment but yeah they did it it is 50
artists doing essentially a cover
version of Terminator 2 wholly generated
by AI uh you know obviously as we see
here you know a lot of extra work went
into that AI but it is all still AI
generated the entire project is
basically this Rogues gallery of really
awesome and talented artists uh many of
whom have been featured on this channel
as well uh so yeah kudos to every single
one of you for pulling this Insanity off
the film will have a theatrical Premiere
in Los Angeles at the newart theater on
March 6th uh but don't worry if you
don't live in Los Angeles or near Los
Angeles you can still see the movie
online there'll be a live stream for it
on March 9th in which the cast and crew
will be in attendance so you can watch
it alongside them I did also want to
point out that because this is being
theatrically screened there was like a
ton of work put into it even after
everything was done because you know you
still have to conform the picture to
theatrical standard and like the sound
mix has to be ready for you know a
theater system overall from the stuff
that I've seen from this remake I mean
it's it's a parody it's hilarious
definitely please do check it out link
is down below well that's it for this
week I mean that's it like there was
like 80 things that happened this week
and I did not even get to everything but
I don't know we'll see what's in store
for next week I thank you for watching
my name is
Tim
Посмотреть больше похожих видео
This Was an Insane Week for AI Use Cases
Easy Guide To Ultra-Realistic AI Images (With Flux)
A Stunning New AI Image Generator & New FREE AI Video!
New AI Video That Does Everything!
THE FUTURE OF AI VIDEO EDITING IS HERE! What's new in Wondershare Filmora 13?
7 OUTILS IA IMPRESSIONNANTS à CONNAÎTRE en 2024 !
5.0 / 5 (0 votes)