Massive AI News from Stability, Adobe, Ideogram, and More!

Summary

This week brought major AI developments, including Morph Studio's new AI filmmaking platform for easily generating and editing video; EmoTalker's impressive lip sync tool for animating still images; Adobe's Project Music AI for music generation and editing; an update to the free AI image generator Idiogram; insights into how YouTube's Sora video generator works via reverse engineering; and the first fully AI-generated remake of Terminator 2, created by 50 artists, which will premiere theatrically in LA before being live streamed.

Takeaways

• 😲 Morph Studio partners with Stability AI to create a new AI filmmaking platform
• 🎥 EmoTalker adds impressive lip sync to still images along with heavy eyeliner!
• 🎵 Adobe releases Project Music AI for music generation and extension
• 🔍 Researchers start reverse engineering Sora for similar video outputs
• 🎞️ The first fully AI-generated remake of Terminator 2 gets a theatrical release
• 🖼️ Idiogram 1.0 upgrades aesthetics and adds a handy prompt helper
• 👄 Pik also has a new lip sync feature for images and video
• 🌌 Sora looks amazing but likely won't be accessible for a while
• 💸 EmoTalker leverages Stable Diffusion, trained on lots of data
• ✏️ Morph's platform allows sharing templates and workflows

Q & A

What new video generation platform was announced?

Morph Studio partnered with Stability AI to create a new AI filmmaking platform called Morph Cloud.

What does the new platform Morph Cloud aim to do?

Morph Cloud aims to streamline the filmmaking process by allowing users to regenerate shots, share workflow templates, and have more control over the editing process using AI.

What is the name of the new lip sync tool featured?

The new lip sync tool is called Emo Talker, created by Alibaba.

What makes the Emo Talker lip sync tool special?

Emo Talker can add lifelike lip sync to any still image and also expresses emotion through facial expressions and eye movement.

What new AI music project did Adobe release?

Adobe released an AI music project called Project Music, developed with UC Berkeley and Carnegie Mellon.

What does Project Music allow users to do?

Project Music allows users to extend and vary musical compositions using AI.

What is the name of the free AI image generator covered?

The free AI image generator is called Idiogram.

How does Idiogram compare to other generators?

Idiogram specializes in coherent text generation within images and released a new 1.0 model with upgraded aesthetics.

What is the historical event related to AI mentioned?

The first fully AI-generated remake of a full length film (Terminator 2) will be screened in theaters.

Who created the AI remake of Terminator 2?

The remake was a collaborative project among 50 different artists using AI to generate the film.

Outlines

00:00

😲 New AI Film Production Platform

Paragraph 1 introduces a new AI film production platform called Morph Studio that partners with Stability AI. It has a node-based interface that connects different AI-generated video clips, allows style transfer, and gives control over the blend. The platform aims to create a community for sharing templates.

05:00

💋 Emo Talker Adds Lip Sync and Emotion

Paragraph 2 covers Emo Talker, a new lip sync tool from Alibaba that adds lifelike facial expressions and lip sync to still images. It was trained on extensive video and image data and leverages Stable Diffusion. The results are impressive but it currently only works with still images.

10:01

🎵 Adobe Project Music AI for Music Generation

Paragraph 3 discusses Adobe's new Project Music, an AI system for music generation and manipulation developed with UC Berkeley and Carnegie Mellon. It has capabilities like extending and changing music but there is no timeline yet for if/when it would be incorporated into Adobe products.

Keywords

💡Creative AI Tools

This refers to artificial intelligence systems and platforms that are designed to enhance and augment human creativity, especially in areas like art, music, writing etc. The video discusses updates and new releases from companies like Anthropic, Stability AI, Pika, Runway ML etc that are building these creative AIs.

💡Video generation

The ability to algorithmically generate or synthesize video content is a growing capability of AI systems. The video talks about new platforms like Morph Studio that allow users to control and tweak AI-generated video.

💡Lip sync

This refers to synchronizing facial animations, especially lip movements, with audio tracks like speech or music. The video demonstrates EmoTalker, an AI tool that can add realistic lip sync to still images.

💡Adobe

A leading technology company known for creative software tools like Photoshop. The video discusses Adobe's new AI music project called Project Music that allows AI-assisted music generation.

💡Idiogram

A free AI image generation platform focused on text-to-image capabilities. The video covers Idiogram's new 1.0 release which enhances both visual quality and text interpretation.

💡Sora

An AI video/animation model developed by Anthropic that produces highly realistic motion and speech animation from text prompts. The video speculates on how soon Sora-like capabilities may become available.

💡Terminator 2

A popular sci-fi action film. The video discusses an ambitious fan-made remake of Terminator 2 that is entirely AI-generated, marking a historic achievement.

💡Theatrical screening

The fan-made Terminator 2 remake will be the first full AI-generated film to be screened in an actual movie theater, representing a milestone for creative AI.

💡Parity

Making fun of something by imitation. The Terminator 2 remake is described as a parody, implying it humorously imitates the original film in various ways.

💡Rogues Gallery

A humorous term for a group of colorful, maverick individuals. It's used to describe the team of talented artists who worked together on the AI-generated Terminator remake.

Highlights

Morph Studio and Stability AI partnered to create an AI filmmaking platform with a node-based workflow

EmoTalker adds lifelike lip sync to still images along with emotive facial expressions

Adobe released Project Music, an AI music generator developed with UC Berkeley and Carnegie Mellon

Idiogram, the free AI image generator, released version 1.0 with improved aesthetics and text rendering

YouTuber MKBHD got early access to test out Google's video generator Sora

Researchers reverse engineered Sora, finding it uses spacetime latent patches for continuity

50 artists created the first AI-generated remake of Terminator 2 to be screened in theaters

Morph Studio's filmmaking platform has a simple node-based workflow for chaining AI models

EmoTalker was trained on 250 hours of video and 150 million images to achieve accurate lip sync

Idiogram received $80 million in funding to keep improving the free AI image generator

Researchers believe Sora-like video generation will be possible within months despite no public release

The AI-generated Terminator remake required extensive post work to meet theatrical standards

Morph aims to foster a community for sharing workflow templates on their filmmaking platform

Adobe's Project Music can extend and vary AI-generated music samples

Idiogram's new magic prompt feature suggests text to accompany images

Transcripts

so it's been a pretty crazy week for

Creative AI Tools in my last video I

went over LTX Studio as well as the

updates to Pika and Runway So today

we're hitting all the other stuff well I

mean at least as much as I can pack in

today we've got news from stability. a

on a new video generation platform a

really impressive new lip sync tool a

big release from Adobe in the music

space details on how Sora actually works

and the first fully AI generated film to

be screened in movie theater all right

grab a cup of coffee and buckle up

kicking off morph Studio have partnered

with stability AI to create a AI film

making platform that has a really kind

of cool workflow taking a look at this

shot from a video that they've released

you can see that it sort of has a

vaguely comfy UI node-based structure to

it but I do assure you this is much more

simple than comfy UI if you've never use

node-based workflows it's it takes a

minute to get your head wrapped around

but once you do it actually makes a lot

of sense since you can see everything

visually laid out here we have three

different videos um with the style

transfer on the third one and then as

you connect them together you can export

them out having control over the amount

of influence each one gives not calling

anybody out here but I did catch a typo

in that first prompt a grill is looking

at the camera uh again I'm not one to

judge you guys catch me misspelling

stuff all the time it does look like

this video generator will be able to

spell uh for example in that first video

morph Cloud we laid see the prompt come

up that says a cloud that spells morph

Billows out morph's co-founder XII I I

hope I pronounced that correctly uh said

filming editing and post- production

used to be separate steps in traditional

film making but AI blurs the boundaries

of these stages and turns them into one

continuous process if you aren't happy

with the shot you can regenerate it on

our canvas AI has introduced a new

workflow to Film Production the platform

aims to create a Vibrant Community by

allowing users to to share their

workflow templates with one another via

the gallery this one does fall under

weight list alert I just signed up

myself so once I get access I will

definitely be bringing you a full look

and I know any new video model that

comes out in the back of everyone's head

is like Sora Sora sore I I know we'll

talk about that in just a second next up

we have Emo talker which will not only

add lip syncing to any still image but

it also add heavy eyeliner to all of

your

characters

seriously no matter what kind of music

you're into go listen to the Black

Parade it is a 1010 album that

transcends any genre good is good anyhow

emo talker which is actually emote

portrait alive which is not an anacronym

for emo is brought To Us by Alibaba

let's take a quick look at it in action

here crying is the most beautiful thing

you can do I encourage people to cry I

cry all the time and I think it's the

most healthy expression of how you're

feeling there are a number of other

examples that you can check check out at

the link down below most of them have

music on them though and like in this

case this is Eminem's rap God uh which

is a very quick way for you know uh

Marshall's lawyers to show up on the

channel doorstep with a copyright strike

so I can't play it here overall the

thing that I'm actually super impressed

with is kind of the emotive aspects of

emo talker's performance there is still

some issues I feel with like the lip

flap but this is also an Eminem track in

which he is rapping extremely fast uh

but there is some issues with kind of

like the lip movement tracking but where

I think it really flies is with animated

or you know kind of CG characters as we

see in this example from the sleeper

game Detroit Being Human game is really

great when I was a kid I feel like you

heard the thing you heard the term don't

cry you don't need to cry digging into

the paper emo talker was apparently

trained on 250 hours of video and more

than 150 million images it can also

speak essentially in multiple languages

it does apparently also use stable

diffusion as its foundational framework

although that said the results are very

impressive but there are limitations for

one you can only lips sync to still

images so you can't like resync to video

the paper also notes that they did not

use any explicit control signals to

control character movement which as they

note uh may result in the inadvertent

generation of other body parts such as

hands leading to artifacts in the video

so you know basically weird AI video my

favorite emo talker has not been

released yet but Pik did release their

lip sync feature I did cover that in the

last video link is down below moving on

Adobe have released a new AI music

project called Project Music gen AI

Control I mean Adobe come on you got to

step it up with the names here Photoshop

Symphony InDesign Rhymes Lightroom

lullabies I mean I'm not giving you any

more for free call me Project Music was

developed in collaboration with

researchers at the University of

California and Carnegie melon they

released a promo video explaining some

of the things that you can do with

Project Music uh we'll take a look at

the section on extending

music all right here's the the

lengthened

one project music is just a research

project we may see it in the future but

not yet sliding back over to imagery

idiogram the free AI image generator

that spells better than I do has

released a 1.0 update yeah this one is

really cool idiogram always kind of sits

in my back pocket when I'm trying to

generate up something with text uh you

know mid Journey claims that it's doing

text but usually it's still kind of a

garbled mess idiogram has always really

had an edge on that front the new 1.0

model has really upped idiograms

aesthetic game and it actually now has a

magic prompt button that you can turn on

that kind of fills out your text if you

want to use it the idiogram Community

Feed is one of my favorites amongst all

of the image generation Community feeds

uh this one is actually really cool this

is from Devil's tuna or this awesome

penguin with the text cool people play

base and use After Effects I don't know

who made this but whoever did you are a

cool person the best part about idiogram

is that it is free it allows for 25

Generations per day obviously if you

want more you can then move into one of

their subscription tiers idiogram also

just secured $80 million in additional

funding so that's great news hopefully

it keeps that free tier free moving over

to some sore news YouTube's own mares

brownley got to play with it uh these

are some of the generations that he got

out of it I guess when you have 18.5

million subscribers on YouTube you get

to play with Sora and yes that is my

subtle way of asking for like 18.4

million of YouTube kindly hit the

Subscribe button anyhow as great as Sora

looks I still do not think that we will

be getting it anytime soon although in

my my last video I did mention that

someone asked Chrystal ball valenzula

Runway ml's CEO if there would be sore

like outputs coming out of Runway

anytime soon and he did say better on

the heels of that a paper was recently

released with a group of researchers who

were basically reverse engineering what

they saw in Sora I'm still going through

the paper it is obviously very dense and

it makes my head hurt quite a bit the

paper does indicate that Sora utilizes

SpaceTime latent patches which basically

Break Down the video into smaller

controllable pieces that it can

understand in both space and time which

allows for that sort of smooth

continuity again the paper is super

dense and while I have read a number of

white papers at this point you know I'm

still pretty much I'm like a caveman

that's sitting in at a meeting at JPL

you know I I can I can nod but you know

at the same time do I really understand

what's happening here but my overall

point is that very smart people have

already started to pull it apart and

they are in the process of developing

their own Sora like models even if open

AI does not release Sora in say the next

6 months I do think that we're going to

see something that looks like Sora

appearing within that time frame or

maybe a little bit later basically

anytime between tomorrow 6 months from

now or some point in the future I've

stopped predicting things because I'm

always wrong I'm always wrong rounding

out a historical event to my knowledge

at least the first fulllength AI

generated film to be screened out of

theater so a little while back I was on

the nerdy novelist podcast where I said

this I think a full feature movie is

kind of out of a fion currently with

with the way the technology is right now

although there are some lunatics right

now that are creating um their own

version of Terminator 2 wh generated by

Ai and just to be clear when I say

lunatics I do mean that as a term of

endearment but yeah they did it it is 50

artists doing essentially a cover

version of Terminator 2 wholly generated

by AI uh you know obviously as we see

here you know a lot of extra work went

into that AI but it is all still AI

generated the entire project is

basically this Rogues gallery of really

awesome and talented artists uh many of

whom have been featured on this channel

as well uh so yeah kudos to every single

one of you for pulling this Insanity off

the film will have a theatrical Premiere

in Los Angeles at the newart theater on

March 6th uh but don't worry if you

don't live in Los Angeles or near Los

Angeles you can still see the movie

online there'll be a live stream for it

on March 9th in which the cast and crew

will be in attendance so you can watch

it alongside them I did also want to

point out that because this is being

theatrically screened there was like a

ton of work put into it even after

everything was done because you know you

still have to conform the picture to

theatrical standard and like the sound

mix has to be ready for you know a

theater system overall from the stuff

that I've seen from this remake I mean

it's it's a parody it's hilarious

definitely please do check it out link

is down below well that's it for this

week I mean that's it like there was

like 80 things that happened this week

and I did not even get to everything but

I don't know we'll see what's in store

for next week I thank you for watching

my name is

Tim