Massive AI News from Stability, Adobe, Ideogram, and More!

Summary

今週はクリエイティブAIツールに関するニュースが満載です。LTX StudioやPika、Runwayのアップデートに続き、Stability AIからの新しいビデオ生成プラットフォーム、印象的なリップシンクツール、Adobeによる音楽分野の大型リリース、Soraの動作詳細、そして映画館で上映される初の完全AI生成映画について紹介します。Morph StudioとStability AIが提携してAI映画製作プラットフォームを作成し、Emo Talkerがリップシンクを改革。AdobeはProject Musicを発表し、Idiogramは1.0アップデートをリリースしました。また、Soraの逆エンジニアリングに関する研究も行われています。最後に、完全にAIで生成された映画が劇場で上映される歴史的瞬間についても触れています。

Takeaways

• 🚀 Morph Studioは、Stability AIと提携して、AIによる映画制作プラットフォームを開発した。このプラットフォームは、直感的なノードベースのワークフローを特徴とする。
• 🎥 新しいビデオ生成プラットフォームでは、スタイル転送を含む複数のビデオを組み合わせてエクスポートすることができる。
• 👄 Emo Talkerは、任意の静止画像にリップシンクを追加し、感情的な表現を強化するツールである。これはAlibabaによって開発された。
• 🎵 Adobeは、音楽空間で「Project Music」を発表し、音楽制作にAIを組み込む新しい試みを行っている。
• 🎨 Soraの動作原理に関する詳細が明らかにされ、ビデオのスムーズな連続性を実現するために「SpaceTime latent patches」を使用していることが示された。
• 🎬 初の完全AI生成映画が劇場で上映される歴史的イベントが発生し、AI技術による映画制作の可能性を示唆している。
• 📝 Idiogramは、テキストを含む画像の生成に特化したAIイメージジェネレーターで、1.0アップデートをリリースし、美的品質を向上させた。
• 🤝 プラットフォームは、ユーザーがワークフローテンプレートを共有し、活発なコミュニティを形成することを目指している。
• 💼 Emo Talkerは、250時間のビデオと1億5000万以上の画像で訓練され、複数言語に対応している。
• 🌐 AI技術の進化は、撮影、編集、ポストプロダクションの従来の境界をぼやけさせ、これらを一つの連続したプロセスへと変化させている。

Q & A

What new video generation platform was announced?

Morph Studio partnered with Stability AI to create a new AI filmmaking platform called Morph Cloud.

What is unique about Morph Cloud's workflow?

Morph Cloud uses a node-based interface that allows users to visually connect different AI generated video clips and style transfers together into a final video.

What does the new Emo Talker tool allow?

Emo Talker allows adding lip syncing and emotive facial expressions to any still image.

What music AI project did Adobe announce?

Adobe announced Project Music AI Control, which can generate and extend music using AI.

What updates were made to the Idiogram image generator?

Idiogram 1.0 has improved image quality and aesthetics. It also added a magic prompt feature to help generate images.

What AI method does Sora apparently use?

Researchers found Sora likely uses SpaceTime Latent Patches to understand video in both space and time dimensions.

What was the first AI generated film screened in theaters?

A remake of Terminator 2 generated by AI and created by 50 artists was screened in theaters.

When and where can the AI Terminator 2 film be watched online?

The film will be live-streamed online on March 9th so people worldwide can watch alongside the cast and crew.

What post-production work was done for the AI Terminator 2 film?

Additional work was required to conform the AI footage to theatrical standards for picture and sound.

Who created the AI Terminator 2 film?

The film was a collaborative project created by a group of talented artists who have worked with AI generative art.

Outlines

00:00

🚀 AI映画製作プラットフォームとリップシンクツールの新着情報

このパラグラフでは、最近のAIツールの進化について語っています。特に、stability.aiの新しい動画生成プラットフォーム、進化したリップシンクツール、Adobeの音楽領域への新規参入、Soraの機能詳細、そして完全AI生成の映画が映画館で上映された最初の事例について触れています。また、morph Studioとstability AIの提携によるAI映画製作プラットフォーム、Emo talkerのリップシンク機能、及びその他のAIツールのアップデートについて詳しく説明しています。

05:00

🎶 AIによる音楽拡張とイメージ生成の最新動向

第二のパラグラフでは、Adobeが開発したAIによる音楽プロジェクト「Project Music」や、テキスト生成能力に優れるAIイメージ生成ツール「idiogram」のアップデートに焦点を当てています。また、YouTubeの人気チャンネル所有者であるMarques BrownleeがSoraを試用した経験や、最初のフルレングスAI生成映画の劇場上映についても言及しています。

10:01

🌟 今週のAIツール進化の要約と次週の予告

最後のパラグラフでは、今週のAI技術の進展に関する概要を提供しています。この週は多くの新発表やアップデートがあったことを強調し、来週に期待できる内容についても触れています。また、視聴者に感謝を示し、次回の更新に向けて期待を煽っています。

Keywords

💡クリエイティブAIツール

クリエイティブAIツールは、芸術やデザイン、映画製作などの創造的なプロセスを支援する人工知能技術を指します。このビデオでは、LTX Studio、Pika、Runwayの更新、そしてその他の多くのツールが紹介されています。これらのツールは、AIがクリエイティブ産業に革命をもたらす方法の例です。

💡ノードベースのワークフロー

ノードベースのワークフローは、プロセスやタスクをノードとして視覚的に表現し、これらを線で接続してデータの流れを示す方法です。Morph StudioとStability AIの提携により開発された映画製作プラットフォームは、このタイプのインターフェースを使用しています。このアプローチは、複雑なプロセスを直感的に理解しやすくする利点があります。

💡スタイル転送

スタイル転送は、ある画像のスタイル（色、質感など）を別の画像に適用するAI技術です。このビデオでは、Morph Studioのデモンストレーションでスタイル転送が使用されており、異なるビデオ間でスタイルの一貫性を保つ方法として紹介されています。

💡AIによるフィルム製作

AIによるフィルム製作は、脚本の生成から編集、特殊効果の適用に至るまで、映画製作の各段階にAI技術を活用することを指します。このビデオでは、AIが伝統的な映画製作の境界をぼかし、創造的プロセスを一元化する新しいワークフローを紹介しています。

💡コミュニティ共有

コミュニティ共有は、ユーザーが自分の作品やワークフローテンプレートを他のユーザーと共有することを可能にする機能です。このビデオで紹介されたプラットフォームは、ユーザーが作成したテンプレートをギャラリーを通じて共有できるように設計されており、クリエイティブなコラボレーションを促進します。

💡リップシンクツール

リップシンクツールは、静止画像に口パクを合わせるAI技術です。ビデオでは、Emo Talkerというツールが紹介されており、静止画像にリアルタイムで口パクを追加することができます。この技術は、アニメーションやビデオゲームのキャラクターをより生き生きとさせるのに役立ちます。

💡AI音楽プロジェクト

AI音楽プロジェクトは、音楽作成プロセスにAIを組み込む試みを指します。Adobeが発表したProject Musicは、このようなプロジェクトの一例であり、ユーザーがAIを利用して音楽を作成、編集、拡張することができます。

💡AI画像生成器

AI画像生成器は、テキストの説明から画像を生成するAI技術です。ビデオでは、Idiogramという画像生成器が1.0アップデートを発表したことが紹介されており、テキストベースのプロンプトを使用して高品質な画像を生成することができます。

💡Sora

Soraは、ビデオ内容生成に特化したAIモデルです。ビデオでは、Soraが新しいビデオモデルとして期待されているが、一般にはまだリリースされていないことが触れられています。このモデルは、ビデオの生成に新しいアプローチを提供し、クリエイティブな表現の可能性を広げています。

💡AIによる映画

AIによる映画は、脚本から撮影、編集までのプロセスにAIを活用して作成された映画を指します。ビデオでは、AIによって生成された「ターミネーター2」のカバーバージョンが、映画館で上映される最初のフルレングスのAI生成映画として紹介されています。この出来事は、AI技術が映画製作に与える影響の大きさを示しています。

Highlights

New video generation platform Morph Studio launched in partnership with Stability AI

EmoTalker adds high-quality lip sync to still images along with emotive facial expressions

Adobe releases Project MusicGenAI for AI-powered music generation and extension

Idiogram 1.0 update improves image quality and adds handy features like magic prompts

Insights into how YouTube's new Sora video generator may work based on reverse engineering

First ever fully AI-generated remake of Terminator 2 to be screened in theaters

Morph Studio offers easy nodal workflow for generating and combining AI videos

EmoTalker trained on large dataset of video and images to enable multilingual speech

Project MusicGenAI focuses on AI-powered music generation, extension and control

Idiogram's new 1.0 model improves image quality and aesthetics

Researchers attempting to reverse engineer Sora and develop similar video generators

Full AI remake of Terminator 2 involved 50 artists and extensive post work

Morph Studio allows combining multiple AI-generated videos into one workflow

EmoTalker utilizes Stable Diffusion but has limitations in tracking body movements

AI Terminator 2 remake getting theatrical premiere and livestream

Transcripts

so it's been a pretty crazy week for

Creative AI Tools in my last video I

went over LTX Studio as well as the

updates to Pika and Runway So today

we're hitting all the other stuff well I

mean at least as much as I can pack in

today we've got news from stability. a

on a new video generation platform a

really impressive new lip sync tool a

big release from Adobe in the music

space details on how Sora actually works

and the first fully AI generated film to

be screened in movie theater all right

grab a cup of coffee and buckle up

kicking off morph Studio have partnered

with stability AI to create a AI film

making platform that has a really kind

of cool workflow taking a look at this

shot from a video that they've released

you can see that it sort of has a

vaguely comfy UI node-based structure to

it but I do assure you this is much more

simple than comfy UI if you've never use

node-based workflows it's it takes a

minute to get your head wrapped around

but once you do it actually makes a lot

of sense since you can see everything

visually laid out here we have three

different videos um with the style

transfer on the third one and then as

you connect them together you can export

them out having control over the amount

of influence each one gives not calling

anybody out here but I did catch a typo

in that first prompt a grill is looking

at the camera uh again I'm not one to

judge you guys catch me misspelling

stuff all the time it does look like

this video generator will be able to

spell uh for example in that first video

morph Cloud we laid see the prompt come

up that says a cloud that spells morph

Billows out morph's co-founder XII I I

hope I pronounced that correctly uh said

filming editing and post- production

used to be separate steps in traditional

film making but AI blurs the boundaries

of these stages and turns them into one

continuous process if you aren't happy

with the shot you can regenerate it on

our canvas AI has introduced a new

workflow to Film Production the platform

aims to create a Vibrant Community by

allowing users to to share their

workflow templates with one another via

the gallery this one does fall under

weight list alert I just signed up

myself so once I get access I will

definitely be bringing you a full look

and I know any new video model that

comes out in the back of everyone's head

is like Sora Sora sore I I know we'll

talk about that in just a second next up

we have Emo talker which will not only

add lip syncing to any still image but

it also add heavy eyeliner to all of

your

characters

seriously no matter what kind of music

you're into go listen to the Black

Parade it is a 1010 album that

transcends any genre good is good anyhow

emo talker which is actually emote

portrait alive which is not an anacronym

for emo is brought To Us by Alibaba

let's take a quick look at it in action

here crying is the most beautiful thing

you can do I encourage people to cry I

cry all the time and I think it's the

most healthy expression of how you're

feeling there are a number of other

examples that you can check check out at

the link down below most of them have

music on them though and like in this

case this is Eminem's rap God uh which

is a very quick way for you know uh

Marshall's lawyers to show up on the

channel doorstep with a copyright strike

so I can't play it here overall the

thing that I'm actually super impressed

with is kind of the emotive aspects of

emo talker's performance there is still

some issues I feel with like the lip

flap but this is also an Eminem track in

which he is rapping extremely fast uh

but there is some issues with kind of

like the lip movement tracking but where

I think it really flies is with animated

or you know kind of CG characters as we

see in this example from the sleeper

game Detroit Being Human game is really

great when I was a kid I feel like you

heard the thing you heard the term don't

cry you don't need to cry digging into

the paper emo talker was apparently

trained on 250 hours of video and more

than 150 million images it can also

speak essentially in multiple languages

it does apparently also use stable

diffusion as its foundational framework

although that said the results are very

impressive but there are limitations for

one you can only lips sync to still

images so you can't like resync to video

the paper also notes that they did not

use any explicit control signals to

control character movement which as they

note uh may result in the inadvertent

generation of other body parts such as

hands leading to artifacts in the video

so you know basically weird AI video my

favorite emo talker has not been

released yet but Pik did release their

lip sync feature I did cover that in the

last video link is down below moving on

Adobe have released a new AI music

project called Project Music gen AI

Control I mean Adobe come on you got to

step it up with the names here Photoshop

Symphony InDesign Rhymes Lightroom

lullabies I mean I'm not giving you any

more for free call me Project Music was

developed in collaboration with

researchers at the University of

California and Carnegie melon they

released a promo video explaining some

of the things that you can do with

Project Music uh we'll take a look at

the section on extending

music all right here's the the

lengthened

one project music is just a research

project we may see it in the future but

not yet sliding back over to imagery

idiogram the free AI image generator

that spells better than I do has

released a 1.0 update yeah this one is

really cool idiogram always kind of sits

in my back pocket when I'm trying to

generate up something with text uh you

know mid Journey claims that it's doing

text but usually it's still kind of a

garbled mess idiogram has always really

had an edge on that front the new 1.0

model has really upped idiograms

aesthetic game and it actually now has a

magic prompt button that you can turn on

that kind of fills out your text if you

want to use it the idiogram Community

Feed is one of my favorites amongst all

of the image generation Community feeds

uh this one is actually really cool this

is from Devil's tuna or this awesome

penguin with the text cool people play

base and use After Effects I don't know

who made this but whoever did you are a

cool person the best part about idiogram

is that it is free it allows for 25

Generations per day obviously if you

want more you can then move into one of

their subscription tiers idiogram also

just secured $80 million in additional

funding so that's great news hopefully

it keeps that free tier free moving over

to some sore news YouTube's own mares

brownley got to play with it uh these

are some of the generations that he got

out of it I guess when you have 18.5

million subscribers on YouTube you get

to play with Sora and yes that is my

subtle way of asking for like 18.4

million of YouTube kindly hit the

Subscribe button anyhow as great as Sora

looks I still do not think that we will

be getting it anytime soon although in

my my last video I did mention that

someone asked Chrystal ball valenzula

Runway ml's CEO if there would be sore

like outputs coming out of Runway

anytime soon and he did say better on

the heels of that a paper was recently

released with a group of researchers who

were basically reverse engineering what

they saw in Sora I'm still going through

the paper it is obviously very dense and

it makes my head hurt quite a bit the

paper does indicate that Sora utilizes

SpaceTime latent patches which basically

Break Down the video into smaller

controllable pieces that it can

understand in both space and time which

allows for that sort of smooth

continuity again the paper is super

dense and while I have read a number of

white papers at this point you know I'm

still pretty much I'm like a caveman

that's sitting in at a meeting at JPL

you know I I can I can nod but you know

at the same time do I really understand

what's happening here but my overall

point is that very smart people have

already started to pull it apart and

they are in the process of developing

their own Sora like models even if open

AI does not release Sora in say the next

6 months I do think that we're going to

see something that looks like Sora

appearing within that time frame or

maybe a little bit later basically

anytime between tomorrow 6 months from

now or some point in the future I've

stopped predicting things because I'm

always wrong I'm always wrong rounding

out a historical event to my knowledge

at least the first fulllength AI

generated film to be screened out of

theater so a little while back I was on

the nerdy novelist podcast where I said

this I think a full feature movie is

kind of out of a fion currently with

with the way the technology is right now

although there are some lunatics right

now that are creating um their own

version of Terminator 2 wh generated by

Ai and just to be clear when I say

lunatics I do mean that as a term of

endearment but yeah they did it it is 50

artists doing essentially a cover

version of Terminator 2 wholly generated

by AI uh you know obviously as we see

here you know a lot of extra work went

into that AI but it is all still AI

generated the entire project is

basically this Rogues gallery of really

awesome and talented artists uh many of

whom have been featured on this channel

as well uh so yeah kudos to every single

one of you for pulling this Insanity off

the film will have a theatrical Premiere

in Los Angeles at the newart theater on

March 6th uh but don't worry if you

don't live in Los Angeles or near Los

Angeles you can still see the movie

online there'll be a live stream for it

on March 9th in which the cast and crew

will be in attendance so you can watch

it alongside them I did also want to

point out that because this is being

theatrically screened there was like a

ton of work put into it even after

everything was done because you know you

still have to conform the picture to

theatrical standard and like the sound

mix has to be ready for you know a

theater system overall from the stuff

that I've seen from this remake I mean

it's it's a parody it's hilarious

definitely please do check it out link

is down below well that's it for this

week I mean that's it like there was

like 80 things that happened this week

and I did not even get to everything but

I don't know we'll see what's in store

for next week I thank you for watching

my name is

Tim