AI Building in Creative Mode
Summary
TLDRこのビデオでは、クリエイティブモードでAIと共にさらに多くのものを作り上げる Minecraft エピソードを紹介します。砂漠の環境で大きなピラミッドを作り、言語モデル Claude Opus によって自動生成された構造です。さまざまなモデルを比較しながら、ロマン風の建物や砂漠城、レッドストーン回路など、さまざまな建造物に挑戦します。GPT 4 は最も高価で最先端のモデルですが、ネザーポータルの構築に成功し、他のモデルと比べて驚くべきパフォーマンスを見せました。このビデオは、AIがMinecraftで何を作り上げることができるかという膨大な可能性を示しており、今後もさらなる探求が期待されます。
Takeaways
- 🏰 スクリプトでは、MinecraftのクリエイティブモードでAIを用いて様々な建造物を建設している様子が紹介されています。
- 🤖 AIの制御は、Claude Opusという言語モデルによって行われており、コードを生成して構造を作っています。
- 🕒 大規模な建造物を作るには時間がかかるため、ビデオではMinecraftの時間を使用して建造物の進行状況を表現しています。
- 🛠️ ブロック配置の順序は大きな建造物にとって非常に重要で、効率を上げるために逆順で行う方法が提案されています。
- 💎 建造物の頂上にダイアモンドブロックを設置するなど、AIは特定の要求に応じて建造物を装飾することができます。
- 📈 ビデオでは、Claude Opus、Gemini、llama llama 3という3つの異なるAIモデルを比較しています。
- 🔍 llama llama 3はFacebook(メタ)によって開発されたオープンソースモデルで、特定のハードウェアがあれば自分のコンピュータで実行できます。
- 🏠 各AIモデルは、与えられたブロックパレットから選んで建造物を作り、複雑な構造を理解し、実行する能力を示しています。
- ️ Redstone回路の構築に挑戦し、シンプルな回路から始めて徐々に複雑さを増やしていきます。
- 💥 最後に破壊的な方向にシフトし、爆発物を作成し、Minecraftのヌークを作り上げるまでの道のりを示しています。
- 🌐 最後の挑戦として、Netherポータルの作成を行い、AIが理論上はそれを可能としていることがわかります。
Q & A
ビデオではどのようなゲームをプレイしていますか?
-ビデオではMinecraftというゲームをプレイしています。
ビデオの主人公は誰を操作していますか?
-ビデオの主人公はAndyを操作しています。
ビデオではどのような言語モデルを使っていますか?
-ビデオではClaude Opusという言語モデルを使っています。
ビデオで建造されるものは何ですか?
-ビデオで建造されるものは砂漠地形にある大きなピラミッドです。
ビデオで紹介されている言語モデルには何が特徴的ですか?
-ビデオで紹介されている言語モデルは、コードを生成して構造を作り上げる能力があるという点が特徴的です。
ビデオ内で建造物を作るのにどれくらいの時間がかかりましたか?
-ビデオ内で建造物を作るのに数日のMinecraft時間がかかりました。
ビデオではどの建造物が最も技術的に印象的ですか?
-ビデオでは特に技術的に印象的な建造物は直接言及されていませんが、大きなピラミッドが完成度が高いと言えます。
ビデオ内で紹介されている言語モデルには何が欠点ですか?
-ビデオ内で紹介されている言語モデルには、時には文字を省略したり、間違ったブロックを書くなどの欠点があります。
ビデオ内で言語モデルがどのようにブロックを置くことを指示していますか?
-ビデオ内で言語モデルは一行ずつ交互に逆順にブロックを置く方法で建造物を作るように指示しています。
ビデオ内で紹介されている言語モデルはどれが最も優れていると言えますか?
-ビデオ内で紹介されている言語モデルの中では、GPTとClaude Opusが比較的優れていると言えます。
ビデオ内で言語モデルが建造物を作っている際にどのくらいの時間をかかる予定でしたか?
-ビデオ内で言語モデルが建造物を作っている際にかかる予定の時間について言及されていないため、具体的な時間を知ることはできません。
Outlines
🏗️ MinecraftでのAI建築プロジェクト
ビデオでは、クリエイティブモードでAIを使って様々な建造物を作り上げるMinecraftのエピソードが紹介されています。砂漠生地に大きなピラミッドを建設するプロジェクトが進んでおり、Claude Opusという言語モデルがコードを生成して構造を作動させています。建造物は数多くの失敗を経て完成し、全体の建造過程はPatreonでタイムラプスで見ることができます。また、効率的な建造方法として、行を交互に逆順に積む方法が取り入れられています。
🤖 AIモデルの紹介と建造物作りの挑戦
ビデオでは、Claude Opus、Gemini、llama llama 3という3つのAIモデルが紹介され、それぞれが異なる建造物を作り上げる試みを行っています。各モデルは独自の特性を持っており、建造物のスタイルや完成度にばらつきがあります。特にllama llama 3はオープンソースモデルで、高性能ながらも不安定な動作が見られます。建造物作りの過程で、モデル同士が協力し合う様子も描かれています。
💥 レッドストーン回路の挑戦と破壊的な方向へのシフト
AIモデルたちはレッドストーン回路の作り方を試みていますが、複雑な部分が多く、完璧な結果は得られません。GPTは一度目で完璧な結果を出しますが、他のモデルは多少の混乱を経験します。その後、ビデオは破壊的な方向にシフトし、爆発物作りに挑戦します。最後に、ネザーポータルの作成という難しい課題に取り組みますが、全てのモデルが成功するわけではありません。GPT 4は驚くべき結果を出し、ネザーに到達するまでを記録していますが、すぐに混乱して出てしまいます。
Mindmap
Keywords
💡Minecraft
💡AI
💡創造モード
💡シェーダー
💡砂漠生物
💡コード
💡パラメーター
💡Redstone
💡Nether portal
💡GPT
Highlights
Building in creative mode with AI in Minecraft using shaders and a desert biome.
Construction of a large pyramid controlled by Claude, Opus, a language model.
Efficient row construction method by reversing order for each row.
Claude Opus' ability to follow specific construction requests.
Introduction of models: Claude Opus, Gemini, and Llama Llama 3.
Llama Llama 3 is an open-source model from Meta with 70 billion parameters.
Replicate API used for running Llama Llama 3, though not free.
Llama Llama 3's issues with character omissions and incorrect block placements.
Bots' ability to place blocks through other blocks without a clear line of sight.
GP Omni's new text model is fast, cheap, and has a casual personality.
Building a Roman-style building with columns as a tricky challenge.
GPT and Claude's close attempts at building a redstone contraption.
Llama Llama 3's failure to build the redstone contraption correctly.
Challenge of building a desert castle with specific features.
GPT's successful construction of a desert castle with towers and torches.
Claude's creative castle with torches but lacking doors and staircases.
Llama Llama 3's weak performance and simplified prompt outcome.
Exploration of redstone contraptions and electrical systems in Minecraft.
GPT's successful completion of redstone contraption challenges.
GPT 4's incorrect but impressive attempt at building a nether portal.
The potential and future exploration of AI in Minecraft construction.
Transcripts
hello welcome to another episode of
Minecraft today we'll be building even
more stuff with AI in creative mode you
can see I've got some fancy shaders on
we're now in a desert biome and we have
a little construction project going on
here this is Andy and what better thing
to build with infinite resources than a
big ass
pyramid Andy is controlled by Claude
Opus a language model that wrote this
piece of code to generate the structure
there were some earlier failed attempts
but this one turned out pretty good and
took several Minecraft days to build the
full time-lapse of this build is on my
patreon also notice that they're
building each row back and forth in
reversing order which is much more
efficient than having to start all the
way over on the same side for each row
the order of block placement matters
especially for big builds like this and
Claude Opus was able to follow my
request to build in this way
and it placed a diamond block on top
ain't that
pretty uh this isn't the most
technically impressive build ever but I
think it looks
[Music]
awesome okay let's meet our models for
this video of course we have Claude Opus
a new model for GPT and the same Gemini
as before and then there's the new guy
llama llama 3 is the open source model
from Facebook I mean meta and this is
the big set 70 billion parameter model
and because it's open source you can run
this model or the smaller model on your
own computer if you have the
hardware hey
Gemini this instance of llama is running
through the replicate API which is not
free but much more convenient for me a
llama is okay I think it's about as good
as GPT 3.5 this house took a couple
tries and it's very simple llama is also
buggy if that's the word for it like it
will sometimes just leave out characters
and write and stone instead of sandstone
which can really ruin the code that it
writes also yes the Bots can place
blocks through other blocks they don't
need a clear line of sight this is
another way that mind flare cheats they
do need to be close enough though okay
so even though we're in creative mode
I'm still giving them a pallet of blocks
to choose from everything you see in
this top section rather than the
hundreds of available blocks in the game
this keeps them from getting confused
and doesn't eat up their contact space
so let's let's get started with GP Omni
no it's not their weird flirtatious
Voice Chat thing that totally isn't
Scarlet Johansson well hello there cutie
this is just their new text model which
is super fast and cheap but supposedly
about as good as GPT 4 and for the
record I like its personality it's a
little more casual less wordy it uses
emojis I like it let's start with
something tricky a Roman style building
with columns
that's pretty good it's missing a roof
but that's fine and yes the Bots are
still using scaffolding I have not added
the flying ability yet and I actually
think the scaffolding problem is a
really interesting one to solve a decent
solution is to just use a unique block
like dirt in a desert and then simply
have the agent remove all dirt when it's
done building that works pretty good but
not
perfectly okay now
Claude a little simple but they are
technically columns good job and now for
llama llama was a little more
ambitious and it doesn't space out the
column so it ends up building a giant
[Music]
Cube and then it stopped right in the
middle of the build because I forgot to
turn off this 10-minute timer that keeps
code from running indefinitely I'll turn
that off for the rest of the video but I
think we can see where it was going not
that impressive let's move on to Gemini
I promise I'll be nicer to Gemini but
this is the same model as before Gemini
1.0 so it can't build anything and it
can hardly follow instructions does
anyone know how to get API access to
Gemini 1.5 not through their AI Studio
through an API key I think I need to be
whitelisted by someone at Google so if
you are someone who can get me access
send me a DM on Twitter or Discord I'd
love to give Gemini a fair Shake but
Google does not make it easy for now
Gemini is on Cactus collecting d
[Music]
all right let's be more ambitious I want
a desert castle with walls doors torches
towers crenellations and staircases I'm
giving each Model A few tries here this
is a big one
[Music]
[Music]
[Music]
[Music]
it started building a second castle
overlapping with the first one so I
stopped it but I'd say it did pretty
good it's Grand if a bit plain there are
Towers a couple torches but no doors no
crenellations no staircases and a lot of
scaffolding which I had it remove here's
a question how do you remove scaffolding
that requires scaffolding to get to
right now it builds up to each block
then removes what it just built only to
build it again for the next block it's
very inefficient but it will eventually
so they remove all
dirt while it's doing that let's give
Claude a shot
[Music]
[Music]
[Music]
[Applause]
[Music]
look GPT finished removing its SC fing
and came to help Claude remove theirs
that is really cute I really like
claude's Castle the Torches are a nice
touch the towers are lopsided and again
no doors or staircases but still very
good
job and here's gpt's castle without
scaffolding yes there are still a few
Sandstone scaffolding blocks that I
think it placed by
[Music]
accident now I tried getting the same
prompt to work with llama but it crashed
about 15 times it kept writing a broken
code block so I had to simplify the
prompt and this is what it gave
me it's just another box and it took
forever to
[Music]
build well okay it did end up adding
Towers so there's that it's okay
definitely the weakest of the three it
did make this monstrosity earlier when I
wasn't recording which I kind of love a
box floating over a layer of torches and
this is the closest we've got to actual
crenellations now onto a new kind of
challenge redstone contraptions if you
don't know Redstone is basically the
electrical system in Minecraft and it
can get complicated we'll try these
three simple builds a redstone lamp
controlled by a lever five lamps
controlled control by the same lever
which requires some redstone dust or
clever positioning and an or gate where
the lamp is only on if the right lever
or the left lever is on or both all
pretty simple but nuanced don't expect
too much from these
models right off the bat GPT does it
perfectly on the first try and even
turns it
on eventually Claude got it too and
happened to build almost exactly the
same thing GPT
built and llama uh well never quite
figured it
out a little confused but it's got the
spirit for five lamps controlled by one
lever GPT for some reason runs over here
to build it but almost gets
it the Redstone needs to be one block up
and it would have worked see Redstone is
kind of tricky llama didn't place a
lever or Redstone so it failed and
Claude didn't quite get it
either I tried to tell them more
specifically how to build the lamps to
put the redstone on top and both GPT and
Claude got really close if it had just
plac the lever on the side it would have
worked so
[Music]
close for the orgate GPT wasn't able to
wire up the
[Music]
Redstone llama actually got really close
it just needed to put the lamp in the
center and Claude technically got it if
we count the Redstone wire as the
output eventually I got a little tired
of these finicky Contraptions and
decided to take things in a more
destructive Direction
Claude even built a fuse for its
bomb GPT definitely had some fun blowing
stuff
up pretty impressive but we have a long
way to go before we build the Minecraft
nuke
[Music]
okay one last challenge the nether
portal it needs to be a square outline
of obsidian blocks with fire lit at its
base to activate it this is tricky but
in theory possible hey Gemini good
try GPT got close but it used the wrong
Dimensions it needs at least an inner
width width of two and height
three Claude keeps stopping early for
some reason and then builds it sideways
which is again the wrong
Dimensions eventually it did get the
right shape but failed to activate it by
[Music]
itself after multiple tries and hints
all models failed to produce the nether
portal even Gemini
as a last Stitch effort I tried out GPT
4 not four Omni not four turbo just four
this is I think the most expensive model
on the market right now but possibly the
best and it's totally wrong I don't even
know what it's doing why is it oh I
think I see what it's doing it might be
the right shape I wonder if
[Music]
[Music]
this is the first time an agent has been
to the nether I've never brought them
here so it's pretty incredible that it
got here by itself sort of and that was
GPT 4's first try it did immediately get
confused and accidentally left as soon
as it arrived but whatever it's still
impressive I'm lucky I recorded it
I think you'll agree that there is
enormous potential here and I am not
done exploring all the things you can
build and do with these
agents I'll see you later
[Music]
[Music]
h
5.0 / 5 (0 votes)