WWDC23: Build spatial experiences with RealityKit | Apple
Summary
TLDRこのビデオスクリプトでは、Appleプラットフォームの新しいRealityKitフレームワークが紹介されています。RealityKitを使用して、3Dモデルとエフェクトをリアルにレンダリング、アニメーション化、シミュレートすることができます。スピーカーは、RealityViewやSwiftUIと組み合わせて3Dコンテンツを追加し、ジェスチャー、アニメーション、空間音を活用してインタラクティブな体験を作り出す方法を解説しています。カスタムコンポーネントやシステムを通じて、アプリの特定のニーズに合わせた3D体験を拡張する方法も紹介されています。
Takeaways
- 🎨 RealityKitは、3Dモデルとエフェクトをリアルにレンダリング、アニメーション化、シミュレートするためのフレームワークです。
- 📅 RealityKitは2019年に導入され、その後多くの新機能が追加されました。
- 🌐 RealityKitを使用すると、アプリの2Dウィンドウに3Dコンテンツを追加したり、3Dコンテンツを別のウィンドウに中央に表示したり、自己と環境を没入型体験に持ち込むことができます。
- 📱 RealityKitはAppleプラットフォームのコア3Dフレームワークであり、特にxrOSでは多くの機能を提供しています。
- 🔑 このプレゼンテーションでは、エンティティ、コンポーネント、RealityViewなどの重要な機能に焦点を当て、RealityKitの使い方を紹介します。
- 🌍 'Hello World'体験の3D要素を作成するために、SwiftUI、RealityKit、Reality Composer Proを使用する方法を紹介します。
- 🛠️ RealityKitとSwiftUIを組み合わせることで、アプリに3D要素を追加し、次の次元へと発展させることができます。
- 🌌 RealityViewは、SwiftUIのビューで、RealityKitのエンティティを用いて3Dコンテンツを構成することができます。
- 👀 RealityKitのエンティティはコンポーネントを通じて特定の動作を実装し、モデルコンポーネントや変形コンポーネントなどがあります。
- 🤖 RealityKitのカスタムコンポーネントとシステムを定義することで、アプリの特定のニーズに合わせた機能を構築できます。
- 🎧 RealityKitのSpatial Audioを使用すると、3D空間内で音がリアルなように聞こえる効果を作り出せます。
Q & A
RealityKitは何のためのフレームワークですか?
-RealityKitは、3Dモデルとエフェクトをリアルにレンダリング、アニメーション化、シミュレートするためのフレームワークです。
RealityKitはいつ導入されたのか、またその後でどのような新機能が追加されましたか?
-RealityKitは2019年に導入され、その後多くの新機能が追加されています。
RealityKitを使用することで、どのような3Dコンテンツの体験を作成できますか?
-RealityKitを使用することで、アプリの2Dウィンドウに3Dコンテンツを追加したり、独立したウィンドウで3Dコンテンツを前面にしたり、自己と環境を没入型体験にすることができます。
RealityKitはどのようにしてAppleプラットフォームのコアフレームワークとなっていますか?
-RealityKitはAppleプラットフォーム上で3Dコンテンツを作成するための主要なフレームワークであり、特にxrOSでは多くの機能を提供しています。
RealityViewとは何で、どのような機能を提供していますか?
-RealityViewは新しいSwiftUIビューで、3Dモデルとエフェクトを表示するために使用されます。これはRealityKitのエンティティを使用して3Dコンテンツを構成することができます。
RealityKitで扱える3Dコンテンツの形式として、何が挙げられますか?
-RealityKitで扱える3Dコンテンツの形式として、USDZファイルなどが挙げられます。
RealityKitでエンティティに追加できるコンポーネントの例をいくつか挙げてください。
-RealityKitでエンティティに追加できるコンポーネントの例として、モデルコンポーネント、トランスフォームコンポーネント、入力ターゲットコンポーネント、コリジョンコンポーネントなどがあります。
RealityKitにおける「エンティティ」と「コンポーネント」の違いは何ですか?
-エンティティはレンダリングや動作を持たせるためのコンテナーオブジェクトであり、コンポーネントはエンティティに特定の動作を提供するものです。
RealityKitでカスタムコンポーネントを作成するにはどうすればよいですか?
-カスタムコンポーネントを作成するには、コンポーネントプロトコルに準拠し、必要なデータを含めます。Codableプロトコルに準拠することで、Reality Composer Proのインターフェイスで直接追加できるようになります。
RealityKitにおける「システム」とは何で、どのような役割を果たしますか?
-システムはエンティティとコンポーネントに対してコードを実行する仕組みで、3D体験の外観や動作をモデル化するECS(Entity Component System)の一部分です。
RealityKitでの実空間音声を扱うためにはどのようなタイプのオーディオコンポーネントがありますか?
-RealityKitでの実空間音声を扱うためには、空間オーディオコンポーネント、環境オーディオコンポーネント、チャンネルオーディオコンポーネントがあります。
Outlines
😀 リアリティキットの紹介と基本機能
ジョンはRealityKitチームのエンジニアとして、新しいRealityKitフレームワークを紹介します。これは3Dモデルとエフェクトをリアルにレンダリング、アニメーション化、シミュレートするためのツールです。2019年に発表され、その後多くの新機能が追加されました。RealityKitを使用して他のデバイスでアプリを構築していた場合、共通点が多く見られるでしょう。RealityKitはAppleプラットフォームのコア3Dフレームワークであり、xrOSでは特に多くの機能を提供しています。このプレゼンテーションでは、エンティティ、コンポーネント、RealityViewなどの主要な機能に焦点を当て、RealityKitの使い方を紹介します。
🌐 3DコンテンツとSwiftUIの統合
RealityKitとSwiftUIを組み合わせることで、2Dウィンドウに3Dコンテンツを追加し、アプリを次の次元へと引き上げることができます。SwiftUIはビューとウィンドウを定義するのに使用され、RealityKitは3D要素を追加するのに使われます。例えば、WorldアプリではSwiftUIを使って標準の2Dウィンドウとボタンを表示していますが、Planet Earthボタンをタップすることで3D地球儀を表示する詳細ビューに移動できます。RealityKitのModel 3D Viewを使用して2Dウィンドウに3Dコンテンツを追加する方法や、ボリュームウィンドウを使用して3Dコンテンツを前面にする方法について説明します。
🎨 RealityViewを使った3Dコンテンツの作成
RealityViewはRealityKitのエンティティを使用して3Dコンテンツを作成する新しいSwiftUIビューです。エンティティはコンテナオブジェクトであり、コンポーネントを通じて特定の動作を実装します。モデルコンポーネントとトランスフォームコンポーネントがEarthエンティティに追加され、3D空間に配置されます。RealityKitはARKitと同じ3D座標規約を使用しており、RealityViewのコンテンツインスタンスには座標空間を変換する機能があります。RealityViewはエンティティを追加し、レンダリング、アニメーション化、シミュレーションを行うためのシンプルな方法を提供します。
🤖 エンティティとコンポーネントの詳細
エンティティはRealityKitの基本的なオブジェクトであり、コンポーネントを通じて振る舞いが定義されます。モデルコンポーネントはメッシュとマテリアルをエンティティに提供し、トランスフォームコンポーネントは3D空間内の位置、向き、スケールを制御します。エンティティは複数の子エンティティから成り立ち、それぞれが独自のコンポーネントセットを持っています。RealityKitには様々なコンポーネントがあり、コリジョン、入力ターゲット、ホバーエフェクトなどがあります。さらに独自のコンポーネントを作成する方法も紹介されています。
🎭 リアリティビューとインタラクティブな3Dエクスペリエンスの作り方
RealityViewを使用して3D空間にエンティティを配置し、ジェスチャーやアニメーション、スペシャルオーディオでインタラクティブな3Dエクスペリエンスを作成する方法について説明します。RealityViewはSwiftUIビューにジェスチャーを追加し、エンティティに入力ターゲットコンポーネントとコリジョンコンポーネントを追加することで、インタラクションに対応させます。さらに、ホバーエフェクトコンポーネントを追加して、エンティティがインタラクティブであることを示します。アニメーションとスペシャルオーディオを用いて、シーンをより生き生きとさせ、リアルな3D体験を作り出します。
🛰️ カスタムコンポーネントとシステムの活用
RealityKitでは、既存の機能を組み合わせて独自のコンポーネントやシステムを作成し、3Dエクスペリエンスを構築することができます。コンポーネントは3D体験の1つの側面を制御するデータであり、エンティティにグループ化されています。カスタムコンポーネントの例として、トレースメッシュを持つコンポーネントが紹介され、その実装方法について説明します。システムはエンティティとコンポーネントに作用するコードであり、ECSと呼ばれるツールを使用して3D体験の外観と振る舞いをモデル化します。トレースシステムの例として、衛星エンティティの後を追うラインメッシュを更新するシステムが紹介されています。
🎉 RealityKitの機能紹介と今後の展望
RealityKitの紹介を締めくくり、RealityKitとRealityViewを使用してSwiftUIで定義されたビュー、ウィンドウ、イマーシブスペースに3D要素を追加する方法について要約説明します。USDファイルを読み込み、ジェスチャーを処理し、アニメーションとスペシャルオーディオを再生する機能があります。さらに、カスタムコンポーネントとシステムを定義してアプリの特定のニーズに応じた機能を構築できます。他のセッションも参照され、RealityKitの豊富な機能を使用してアプリを構築する際の参考になります。
Mindmap
Keywords
💡RealityKit
💡3Dモデル
💡スペーシャル体験
💡SwiftUI
💡エンティティ
💡コンポーネント
💡RealityView
💡アニメーション
💡スペーシャルオーディオ
💡カスタムコンポーネント
Highlights
RealityKitは、3Dモデルとエフェクトをリアルにレンダリング、アニメーション化、シミュレートするためのフレームワークです。
RealityKitは2019年に導入され、その後多くの新機能が追加されました。
RealityKitを使用すると、2Dウィンドウに3Dコンテンツを追加したり、独立したウィンドウで3Dコンテンツを前面に表示したり、没入型体験に自分と環境を持ち込むことができます。
RealityKitはAppleプラットフォームのコア3Dフレームワークであり、xrOSでは特に多くの機能を提供しています。
プレゼンテーションでは、エンティティ、コンポーネント、RealityViewなどの主要な機能に焦点を当て、RealityKitの使用方法を紹介します。
「Hello World」体験を用いて3D要素の構築方法をSwiftUI、RealityKit、Reality Composer Proで説明します。
RealityKitとSwiftUIを組み合わせることで、アプリを次の次元に持ち上げることができます。
RealityKitのビルディングブロックであるエンティティとコンポーネントについて学びます。
RealityViewは、3Dモデルとエフェクトのための新しいSwiftUIビューで、RealityKitエンティティを使用して3Dコンテンツを組合わせることができます。
入力の処理やアニメーション、Spatial Audioでアプリを活性化する方法について説明します。
カスタムコンポーネントとシステムでRealityKitの可能性を解き放す方法について話します。
SwiftUIはビューとウィンドウを定義する方法で、RealityKitは3D要素を追加することができます。
2Dウィンドウに3Dコンテンツを追加するには、RealityKitのModel 3D Viewを使用して簡単に実現できます。
3Dコンテンツを別々のウィンドウで前面に表示するために、体積ウィンドウを使用することができます。
Immersive Spaceを使用すると、アプリはウィンドウの範囲を超え、没入型体験を提供できます。
RealityKitのエンティティはコンテナオブジェクトであり、コンポーネントを通じて特定の振る舞いを実装します。
RealityViewはRealityKitエンティティを含むSwiftUIビューで、エンティティをレンダリング、アニメーション化、シミュレートするために使用されます。
RealityKitのカスタムコンポーネントとシステムを使用して、アプリの特定のニーズに合わせた機能を構築できます。
RealityKitとReality Composer Proを使用して、3Dコンテンツを作成し、XcodeでSwiftUIと組み合わせて没入型アプリを開発できます。
Transcripts
♪ Mellow instrumental hip-hop ♪
♪
Hello. I'm John, and I'm an engineer
on the RealityKit team.
Today, I'm delighted to introduce you
to the new RealityKit for creating spatial experiences.
RealityKit is a framework for realistically rendering,
animating, and simulating 3D models and effects.
We introduced RealityKit in 2019
and have added a lot of new features since then.
If you've already used RealityKit
for building apps on other devices,
you'll find that there's a lot in common.
With RealityKit, you can augment your app's 2D windows
with 3D content, bring 3D content front and center
in a separate window, or bring yourself and your environment
into an immersive experience.
RealityKit is a core 3D framework on Apple platforms.
And especially on xrOS, it offers a lot of features.
In this presentation, I'll focus on some key features,
like entities, components, and RealityView,
which will introduce you to RealityKit
and show you how to use it.
I'll also mention sessions that cover other concepts
or go into more detail.
Let's get started.
I'll use the Hello World experience to explain
the concepts in this presentation.
But before I get to those concepts,
let me take you through the three different modules
that are a part of this experience.
The first module, Planet Earth,
presents a 3D globe in its own window
that you can interact with from any angle.
The second module, Objects in Orbit,
immerses you in a model of the Earth, the Moon,
and a satellite, demonstrating animations,
Spatial Audio, and custom behaviors,
like the trace which follows the satellite.
The third module, the Solar System,
contains a fully immersive experience,
which you can learn more about in other sessions.
I'm going to show you how to build the 3D elements
of this Hello World experience using SwiftUI, RealityKit,
and Reality Composer Pro.
Let's dive in.
I'll get started by talking about how you can use RealityKit
together with SwiftUI to bring your app
into the next dimension.
Next, I'll examine the building blocks of RealityKit,
entities like the Earth model and the components
which implement its behaviors.
You'll learn about the features of RealityView,
which is a new SwiftUI view for 3D models and effects.
Then I'll explain how you can handle input
and bring your app to life with animation and Spatial Audio.
Finally, I'll talk about unlocking the potential
of RealityKit with custom components and systems.
Let's begin by exploring how RealityKit
and SwiftUI work together.
SwiftUI is how you define your views and windows,
and RealityKit lets you add 3D elements.
For example, the World app uses SwiftUI to display
a standard 2D window with a few buttons.
Tapping the Planet Earth button on the left
navigates to a detail view,
which shows a 2D illustration of the Earth.
But what if I want to replace that 2D image with a 3D globe?
Adding 3D content to a 2D window is easy
using the model 3D view in RealityKit.
Let's go over the code.
Here's the SwiftUI view, which displays that globe image.
I'll change it to display a 3D globe by importing RealityKit
and changing the image to a model 3D view,
referencing a USD file in my project called Globe.
We can customize the loaded model before displaying it
by adding two pieces of code:
a content closure for the model that is returned
and a placeholder view builder to specify a view
that's displayed while the model is loading.
In the content closure,
I'll add resizable and scaledToFit modifiers
to make sure that the model fits into the available space
in my UI.
And because Model 3D loads its model asynchronously,
there is a placeholder view builder,
which you can use to customize the view
that's displayed during the loading process.
In this case, I'm just using the built-in ProgressView
to display a spinner.
Finally, I need to add the globe 3D model as a resource
to the app or Swift package.
And now the model appears in line in the app
with the same appearance that it has in Quick Look
or Reality Composer Pro.
Placing 3D content in a 2D window is great,
but I want my 3D model to be front and center.
To accomplish that,
I'm going to put the globe in a separate window.
I'll add a button to the app's detail view
to open that new window.
And instead of using a regular window,
which displays its contents against a 2D background,
I'll use a new window style
which places its contents in a volume.
This is called a volumetric window,
and it's great for viewing 3D content.
Unlike a 2D window,
a volumetric window is meant to be used from any angle,
so it's ideal for 3D models.
A volumetric window also keeps a fixed size
that is true to life.
You can scale a model in a volumetric window
to be one meter across,
and it will always be displayed at exactly that size.
I think that's a great fit for the globe in Hello World.
Let's go through the process of adding a volumetric window.
First, I'll add a window group to my app.
Window groups act as a template that an app can use
to open new windows.
I'll give the new window an identifier to distinguish it
from this app's main window.
Next, I'll add a windowStyle volumetric modifier
to the window group.
I'll also add a defaultSize modifier
to give this window a size in meters.
Finally, I'll add a button to the detail view.
To make the button open the window I've just added,
I'll add a property that gives me access
to the open window action from the SwiftUI environment.
Then I'll call that action from my button.
Let's run this app in the simulator.
When I press the View Globe button,
a volumetric window appears.
Now I can interact with the globe from multiple angles,
not just the front.
But sometimes, the key to unlocking the experience
you want to create is immersion.
In the Objects in Orbit module of the World app,
you're immersed in an animated model of the Earth
and its satellites that demonstrates their orbits.
This uses an immersive space, a new scene type
which allows your app to place 3D elements anywhere in space.
When you open an Immersive Space,
your app can go beyond the bounds of a window
and provide a magical experience.
Adding an Immersive Space
is similar to adding a window group.
It's a new scene in the body of my app.
Here I'm using RealityView,
which will give me more control over the scene than Model 3D.
I'll go over RealityView in more detail in a few minutes.
Like before, I'll add a button to the app's detail view.
I'll get the openImmersiveSpace action from the environment
and call it with the identifier of the scene I've defined.
Note that the openImmersiveSpace action is asynchronous.
It completes once the space has finished opening.
When I press the View Orbits button,
an Immersive Space appears.
This is already stunning,
but you can make it more engaging
by adding interactivity, animation,
and audio with RealityKit.
Whether you're working with 2D windows containing 3D content,
or volumetric windows that emphasize your 3D models,
I encourage you to check out these SwiftUI sessions
if you haven't already.
The "Meet SwiftUI for spatial computing" session
is an overview of what's new with SwiftUI on this platform.
The "Take SwiftUI to the next dimension" session
demonstrates how to get the most out of 3D content in a window.
There are also multiple styles of immersion.
The solar system module of Hello World
uses a fully immersive space that hides passthrough
and displays its own backdrop.
The "Go beyond the window with SwiftUI" session
goes into detail about all styles of Immersive Spaces.
If you're thinking about creating
an immersive experience, I highly recommend this talk.
You've encountered two ways to use RealityKit
in your SwiftUI views,
the easy-to-use Model 3D and RealityView.
RealityView is what I'll use for the rest of this session
because it allows you to compose your 3D content
using RealityKit entities.
So what is a RealityKit entity?
An entity is a container object.
If you create an empty entity from code,
it won't do anything.
To make an entity render or give a behavior,
it must have components.
Each component enables some specific behavior for an entity.
Here are some examples.
The Earth entity in this app is implemented with two components:
a model component, which gives the entity a mesh and materials,
and a transform component,
which places the entity in 3D space.
The same is true of the satellite entity.
The model component renders a mesh
and applies materials to it.
These Earth and satellite models
were created in a digital content creation tool,
exported to a USDZ file, and loaded into RealityKit.
These meshes have a physically-based material
applied to them to give them their final appearance.
A material uses a set of textures and shaders
to describe how the surface of a mesh responds to light.
To learn more about materials,
I recommend watching Niels' session,
"Explore materials in Reality Composer Pro."
In addition to a model,
these entities have a transform component.
The transform component places an entity in 3D space.
You can control an entity's position, orientation,
and scale by setting properties on its transform component,
as well as by setting the entity's parent.
RealityKit uses the same 3D coordinate conventions as ARKit
and many other 3D engines.
The origin is at the center of the RealityView.
The y-axis points upward, the z-axis points towards you,
and the x-axis points to your right.
One unit is one meter.
Note that these conventions are different
from SwiftUI's conventions.
There are functions on RealityView's content instance
that make it easy to convert back and forth
between the RealityKit and SwiftUI coordinate spaces.
Every entity has a transform,
but not every entity has a model.
Sometimes an entity is assembled out of multiple child entities,
each with their own set of components.
This gives you more programmatic control.
For example, you could play individual animations
on the transforms of child entities.
RealityKit contains a variety of components
depending on what you want to do.
I'll talk about some specific components today,
collision, input target, and hover effect
just to name a few.
I'll also demonstrate how to create your own components.
Now that we've got a sense of how entities
and components work, let's use RealityView
to place those entities in your app.
RealityView is a SwiftUI view
that contains any number of entities.
Entities need to be added to a RealityView
in order to be rendered, animated, and simulated.
So, how does a RealityView work?
RealityView provides a content instance
which lets you add entities to the view.
This is an easy way to get started
if you have already loaded an entity
or if you want to create an entity programmatically.
But this closure is asynchronous,
so it's simple to load an entity from a file
and display it in your view.
Here I asynchronously load an Earth model from a USD file
and add it to the content instance
once the load has completed.
You can also load more than one model
and place both of them in your RealityView.
Instead of being laid out next to each other,
these models will coincide in space.
If that's not what you want,
you can change the position of the entities added to the view.
This example positions the moon entity
half a meter to the right.
Once you've set up your RealityView,
you may want to connect state in your app to properties
stored on RealityKit components.
RealityView lets you express the connections
between SwiftUI managed state and the entities
in a RealityView with an update closure.
This makes it easy to drive the behavior of 3D models
with a source of truth from your app's data model.
This view loads a model and applies a rotation
controlled by whoever uses the view.
Note that the code in the update section only runs
when the values that it depends on change.
If you're building a UI with a combination of 2D
and 3D elements, you'll sometimes need to convert
coordinates between views and entities.
RealityView provides coordinate conversion functions
between SwiftUI view coordinate spaces
and RealityKit entity coordinate spaces.
The RealityView's content instance
provides a convert function that converts points,
bounding boxes, and transforms from a SwiftUI coordinate space
to an entity's local space or vice versa.
Here, I get the minimum length of any of the view's dimensions
and scale the loaded entity to fit in the available space.
RealityView also provides a mechanism
for subscribing to events published by entities
and components.
In this example, I play an animation
authored in the loaded USD file after the load completes.
The content instance has a subscribe to: function,
which adds an event handler.
This example runs some code when an animation completes.
There are RealityKit events published
for all kinds of things, from animation to physics to audio.
You can also attach SwiftUI views to entities.
The attachments feature of RealityView makes it easy
to position views in 3D space.
Check out Yujin’s session,
"Enhance your spatial computing app with RealityKit,"
to learn more.
There's a lot you can do with RealityView.
But let's get back to our celestial bodies
and bring them to life.
First, I'll show you how to add a drag gesture
so that you can reposition the Earth entity.
And then I'll explain animations and Spatial Audio.
Here's an example RealityView containing three entities.
You can add a gesture to a RealityView,
like any other SwiftUI view,
and it will hit test against entities in that view.
To receive input, the entity must have both
an input target component and a collision component.
When a touch event is handled by RealityView,
it will ignore any entities that don't have both collision
and an input target.
Only this last entity has both components,
so gestures added to this RealityView
will only react to input directed at this entity.
To make the Earth entity draggable,
I'll give it an input target component
and a collision component, and add a drag gesture
to the RealityView.
To add the components, I'll use Reality Composer Pro.
Reality Composer Pro is a new developer tool
that lets you compose, edit, and preview 3D content.
I'll just use it to add a few components to an entity.
To learn more about what you can do with Reality Composer Pro,
check out Eric's session, "Meet Reality Composer Pro."
The World app already has a World Assets package set up
containing the USD files that this experience uses.
I'll open that package in Reality Composer Pro.
The Earth model is in a USDZ archive,
which is self-contained and not meant to be modified.
Instead of modifying that asset, I'll create a new USD scene file
and reference the Earth asset.
USD files can reference other USDs and modify them in place
without actually changing the referenced file.
Nondestructive editing like this is really handy
when you need to make small changes to a USD file
that someone else is working on.
I'll create a new scene named DraggableGlobe
and drag in the globe file to create a reference to it.
Now I can add components to it.
I'll add an input target component
and also a collision component.
The default shape for the collision component
is a cube.
I'll change it to a sphere so it better matches the model.
It's important for the collision shape to be
a reasonable approximation of the visual model.
The closer the match,
the more intuitive interactions with the model will be.
I want to be able to move the Earth model around,
so I'll add a drag gesture to the RealityView.
A standard SwiftUI drag gesture will work,
but I can enable the gesture to manipulate specific entities
rather than the entire view
by adding a targetedToEntity modifier on the gesture.
When the gesture's value changes,
I'll change the entity's position to match.
There's one critical detail though.
The gesture's value is in SwiftUI's coordinate space,
so I have to convert it to RealityKit's coordinate space
in order to change the entity's position.
All the pieces are now in place.
So in the Objects in Orbit module,
I can now pinch and drag to move the Earth around.
Great, our app is now interactive.
But I'd like my Earth entity
to indicate that it's interactive.
There's a RealityKit component we can use for this,
the HoverEffectComponent.
Hover effects provided by SwiftUI and RealityKit
are the only way to make your app react
to where you're looking.
This effect is applied outside of your app's process
in a privacy-preserving manner.
I'll add a hover effect component to the Earth entity
when it's added to the RealityView.
Now, the Earth model lights up when the pointer is over it
to indicate that I am able to interact with it.
Next, let's move on to animations.
RealityKit has a number of built-in animation types,
such as from-to-by animations,
which animate a property from an initial value to a final value,
orbit animations that cause an entity to revolve
around its parent, and time-sampled animations
that progress frame by frame through a series of values.
I'll set up an orbit animation on the Moon.
The Moon will take 30 seconds to complete a full orbit
and the orbital axis will be the y-axis.
And I'll make sure that the orbit starts
from the Moon's current position.
Once I've defined the properties of this orbit animation,
I'll generate an animation resource for it
and play that animation on the Moon entity.
And now the Moon orbits the Earth.
To me, this is the magical moment.
With an animation in place, the scene feels alive.
But while animation helps bring your 3D content to life,
Spatial Audio makes your model feel like it's really there.
There are three types of audio in RealityKit:
spatial, ambient, and channel.
Let's look at each one of them in more detail.
RealityKit sounds are spatial by default,
so audio sources sound like they actually exist
in your surroundings.
Spatial Audio Component lets you customize how your objects
emit sound into your space to make them even more realistic
or more artistic.
Use directivity to emit sounds in all directions
or project sound in a specific direction.
Ambient Audio Component is great for multichannel files,
which capture the sound of an environment.
No additional reverb is added to ambient sources.
Each channel of the ambience is played from a fixed direction.
And finally, Channel Audio Component
sends the audio files channels directly to the speakers
without any spatial effects.
This is ideal for background music
which is not connected to any visual element.
You can add audio to your scene in Reality Composer Pro
and interface with it using RealityKit.
Or you can hook up audio in code.
Let's take a look.
I'll add a bit of looping audio to an orbiting satellite.
First, I'll add a Spatial Audio Component to an empty entity
that will act as the audio source.
A directivity of 0.75 creates a tight beam of sound
in a particular direction.
I'll rotate that audio source entity around its y-axis
so that the audio is projected in the direction I want.
I'll then load a looping audio clip from a resource
and play it on the audioSource entity by calling playAudio.
Let's see this in action.
[DISTANT BEEPING]
Since the Spatial Audio source is configured
with a tightly focused directivity,
the audio can be heard clearly on my side of the Earth,
but it is quieter when the satellite is on the other side.
That was input, animation, and audio.
You can build more functionality in RealityKit by combining
its existing functionality in different ways.
There are two primary tools you can use for this purpose,
defining your own components and defining your own systems.
A component contains the data controlling one aspect
of a 3D experience.
Components are grouped into entities.
Without components, an entity does nothing.
Each component supplies a single element
of an entity's implementation.
You've learned that a transform component
positions an entity and that a model component
renders a 3D model.
In addition to the predefined components
that RealityKit provides,
you can define your own components.
Here's an example custom component
that contains a traceMesh object
that my coworker Paul has created.
The trace component type
conforms to the component protocol,
so you can get and set this component
on any entity at runtime.
You can also adopt a data-driven workflow
by defining a component in a Swift package
and conforming it to the Codable protocol.
Codable components will appear
in the Reality Composer Pro interface
and can be directly added to entities at design time.
You can learn more about custom components in the talk
"Work with Reality Composer Pro content in Xcode."
I already went through entities earlier in this talk,
and I just covered components.
Next, let's talk about systems.
Systems contain code that act on entities and components.
Taken together, entities, components, and systems, or ECS,
are a tool for modeling the appearance
and behavior of your 3D experience.
Systems are a way to structure the code
that implement your app's behaviors.
Code in a system runs at a regular interval,
acting upon your component's current state.
For example, this TraceSystem updates a line mesh
that's traced behind the satellite entity
as it orbits the earth.
Each update, it adds the entity's current position
to the trace.
Once a system is registered,
it automatically applies everywhere in your app
that you use RealityKit.
Registering the trace system in my app's initializer
causes it to update for all relevant entities.
But what entities are relevant, and when does the system update?
This system only wants to update entities with a trace component,
so I create an entity query that filters to entities
which have a trace component.
In the update function, the system passes in
the entity query and also specifies
that it wants to update entities when rendering.
The rendering condition means that this system will update
at an appropriate rate for smooth animations.
Here's the trace system in action,
adding the entity's position to the line mesh
in order to produce a fluid custom animation.
Systems are a really effective way to implement
a variety of effects and behaviors.
RealityKit has a lot of features that make it easy
to build 3D apps.
You can use RealityKit and RealityView to add 3D elements
to views, windows, and immersive spaces defined with SwiftUI.
You can load USD files, handle gestures, and play animation
and Spatial Audio, all using RealityKit.
RealityKit provides many predefined components,
but you can also define custom components and systems
for your app's specific needs.
With that, I've covered the concepts you need
to get started with RealityKit.
Yujin’s session,
"Enhance Your spatial computing app with RealityKit"
will take you through more features of RealityKit,
like portals, particle emitters, attachments, and more.
And Amanda's session
"Work with Reality Composer Pro content in Xcode"
takes you through the process of building an immersive app
using Reality Composer Pro, RealityKit,
Xcode previews, and the simulator.
There are a lot of exciting features in RealityKit
that you can use in your app.
I'm really excited to see the wonderful experiences
you'll create.
Thanks for watching.
♪
Ver Más Videos Relacionados
5.0 / 5 (0 votes)