Smooth VR Spectator Tutorial with Unreal Engine 5

Summary

In this tutorial, the speaker explains how to enhance the experience for VR spectators by smoothing camera movements. The focus is on reducing jittery visuals when users in VR move quickly, which can be disorienting for viewers watching on a screen. The tutorial walks through using Unreal Engine 5's VR Spectator Blueprint to assign input keys and adjust camera settings for smoother transitions. The guide also covers how to manage pixel density for better performance and adjust the spectator camera to mimic the player's head movement in VR, providing a more stable viewing experience.

Takeaways

• 🎮 The tutorial focuses on improving the VR spectator experience by smoothing out camera movements to reduce jitter and enhance viewer comfort.
• 👓 The VR experience involves two main components: the VR player character and the VR spectator camera, which are controlled by different blueprints.
• 🔧 To adjust the quality of the VR experience, one can modify the pixel density in the 'Event Begin Play' to balance quality and frame rate.
• 🕹️ A key is assigned within the VR spectator blueprint to toggle between a dampened (smooth) and non-dampened (jittery) camera view.
• 📊 The tutorial demonstrates how to use a Boolean variable to flip between dampened and non-dampened camera modes when the assigned key is pressed.
• 📹 The VR spectator camera's position and rotation are interpolated smoothly towards the player's headset position and rotation to achieve a dampened effect.
• 🛠️ The 'Event Tick' is utilized to update the camera's transformation every frame, ensuring the camera follows the headset's movements.
• 🖥️ The tutorial addresses potential quality issues in the VR spectator view, suggesting adjustments to the texture target settings.
• 💻 The script mentions that certain graphical settings, like Lumen and indirect lighting, may cause performance issues and need further investigation.
• 🔗 Patreon supporters are offered access to the tutorial files for further experimentation and learning.

Q & A

What is the main issue discussed in the tutorial?

The main issue discussed is the jittery and shaky experience for VR spectators viewing the VR environment on their screens, especially when the VR player moves quickly or looks around rapidly.

What are the two main actors involved in the VR experience mentioned in the script?

The two main actors involved in the VR experience are the VR player character and the VR spectator camera.

What is the purpose of adjusting the pixel density in the VR headset?

Adjusting the pixel density in the VR headset can increase the frame rate by reducing the quality of the 3D model, which can help in reducing the jagged and messy appearance of the preview for the viewer.

How can the VR spectator camera be controlled to provide a smoother experience?

The VR spectator camera can be controlled by assigning a key that, when pressed, toggles the camera between a dampened (smooth) and non-dampened (jittery) state.

What is the role of the 'VR Spectator' blueprint in Unreal Engine 5?

The 'VR Spectator' blueprint in Unreal Engine 5 is used to customize the spectator's view, allowing for the setup of different ways to spectate and control the camera's behavior in the VR environment.

What is the significance of the 'Follow Damped' Boolean variable in the tutorial?

The 'Follow Damped' Boolean variable is used to toggle the camera's state between a smooth, damped follow mode and a non-damped mode, which affects how the spectator camera moves and responds to the VR player's movements.

How does the tutorial suggest to make the transition between camera states?

The tutorial suggests using a flip-flop method where pressing the assigned key once enables the damped camera, and pressing it again disables it, creating a toggle effect.

What is the purpose of using 'Get All Actors of Class' in the VR spectator blueprint?

The 'Get All Actors of Class' node is used to find the VR Pawn (player character) in the scene, which is necessary for the spectator camera to follow and interpolate its position and rotation.

How is the spectator camera's position and rotation determined in the tutorial?

The spectator camera's position and rotation are determined by interpolating its current state to the head-mounted display (HMD) data of the VR Pawn, which includes the headset's position and rotation.

What is the issue with the quality of the preview when using the VR spectator camera?

The issue with the quality of the preview is that it may appear lower in quality or laggy, especially when using certain settings like Lumen, which might be a limitation of the current setup or a need for further adjustment.

What is the suggested solution for improving the quality of the VR spectator preview?

The suggested solution is to adjust the settings of the render target associated with the VR spectator camera, such as changing the texture target to a higher quality format like 'RTF RGBA8 sRGB'.

Outlines

00:00

🎮 Enhancing VR Spectator Experience

The paragraph introduces a tutorial focused on improving the quality of life for VR spectators. The speaker discusses the issue of jittery visuals when moving quickly in VR and how it affects the viewer's experience. The tutorial aims to smooth out this experience. The speaker explains the two main components involved in VR: the VR player character and the VR spectator camera. The tutorial will focus on adjusting the VR spectator blueprint to improve the viewing experience for those watching through a screen.

05:03

🛠️ Setting Up VR Spectator Controls

This section delves into the technical setup for controlling the VR spectator camera. The speaker guides on how to access and edit the VR spectator blueprint in Unreal Engine 5. The tutorial covers assigning a key to dampen the spectator camera movement, using a brute force method to enable input for the VR spectator blueprint. The speaker demonstrates how to create a Boolean variable to toggle between dampened and non-dampened camera states, using the asterisk key as the toggle switch.

10:04

🔧 Implementing Camera Dampening

The tutorial continues with the implementation of camera dampening. The speaker explains how to use the 'Event Tick' to continuously check the camera state and adjust it accordingly. The process involves creating a branch that triggers based on the dampened state, using print statements to confirm the state change. The speaker also discusses the importance of checking which key is mapped to enable the VR spectator camera and how to test the functionality in the game environment.

15:04

📸 Smoothing Camera Movements

In this part, the speaker focuses on smoothing the camera movements to mimic the headset's movements. The tutorial involves setting the VR spectator actor's transformation to match the headset's position and rotation. The speaker demonstrates how to use 'Interpolate To' to smoothly transition the camera to the headset's location and rotation over time. The goal is to create a natural and less jarring viewing experience for spectators by making the camera follow the headset's movements smoothly.

20:06

🖥️ Adjusting Preview Quality

The final paragraph addresses the issue of adjusting the preview quality for the VR spectator view. The speaker explores different settings within the VR spectator mode and attempts to adjust the texture target to improve color data information. However, the speaker encounters limitations with certain graphics settings, particularly with Lumen, and seeks solutions or suggestions from the audience. The tutorial concludes with a call for feedback and the offer of the tutorial file to Patreon supporters.

Keywords

💡VR Spectator

The VR Spectator refers to a mode in virtual reality (VR) that allows people not wearing a VR headset to observe what the VR player sees. In the video, the speaker explains how to improve the quality of the spectator's experience by smoothing the camera movements that follow the player’s head in the game. The VR Spectator is a critical element because it bridges the gap between the immersive VR experience and the outside viewer's perspective.

💡Dampening

Dampening refers to the technique of reducing sudden or erratic movements, particularly of the camera, to create smoother motion in the spectator view. The video tutorial focuses on implementing a system where camera movements are 'dampened' to provide a better experience for VR spectators, preventing them from feeling disoriented due to the player's quick or shaky head movements.

💡VR Pawn

The VR Pawn is the virtual character or player model that represents the person wearing the VR headset. In this context, the VR Pawn controls the player's in-game movements and actions. The video mentions that although the VR Pawn itself doesn't need modification for spectator purposes, understanding its configuration helps contextualize how the spectator camera will be applied.

💡Blueprint

Blueprints are a visual scripting system used in Unreal Engine 5, allowing developers to design complex game logic without coding. In the video, the speaker refers to the 'VR spectator blueprint' and the 'VR Pawn blueprint' to explain how these blueprints control different elements of the VR experience, such as movement and camera functions.

💡Pixel Density

Pixel density determines the quality of the image displayed in the VR headset by controlling how many pixels are rendered. Lowering pixel density can increase frame rates but decrease image quality. In the video, the speaker advises adjusting the pixel density for smoother performance during VR gameplay, particularly for lower-end systems.

💡Event Begin Play

Event Begin Play is an Unreal Engine event that is triggered when a level or game starts. In the video, the speaker explains how the VR spectator camera setup begins with the 'Event Begin Play,' which initializes the functions that will manage how the spectator camera behaves throughout the gameplay.

💡Interpolation

Interpolation in this context refers to the process of smoothing out the movement between two points, such as the camera’s current position and the VR headset’s position. The video explains how to interpolate the VR spectator camera’s movements so that it gradually follows the player's head, creating a more stable and pleasant viewing experience for the spectator.

💡Flip-Flop

Flip-Flop is a control structure in Unreal Engine's Blueprint system that alternates between two states, such as 'True' or 'False.' In the video, Flip-Flop is used to toggle between the dampened and non-dampened camera movements, allowing the spectator to switch between smooth following and direct, unfiltered camera motion.

💡Tick Event

The Tick Event is a function in Unreal Engine that executes every frame of the game. In the video, the speaker describes how the Tick Event is used to continuously check and update the position of the spectator camera relative to the player's head, ensuring smooth and real-time adjustments.

💡HMD (Head-Mounted Display)

HMD stands for Head-Mounted Display, which refers to the VR headset used by the player. The video tutorial explains how to track the HMD's position and orientation to synchronize the spectator camera with the player’s head movements. The HMD is a critical part of VR because it defines the viewpoint from which the player experiences the virtual world.

Highlights

Introduction to improving the VR spectator experience by smoothing out the jittery camera movements.

Explanation of how movement in VR is natural for the player but can cause a jittery experience for spectators on screens.

Introduction to Unreal Engine 5’s default VR spectator blueprints, focusing on improving the spectator camera experience.

Explanation of the importance of controlling pixel density in VR to balance visual quality and frame rate.

Guide on adjusting the pixel density setting under the VR Pawn blueprint to improve performance.

Detailed step-by-step instructions on how to edit the VR spectator blueprint to dampen the spectator camera movement.

Use of a brute-force method to assign a key to control the spectator camera's dampening effect.

Creation of a Boolean variable 'Follow Damped' to toggle between dampened and non-dampened camera movement.

Explanation of the 'flip-flop' function to switch between smooth camera following and normal camera behavior using the assigned key.

Guide on how to connect the VR spectator camera's movement to the VR player's headset using Unreal Engine's 'T Interp To' for smooth transitions.

Demonstration of testing the new camera dampening functionality in VR spectator mode.

Overview of setting the spectator camera to follow the VR player's headset smoothly by interpolating its position.

Exploration of how to adjust render target settings for the VR spectator camera to improve visual quality, despite some limitations.

Acknowledgement of performance limitations with Unreal Engine’s Lumen and indirect illumination in the VR spectator mode.

Encouragement to experiment further with the blueprint file, which is available for Patreon supporters, for enhanced spectator experience.

Transcripts

hello hello hello so some people were

asking me to do a small tutorial on how

we can improve the experience the

quality of life for the VR Spectators

that are looking at what you're

experiencing in VR on their screens what

I mean by that is if I press play right

now and I start looking around you can

see that as long as I'm moving quite

slowly it's fine but the moment I start

really kind of looking around as I would

normally let me just jump right here

outside take a look at this you know

this structure the structure moves as

well and I'm looking at the gears at the

top and so it becomes very jittery right

and that Jitter is while for me it's

quite normal you know for for my eyes

when I have the headset on for the

viewer it's a little bit too too shaky

so in this tutorial what we're going to

do is we're going to smooth out that

experience right and before we begin you

need to understand that there are two

main actors involved in giving you the

experience into the headset through the

headset you know um generally speaking

the player

character the VR player character and

the second one is the uh VR uh how is it

called I I want to do a proper V

R um camera spectator camera something

like that uh let me just

spectator blueprint V are spectator and

those are just default blueprints that

come in with the Unreal Engine 5 VR

template right so first of

all and this is how the VR spectator

looks like and our VR Pawn the player

character is actually um not present

here in instead we're using uh start

player start blueprint or node right

here so that's the first thing that I do

is I fish them out fish both of them out

and open them up so you can find your VR

um uh

template uh sorry you can find your VR

Pawn under your VR template folder under

blueprints VR Pawn right here double

click on it

and it's going to open up the whole um

chain of logic of how the VR um

character the pawn that you control uh

how it should behave and what kind of

actions can it make and if you want to

ner out uh about it you can just kind of

read through all of these and try to get

a sense of it but in our case we don't

really uh we don't really care the only

reason why I'm showing it to you is

right here I feel like this is an

important setting um this is where right

at the begin uh event begin play

connected to that you can control the

pixel density that is produced by the

headset in into your eyes right so you

can drop down the quality like so and

thus increase the frame rate so if you

feel like the 3D model that your

previewing is too slow and it's all

Jagged and messy consider dropping down

the pixel density a little a little bit

anything below 0.6 is a little bit too

too blurry but 0.8 seems to be um

holding up just fine if you want to you

can also increase the pixel density uh

but I wouldn't recommend it it's uh it

becomes very expensive very fast anyway

so in terms of VR Pawn the character

itself we will not be doing anything

with it instead what we will be using is

VR spectator blueprint right so under VR

spectator theer here you have the

blueprint class but also if you have

started with the VR template you will

have um the VR spectator camera right

here somewhere on in in the scene right

so you can select it and then here on

the top right corner you can click on

edit VR spectator so that is going to

navigate you to the blueprint of VR

spectator

automatically then in our event graph

right here this is where we can set up

different way of how you can um how you

can spectate you know the things that

the user sees right so we will um be

creating our blueprint right here so the

first thing that we need to do here is

to assign a key that you press to dampen

The Spectator camera right and I think

okay so there are two systems of how we

can do it do it we can do it brute

forced right just straight straight up

we assign a key in this template right

here or in this blueprint right here and

we use it or we can do through the

enhanced input Action System I think for

this tutorial since we're using um since

it's focused on the dampening portion

rather than the input actions we will be

using the simplified The Brute Force way

or I should say probably the old school

way so the way we do this is first of

all well during begin play we need to

enable um the VR spectator uh blueprint

to receive inputs or to generate rather

sorry to generate inputs right so for um

where you have begin Play tab right here

the last node at the chain should be uh

enable sorry things are in the way

enable

input

there we go right target itself player

controller I think as long we we can

just keep it default it's going to uh

it's just going to work this makes it so

that uh you can press keys and keys will

do things in VR um spectator

blueprint um speaking of keys now we can

um decide on which key to um press right

so we can do

key uh is it event key key event uh

keyboard event keyboard event and you

can scroll through this and use um you

know find any key that you want uh

honestly the faster way is just to

select any key select the node here and

on the top right corner where you see

input key just click on the little

keyboard and then just press any key

that you want to use to um Drive the

change in the dam dampening versus not

dampened or soft camera versus not soft

camera right so I'll press the

asterisk I guess so now you can see

numpad asterisk is going to be the one

that changes my dampened camera and then

I

will probably create

um new um variable on the bottom left

hand side under variables click on the

little plus sign here so to create a new

variable and choose um

follow

damped follow

damped like that it should be a Boolean

um variable yeah it's either yes or no

you are either following with the damped

camera or you're not and when you

press the asterisk key once it should

set uh so you just drag in follow damped

from bottom left to the screen set

follow damped it should be tick marked

so it's going to be following but when

you press it again I will drag it out

again set follow damp it should be

unticked so it it's not um following

right so you press uh the key once it

follows um softly you press the key

again it doesn't follow uh tool that

does that is called flip flop flip

flop like that a connects to um setting

it f uh setting it to True B connects it

to setting it to

false that's it C uh select everything

press C to comment and you can write it

uh uh toggle

follow or something like that you know

just just to keep things clean okay with

that done now we

can place the following under a tick so

right next to uh begin play right under

begin play there's the event tick that

happens at every frame let's say frame

it's not really but let's say every

frame of

the game play and here under um these

two different modes either first person

mode for spec this spectation spectator

or the fly mode for spectator you can

um continue on the Chain right so what

we can do is we can create a

branch from one of these nodes doesn't

matter which one and we just connect the

second one into the branch as well so it

doesn't matter which one of these nodes

is triggered the branch is still going

to be triggered right and the condition

is it true or false false is going to be

our follow damped right so get follow

damped I just drag it from bottom left

the variable drag it follow damped bam

the condition is here right and right

now I can get I guess I can kind of show

you uh so let's say if it's true it

should print

string

um

following right

if it's false it should print

string not

following like that this is just for me

to show you how it's going to work right

so I compile I click save here I press

play right it's doing the

thing it's not working oh yeah yeah uh

cuz I need to actually click on the the

the button for for it to trigger one

second so to check what kind of uh

button you need to press to actually

enable this VR spectator camera uh you

just need to go to the VR spectator um

folder right here expand it

input find the input folder double click

on the IMC VR spectator this one right

here um the input mapping controls I

think and and here under mappings you

expand mappings you check I a VR

spectator toggle expand that and here

you can see that it's set to tab I think

tab is the default right so tab will

change

the um from from the headset display to

the VR spectator display that that is

useful information right so if I press

play now this is the VR head said being

shown but then if I press tab you can

see that's the that that's the spectator

uh camera uh which is not following and

by the way the cool thing is that I can

now um wait let's see if this will work

I can stand uh where is

it I can stand in front and I can wave

right

so before we were seeing the view

through the headset and now we were we

are seeing the view through that this

camera right and if I press the asterisk

it's not printing not following right

now but if I press it it's now following

right not following following okay so

the functionality is there it it all

works all we need to do right now is the

with the VR

spectator um function instead of using

print string we can delete this we need

to when it needs to follow

true um needs

to when this is true it need the camera

needs to be placed on the head of the

viewer and it needs to be placed

smoothly right so it needs to

interpolate between the movements of the

head smoothly so the way we do it is we

get uh get

all actors of

class and our class that we want to get

is the VR Pawn the player character

right

VR

Pawn like that VR

Pawn then uh out of all of the actors I

mean we're getting only one right

there's only one VR Pawn uh that we have

so we will get a

copy so we're getting the only one

because this is like a list we don't

need a list we need a single one so we

use get to get a single

one and we get

the

hmd I think it's called

hmd get hmd

data right head mounted display by the

way

hmd we do that hm it seems like it's

it's uh that data is a little bit off um

let's not do that sorry sorry weird

tutorial I know let's not do that

instead let's first of all uh for our

general actor for this VR uh spectator

actor let's set its translation where

where or transformation where it needs

to be and how it needs to be rotated

right so we will set

actor

transform set actor transform the target

itself right so the camera itself will

be will be moving and for transform we

will be um

getting get actor

transform so find its own

transformation and then interpolate to a

new transformation which needs to be

where the headset is at and where it's

looking at right so a tool that does

that is called T interp 2 T interpolate

to T stands for time so Delta time

right

uh rather Del Delta time is the counter

the interpolate speed is the the time

right so we want to soften it by like

let's say every second it gets an

average so it softens out so interpolate

speed is one while Delta time we just uh

get World Delta seconds that's easy get

World Delta Delta seconds right so it's

just counting uh and and interpolating

according to the count and then the okay

and the return value is our new

transform that's that's also easy the

tricky part is

the

um the target towards where so you know

again the position and the rotation of

this headset during game play so the

target needs to

be get world

uh again get World

transform

uh one

second H not like that maybe we do it

other way around so from this VR Pawn we

need to uh

get

hmd well that's hmd data but oh yeah

here right at the bottom of it of the

list get

hmd perfect get H hmd and now we get

World

transform like that and connect that to

the

Target I just want to get hmd data I

just want to check

one break hmd data

bam yeah okay so it does have position

and rotation listed here as well but in

this case um it gets the whole transform

scale as well even though the scale

doesn't matter as much um anyway so with

this setup it should work so the the way

it works is the camera uh the VR

spectator camera will set its actor

transformation so location rotation

scale um it's going to set its actor

transformation by measuring where it is

by measuring where the headset is and by

placing itself to the headset moving

itself to the headset and rotating to

the headset slowly over the course of 1

second after the second it checks again

where the headset is and it's moving

towards that that new place again right

so if I click compile save

play

hello my battery is

dead okay we can uh test this out now so

if I press

play right we're back to normal

environment but then uh if I

press

tab you can see the VR spectator camera

but now the

asterisk whoop moves us right into here

and the camera itself seems to be

previewing everything at a lower uh

lower quality that is something

that I will need to that I will need to

adjust but with that being said if I

look around and I move in here you can

see that the camera uh the spect

spectator camera does catch up quite

quite nicely right so that's that's in

principle that's that's what I wanted

and that's that's how it works right so

that's how you dampen the camera camer

in terms of the

quality um the preview quality uh let me

take a look into that and see if we can

adjust it okay in terms of quality I'm

not sure of how to fix it but what I

manage to dig out is that uh here in the

VR spectator mode for the screen capture

component 2D we have our texture Target

that should be displayed right and the

texture Target is this RT VR spectator

component to which if I browse to this

asset there's the render Target double

click on that and here we have the

settings

for the the the preview right and

basically if if I were to change it to

RTF uh

rgba

8 uh

sure srgb or yeah whatever hit save and

if I were to play this hit tab now we're

looking at it with um oh that's lagging

a little bit but we're looking at it

with uh 8bit data information or color

color data information but still if I

get in here Lumen just completely breaks

um so I guess it might be a limitation

of that I'm not sure so if anyone of you

uh out there knows how to get a

consistent throughput from uh what we

see like like that and what we see like

that that would be very much appreciated

because with that being said if I were

to go outside let me just

quickly like

that go in

here and I were to hit Tab and we follow

in you know this seems to be fine uh

let's hit tab so this is what my headset

sees and this is what is displayed in

the um VR preview or uh VR spectator by

the VR spectator camera right this is

fine it's just basically Lumen um and

indirect elimination that that that's

breaking so again if anyone knows how to

fix that portion very much appreciate it

leave a comment if you want to

experiment by yourselves with this file

I will leave it for patreon supporters

just like every other file patreon

supporters get everything we do on this

channel for free so again Link in the

video description below and uh thanks

for watching now you have a

smooth

experience I'll see you the next one

bye