I Made a Mind-Controlled Game Controller
Summary
TLDRThe script details an individual's innovative journey to develop a mind-controlled gaming experience using EEG technology and eye tracking. After overcoming numerous challenges, including technical setbacks and the limitations of thought recognition, the creator successfully builds a hands-free game controller. The project not only showcases the potential of brain-computer interfaces but also the creator's resilience and the profound appreciation for those who cannot game conventionally.
Takeaways
- ๐ง The creator has a background in Psychology and experience with EEG for medical and research purposes.
- ๐ฎ The project aimed to merge gaming with EEG technology to control games using brain activity.
- ๐๏ธ The discovery of a user-friendly EEG device was a significant enabler for the project.
- ๐ฌ The initial plan involved using EEG to record brain signals during different emotional states in games.
- ๐ The creator utilized software that came with the EEG to record and recognize different thinking patterns.
- ๐ After many attempts, the project faced challenges in getting the EEG to consistently recognize thoughts for in-game actions.
- ๐ A breakthrough came with the recognition of imagining lifting a cube, which boosted confidence in the project.
- ๐ ๏ธ The project evolved to code the EEG signals to perform in-game actions like left-clicking in Minecraft.
- ๐ The creator encountered issues with maintaining focus and eye movement while trying to control the game.
- ๐ค A shift in strategy led to considering more universal visualizations for game actions, like pushing something forward.
- ๐ฑ๏ธ The introduction of an eye tracker allowed for hands-free movement and camera control in 3D spaces.
- ๐ Despite successes, the project hit a low point with difficulties in distinguishing between different mental commands.
- ๐ A consultation with the EEG manufacturer and a team from Ear Switch provided insights into multisensory brain activation.
- ๐ Starting anew with mental command training, the creator implemented a system with virtual joysticks and buttons.
- ๐ฒ The final setup included a combination of eye tracking and four mental commands for a hands-free gaming experience.
- ๐ The project concluded with a successful demonstration of playing a game using the mind-controlled setup.
- ๐ก The creator gained a deep appreciation for the complexity of controlling thoughts and the potential of brain-computer interfaces.
Q & A
What is the main concept of the video?
-The video discusses the creator's experiment to develop a hands-free game controller using EEG and eye-tracking technology to control games with thoughts and eye movements.
What background does the creator have that inspired this project?
-The creator has a background in Psychology and experience with using EEG for medical and research purposes, which inspired the idea to combine gaming with brainwave technology.
What was the initial plan for using EEG in the project?
-The initial plan was to use EEG to record the brain's electrical signals to analyze the creator's brain activity during different gaming experiences, such as getting jump scared or being relaxed.
How did the creator find out about user-friendly EEG devices?
-The creator discovered user-friendly EEG devices while browsing the internet, which was a significant revelation that excited them about the potential applications of this technology.
What software feature of the EEG device did the creator utilize to recognize brain activity?
-The creator used the EEG's software that allows recording of brain activity while thinking about different things, which helped in recognizing specific thoughts for controlling the game.
What game was chosen to test the mind-controlled gaming concept?
-The creator chose Minecraft as the game to test the initial concept of mind-controlled gaming due to its simplicity and familiarity.
What was the main challenge faced when trying to control the game with thoughts alone?
-The main challenge was maintaining focus and not losing the mental image needed to control the game, as the eyes would drift and the creator couldn't hold the image long enough for continuous control.
How did the creator address the issue of needing more commands for a more universal gaming experience?
-The creator decided to find a universal visualization that could be applied to different games, such as imagining pushing or moving something forward, which was easier to maintain and less distracting.
What additional technology was integrated to enhance the mind-controlled game controller?
-An eye tracker was integrated to create virtual joysticks on an Xbox controller, allowing the computer to know where the creator was looking and the direction of head tilt for camera or character movement.
What was the final outcome of the project after overcoming various challenges?
-The final outcome was a hands-free mind-controlled game controller that included virtual joysticks and four mental commands for virtual buttons, making it a reliable and functional gaming tool.
What personal realization did the creator have after completing this project?
-The creator realized the difficulty of controlling thoughts and emotions and gained a deep appreciation for those who are unable to game conventionally and the challenges they face daily.
Outlines
๐ง Mind-Powered Gaming Experiment
The script introduces a personal journey of combining gaming with psychology and EEG technology. The creator, with a background in Psychology, explores the possibility of using brainwave data from an EEG device to control in-game actions. Initially, the plan was to record brain activity during different emotional states, like fear or relaxation, but evolved into an attempt to control a game using thoughts alone. The creator leverages pre-existing software to recognize different mental commands, such as imagining actions like dropping something or turning directions, to manipulate an in-game cube. Despite initial failures, persistence leads to a breakthrough, enabling the lifting of the cube through thought alone, which then translates into in-game actions like left-clicking in Minecraft.
๐ฎ Advancing Mind-Controlled Gaming
Building upon the initial success, the creator aims to refine the mind-controlled gaming experience, facing challenges with maintaining focus and expanding the range of controllable actions. The script details the experimentation with different visualizations to improve control and the integration of an eye tracker to create virtual joysticks for camera movement. Despite setbacks, including technical issues with the EEG device, the creator remains committed, investing personal funds to continue development. The narrative emphasizes the complexity of distinguishing between similar brainwave patterns for different commands and the innovative approach of using chaos in visualization to create unique mental commands. The culmination of these efforts results in a more advanced mind-controlled game controller capable of basic in-game interactions.
๐ Finalizing the Mind-Controlled Game Controller
The final paragraph describes the culmination of the creator's efforts to develop a hands-free, mind-controlled game controller. After a year of trials, errors, and learning, the creator incorporates four distinct mental commands and an eye tracker for a more comprehensive gaming experience. The journey highlights the difficulty of differentiating between complex thought patterns and the realization that incorporating various sensory inputs could enhance the control mechanism. The creator's collaboration with experts and the integration of these insights lead to a functional prototype that is tested in real-time with an audience. The successful demonstration of the controller in a game environment signifies not only a personal triumph but also a step forward in accessible gaming technology.
Mindmap
Keywords
๐กEEG
๐กPsychology
๐กGaming
๐กMind Control
๐กVisualization
๐กMental Commands
๐กEye Tracker
๐กController
๐กProgramming
๐กInnovation
๐กAccessibility
Highlights
Combining gaming with EEG technology to control games using brain signals.
Using a user-friendly EEG device to record brain activity for gaming purposes.
Initial challenges in getting the EEG to recognize specific thought patterns for game control.
Concept of proof achieved with the ability to lift a virtual cube using thoughts.
Coding the EEG data to translate thoughts into in-game actions, like a left mouse click.
The idea of using a universal visualization for different gaming actions.
Facing difficulties with maintaining focus and consistency in thought recognition.
Incorporating an eye tracker to create virtual joysticks for camera and character control.
Overcoming sensitivity issues with the eye tracker to improve control.
Creating a hands-free mind-controlled game controller with multiple commands.
The struggle with expanding the number of mental commands for more complex gaming.
Experimenting with non-visual mental commands to diversify brain activity recognition.
Collaboration with 'ear switch' for insights on different sensory inputs affecting brain activity.
Re-training mental commands from scratch to improve the usability of the game controller.
Integrating a variety of mental commands for a more functional gaming experience.
Successfully using the mind-controlled game controller in actual gameplay.
The personal realization of the difficulty in controlling thoughts and emotions.
Appreciation for those who cannot game conventionally and the potential of advanced tech.
The limitless possibilities of what can be achieved with the human brain and technology.
Transcripts
this is me playing games with my mind
after I developed a way to do this using
only my
thoughts I like
you okay
[Music]
stop and I'm going to show you how like
a lot of people I love gaming and since
I have a background in Psychology and
using EEG for medical and research
purposes I thought it might be a cool
experiment to try and combine the two at
the time I didn't record or make notes
on anything so there'll be a lot of
dramatic reenactments coming up my
original plan was to use EEG which
records the electrical signals your
brain produces to show what my brain was
doing when getting jump scared in a
horror for
example okay or when I was super relaxed
in a horror for example this was only
possible because one day when I was just
scrolling through the internet I
discovered that you could just buy a
userfriendly
EEG which already was mind-blowing
enough for me to get really excited
about all the things I could do with
this but I wanted to take it a step
further and see if I could use the data
the EEG was getting to actually control
the game so my plan was to use this data
in a program that would send a signal to
press keys on my keyboard but first I
needed to decide what data what part of
my brain activity would press each key
and I needed to teach it to detect and
recognize this so I could actually
control it but I was lucky enough that a
lot of this work had already been done
for me this EEG has a software that lets
you record your brain activity when
you're thinking about different things
this could not be more perfect and it's
such a better idea than me trying to
program my own AI model like I was
originally planning so I started
recording attempt after attempt at
trying to get it to recognize me
thinking about dropping something or
turning left or turning right or
spinning or dangling from a string if it
does recognize it this Cube will move
but nothing no matter what I tried I
couldn't get it to recognize my thoughts
until
[Applause]
that's a good start we have proof of
concept with this confidence boost I was
now more committed than ever and soon I
was able to consistently lift the cube
into the air by just imagining that so
the next thing to do was to code this so
that every time I was able to push the
cube it would push a button to do
something in game I thought the easiest
thing to do was a left click and
naturally when you think game you think
Minecraft
so it wasn't perfect but I was more
convinced than ever that I was on to
something the problem was that as I was
imagining lifting the cube my eyes would
drift upwards and I'd lose focus I just
couldn't hold on to the image for long
enough to mine an entire block so back
to the drawing board I started thinking
about what I actually wanted it to do
just thinking about mining would take a
lot of practice to get the same every
single time and I wanted a more un
Universal visualization so that I could
use it with different games but what did
mining have in common with hitting a
tree or casting a spell or even shooting
a gun they all involv moving or pushing
something forward this worked amazingly
it was easy to maintain and I didn't get
distracted as I wouldn't have to move my
eyes like this soon we had a mouse click
that was easy to control most of the
time so naturally it was time to take
this live and use it to play elen Ring y
alen ring why not I was so excited to
show everyone this insanely cool project
I had spent 3 months of hard work done
long nights and early mornings and blown
through my entire cool project savings
fund I was just about to go live
when it's a ribbon cable can we fix
it no but I wasn't going to give up
after a moment to grief I spent my
entire food and shelter fund and ordered
another
EEG this time with insurance and got
back to
work okay we're ready this was my second
and final shot of making this work if I
blew it I knew I couldn't afford to try
it
again somewhere to hide behind baby
no spurred on I instantly got started on
getting another visualization and
another button to work but when I I
trained one command the software would
get confused and forget the other one it
couldn't hold on to both simultaneously
what my brain was doing was just too
similar so to make the visualizations
different I tried to make the second
command as chaotic as possible imagining
music playing in the cube spinning far
so much it became more of a plate in my
mind and to my surprise this actually
worked one thing that always bothered me
about my mind control game controller
was the fact that I was still using my
hands and a literal controller the
reason for this was because there was no
way to fluidly move a camera or
character in a 3D space when I was just
using yes no thought recognition so I
thought if hands were the problem why
not try and use some other Tech this is
an ey tracker using this the computer
knows where I'm looking at on the screen
and also which direction my head is
tilting I used these two things to make
virtual joysticks on an Xbox controller
and created a Nifty little visual to see
where they were it worked but I I did
have to fix some sensitivity issues
and counteract the dead zones created
because of just the nature of using
joysticks now the only thing left to do
was combine them ladies and gentlemen we
have a handsfree mind controlled game
controller I don't know about you though
but I don't know many games where you
can play them with only one or two
buttons sure you could probably get away
with a Dark Souls boss or two if you
never healed or you can play a shooter
if you don't use grenades or mie or need
to switch which weapon ever or use any
sort of menu but that's not gaming I
needed more commands but no matter what
I did what I tried I just couldn't get
any it can't tell the difference between
my thoughts even if I try to imagine
something that I think is completely
different to anything else that I
trained before it will still get all
mushed together I toyed around with the
idea of using voice commands to fill in
any more buttons I really needed it does
mean though that when we go into the
menu
we don't have the ability to move this
way but this felt wrong it wasn't mind
control without more commands though
this would never be a game controller
and everything I had done felt all for
nothing more long nights and early
mornings desperate to come up with
something and nothing along the journey
this was probably my lowest point I just
can't make my brain do what I want it to
do and usually it's the tech or usually
it's the code but this time it's just me
which somehow feels a lot
worse I still had my phone a friend it
was time to reach out I'm about to hop
on with a motive the maker of the EEG
and software that I'm using but it's all
very NDA so you got to get out of
here after a lot of cool stuff the main
thing that I took from our discussion
was how much different sensors can
actually affect different parts of your
brain a visual idea is going to awaken
different areas than a smell for example
and all all of my commands had been
visual there was a whole world of
possibilities that I was just ignoring I
also spent a lot of time with a team
down at ear switch who are making an
inear gaming input device which got me
thinking about how I could use my head
in different ways so I decided to wipe
the whole slate clean and start training
mental commands from scratch to see how
good I could make this could it be a
usable game controller so here it was
the final plan I tring and dryer for
virtual joystick and four mental
commands for virtual buttons pushing a
cube spinning a plate imagining my fist
clenching while angry and feeling hot
and a little Cricket hopping while I
pulse muscles in my face I also added a
cute little drop- down menu for each of
these so I could make them press any
button I wanted at any time add an
option to toggle to spam or hold the
buttons and some sensitivity sliders for
the eye tracker and she was ready it's
ready to test and if this doesn't work
then I think I'm genuinely out of ideas
I might have to actually accept that I
just don't have the level of technology
or knowledge on how to make this work
right now which would be Soul crushing
after an entire year and hundreds of
hours of just training let alone
everything else and I would have to
break it to all of the really supportive
people that have been following this
journey and the people who are watching
it unfold live with a couple hundred
onlookers I started training I managed
to get a third input and then a
fourth but it's one thing to be able to
move a cube on a white screen could I
use this to actually game pick up this
[Music]
Rock I'm going to hit this
rock great I'm going to
jump and I'm going to try to throw a pal
but I don't have
any mind control and the final
[Music]
Test passing
[Music]
skin
[Music]
yeah
[Music]
so there it is I made a handsfree game
[Music]
[Music]
controller that's a reliable weapon
[Applause]
yeah this experience really taught me
how sometimes we can be so caught up in
our day-to-day lives that we don't
actually appreciate how hard it is to
really control our thoughts and our
emotions and even our Focus the amount
of appreciation that I gained for those
who are unable to game conventionally
and have probably struggled every single
day to fit in was immense since this
Tech is becoming more and more advanced
I feel so lucky to have been right in
the middle of its emergence and I can't
wait for it to become more accessible
and available we take all this for
granted but our brains are amazing and
the things we can do with them are
[Music]
Limitless
Browse More Related Video
5.0 / 5 (0 votes)