Microsoft's New AI AGENT Computer SURPRISES Everyone! - New AI CO-Pilot PC
Summary
TLDRMicrosoft has unveiled its groundbreaking Co-Pilot AI features set to revolutionize computer interaction. The highlight is 'Javis', an AI system akin to Iron Man's, offering real-time gaming assistance. Additionally, 'Recoil' introduces an unprecedented search capability, acting as a PC's unlimited memory. 'Co-Creator' leverages AI to enhance creative tasks, while the new Surface Pro integrates these features with an NPU for efficient AI processing. Live captions and translations are also introduced, breaking language barriers for global communication.
Takeaways
- 🚀 Microsoft showcased a new product called 'Co-Pilot', which is an AI system that could revolutionize computer usage.
- 🎮 Co-Pilot Plus Games is a feature that acts like J.A.R.V.I.S. from Iron Man, providing real-time feedback to assist users, even in activities like playing Minecraft.
- 👀 The demo included a scenario where Co-Pilot helps a user craft a sword in Minecraft, including advice on handling in-game threats like zombies.
- 👴 The technology could be particularly useful for teaching children or assisting the elderly with technology and tasks.
- 🔍 'Recall' is a feature that uses AI to help users find anything they've seen or done on their PC, leveraging the power of the MPU for fast, private searches.
- 💡 'Recall' can understand semantic associations, such as recognizing 'peacock' as a blue hue, to help find items like a blue dress discussed in a Discord chat.
- 🎨 'Co-Creator' is a feature that allows users to generate images based on text prompts and their own ink strokes, with AI enhancing the original drawing.
- 🖌️ The 'Co-Creator' feature includes a creativity slider, enabling users to balance between their original drawing and AI modifications.
- 💻 The new Surface Pro is equipped with an NPU (Neural Processing Unit) that works alongside AI models in the cloud for enhanced performance and efficiency.
- 🌐 Live captions and translations are now possible in real-time on any audio passing through the PC, thanks to the NPU, breaking language barriers for global communication.
Q & A
What is the main focus of the video script?
-The video script focuses on Microsoft's new product demonstrations, particularly the co-pilot AI systems that are designed to change how we use computers.
What is the concept of 'co-pilot plus games' or 'co-pilot plus computer'?
-The 'co-pilot plus games' or 'co-pilot plus computer' concept is a virtual assistant system similar to Javis from Iron Man, which is integrated with various applications and provides real-time feedback and assistance.
How does the co-pilot system assist a user in a game like Minecraft?
-The co-pilot system assists by providing real-time guidance, such as helping a user find materials to craft a sword and advising on how to escape from a zombie attack in the game.
What is the potential use case for co-pilot AI for elderly users?
-The co-pilot AI can help elderly users learn how to play games or navigate technology, such as filling out forms, by providing interactive guidance and assistance.
What is the 'recall' feature and how does it work?
-The 'recall' feature is a capability developed by Microsoft that allows users to find anything they have ever seen or done on their PC by leveraging the power of the AI to maintain content on the device for fast access and privacy.
How does 'recall' differ from traditional search functions?
-Unlike traditional search functions, 'recall' uses natural language processing to understand and find items based on descriptions and context, even if the exact words are not used in the search query.
What is the 'co-creator' feature and how does it enhance creativity?
-The 'co-creator' feature is an AI-powered tool that allows users to generate images based on text prompts and their own ink strokes. It uses machine learning models to refine and enhance the user's original drawing, providing a co-creative experience.
What is the significance of the 'Surface Pro' device mentioned in the script?
-The new Surface Pro device is highlighted as the fastest and most intelligent Surface Pro ever, featuring an npu (neural processing unit) that works with AI models in the cloud to deliver enhanced performance and efficiency for AI tasks.
How does the 'live captions' feature work and what is its benefit?
-The 'live captions' feature turns any audio that passes through the PC into real-time captions on the screen, making content more accessible. It also offers live translations, breaking down language barriers for global communication.
What is the role of the npu in the new Microsoft device?
-The npu in the new Microsoft device is designed to handle specific AI tasks directly on the device, such as image processing, voice recognition, and local data indexing, providing low latency operations and freeing up CPU and GPU resources for other tasks.
Outlines
🎮 Microsoft Co-Pilot: AI Assistant for Gaming and More
The script introduces Microsoft's Co-Pilot, an AI system that revolutionizes computer use, particularly in gaming. It demonstrates Co-Pilot's capabilities through a Minecraft gameplay scenario where a player receives real-time assistance in crafting a sword and evading a zombie attack. The AI's intuitive interaction with the game and the player is highlighted, suggesting a future where AI can assist with various tasks, from gaming to helping the elderly navigate technology.
🔍 Introducing Recall: Unlimited Memory for Your PC
This section presents 'Recall,' a new feature that uses AI to help users find anything they've seen or done on their PC, leveraging the power of the mpu to maintain speed and privacy. The product leader, Carolina Hernandez, showcases Recall's ability to search for specific items, like a blue dress, and to retrieve information from Discord chats. The feature's semantic association capabilities and its potential to streamline work processes by saving time on searching for documents and content are emphasized.
🎨 Co-Creator: AI-Powered Art Creation Tool
The script discusses 'Co-Creator,' an AI-powered tool that enhances users' creative tasks by generating images based on text prompts and ink strokes. It allows users to guide the AI's level of creativity through a slider, blending the original drawing with AI enhancements. The tool's potential to democratize art creation and improve productivity by bridging skill gaps is highlighted, showing how it can transform basic strokes into detailed artwork.
💻 Surface Pro with AI: The Fastest and Most Intelligent Yet
The script introduces a new Surface Pro device, claimed to be the fastest and most intelligent ever, featuring an npu that works alongside AI models in the cloud for enhanced performance and efficiency. The npu handles AI tasks like image processing, voice recognition, and local data indexing with low latency. The device's ability to offload complex tasks to the cloud while keeping CPU and GPU resources free for other tasks is demonstrated through the 'Magic Mask' feature, which allows for complex color corrections without slowing down the PC.
🌐 Live Captions: Real-Time Translations for a Global Community
The final paragraph showcases 'Live Captions,' a feature that provides real-time translations for any audio on a PC, making content more accessible. The feature's ability to switch between languages instantaneously is demonstrated with a video call in Spanish, Mandarin, and Russian. The script suggests that this feature will break down international barriers and enable more meaningful global connections, hinting at future improvements including real-time voice translations.
Mindmap
Keywords
💡Co-Pilot
💡AI System
💡Minecraft
💡Crafting Table
💡NPU (Neural Processing Unit)
💡Recall
💡Semantic Association
💡Co-Creator
💡Live Captions
💡Surface Pro
Highlights
Microsoft has demoed a new product, a co-pilot AI that is set to revolutionize computer usage.
The co-pilot AI is compared to J.A.R.V.I.S. from Iron Man, offering real-time feedback and assistance in applications such as gaming.
In a demo, co-pilot assists a user playing Minecraft, providing crafting tips and helping to evade in-game threats like zombies.
Co-pilot has potential applications beyond gaming, such as aiding children or the elderly in learning to use technology.
The AI's capabilities extend to assisting with computer tasks, suggesting a broad range of use cases beyond traditional AI chat interfaces.
Recoil is a new feature with seemingly unlimited memory, allowing users to find anything they have ever seen or done on their PC.
Product leader Karolina Hernandez introduces Recall, which leverages the CPU to keep content on the device for fast, private experiences.
Recall helps users find specific items, such as a blue dress, by searching through various online platforms and chat histories.
Recall's semantic association capabilities allow it to understand related terms and find items even when the exact phrase isn't used in the search.
Co-creator is a feature that uses AI to enhance creative tasks, allowing users to generate images based on text prompts and their own drawings.
Co-creator's machine learning models run locally to optimize output quality and responsiveness, creating a truly co-creative experience.
The AI in co-creator can be adjusted via a creativity slider, giving users control over how much the AI modifies their original work.
The new Surface Pro is claimed to be the fastest and most intelligent yet, with an NPU that handles specific AI tasks for enhanced performance.
Live captions is a new feature that provides real-time translations, breaking down language barriers and making content more accessible.
The live captions feature works across all apps and can translate from over 40 languages into English, with more languages to be added.
The seamless integration between the NPU and the cloud allows for efficient AI tasks, reducing the load on the CPU and GPU.
The Magic Mask feature allows for complex color corrections to be made quickly and simply with AI, enhancing productivity in video editing.
These innovations are expected to greatly increase productivity and change the way people interact with technology, bringing AI capabilities to a wider audience.
Transcripts
so Microsoft literally just demoed one
of their amazing new products their
co-pilot aips that are literally going
to change how we use computers and if
you don't believe me please watch the
video Until the End before you make a
comment because what I saw is truly
stunning so let's take a look at the
trailer and then I'm going to show you
guys some of the unique features that
this really does offer
[Music]
[Music]
n
[Music]
[Music]
so that was a really quick overview of
some of the things that are going to be
coming but this video is going to show
you guys a few of those things in a lot
more detail so one of the things and
this is where I say I truly do believe
that things are about to change is the
co-pilot plus games or co-pilot plus
computer so this is truly science
fiction because this is essentially
Javis so if you don't know what Javis is
basically in Iron Man there was an AI
system that was connected to pretty much
every application and it was able to see
things and be able to be your virtual
assistant that was really really good so
basically what we're about to see here
is a demo of co-pilot helping someone
who is playing a game they're playing
Minecraft and they're literally able to
in real time get feedback on what they
should be doing I think it is really
really futuristic but take a look at the
demo cuz it's pretty crazy hey co-pilot
how's it going hey I'm doing great and
it looks like you're about to dive into
some Minecraft ready to build explore
and maybe dodge a few moms yeah I'm
trying to figure out how to make a sword
my son plays this game all the time and
I have no idea what I'm doing no worries
I'm here to help you impress your son I
see you've got a crafting table in front
of you which is a great start to craft a
sword you'll need some materials can you
open your inventory just press e on your
keyboard yeah uh do I have what I need
let's see you've got some sticks perfect
for the swords handle but it looks like
you're missing the material for the
blade you can use wood Stone iron gold
or Diamond let's start by gathering some
wood or stone shall we head out and find
some resources yeah that sounds great oh
my God what's that oh no it's a zombie
run you need to get away from it fast
either Sprint away or quickly build a
tower of blocks straight up to get out
of its reach if you can find a shelter
or dig into the side of a hill do it now
just make sure it can't reach you all
right I think I might be safe now that
was a close one great job finding
shelter yeah do I have what I need let's
see you've got some sticks perfect for
the swords handle but it looks like
you're missing the material for the
blade you can use wood Stone iron gold
or Diamond let's start by gathering some
wood or stone shall we head out and find
some resources yeah that sounds great oh
my God what's that oh no it's a zombie
run you need to get away from it fast
either Sprint away or quickly build a
tower of blocks straight so yeah I think
this demo is what surprised me most
because surprisingly no one's built this
just yet and I think this is a product
that we could all use I mean you know I
don't think I need help in certain video
games but I think with regards to
certain computer tasks it actually might
be pretty useful so for those of you who
are pretty tech-savvy you might think
okay why would I ever even use this what
is the point of this but think about it
like this for example you've got a kid
who wants to learn how to play a game or
you've got someone elderly who's trying
to get to grips with a certain maybe a
game a certain piece of technology a
form that they're trying to fill out
this kind of you know I guess you could
say application has a lot more use cases
than the standard generative AI chat
interface so imagine an elderly person
trying to correctly fill out a form and
you know you can talk back and forward
with your system asking it which you
know things I should click on and in
some cases in the future what will
happen and this is what my prediction is
for the future of things and there's
going to be some more cool Demos in a
moment but the AI is going to be able to
do some of those moves for you so you're
going to be able to be like okay uh I
need a paragraph on this and it's going
to be able to write that you're going to
be able to talk with it so I think this
just shows us a small glimpse into the
future of what's to come now there was
also something really really cool that I
think has a lot of different
applications whilst yes I think this
Javis feature is so cool for video games
and just a million different things like
for example I mean you know you could
literally be like editing a project and
you know you could play back the video
see how well it is you could be you know
designing a poster I mean there's just a
million different things you could do
but I think this new feature here this
is called recoil and its ability to have
basically unlimited memory so I want you
guys to look at this cuz this is pretty
pretty crazy and this entire video is
going to show you guys pretty much the
first time that I think we're going to
get that huge technological innovation
that truly changes the way we use
computers to show you recall I'd like to
welcome our product leader on Windows
who built this incredible new capability
karolina Hernandez this welcome
[Applause]
Carolina thanks Carolina give us a
overview of recall please yeah
absolutely I would love to uh my team
developed recall to help you find
anything you have ever seen or done on
your PC we Leverage The Power of the mpu
to ensure content stays on your device
which makes the experience fast and
maintains your privacy okay let me show
you a few ways that recall helps me like
all of you I am constantly jumping
between different task throughout my day
managing my time between a career I am
passionate about staying connected with
my family and friends around the world
and the hobbies that I love Reco helps
me get through all these tasks more
quickly my family and I have been
searching for the perfect dress for my
grandmother who lives in Colombia to
wear at my cousin's wedding I've been
browsing Pinterest in shopping websites
for some options that I can share with
her after looking for options for a few
weeks I wanted to go back and see some
of the favorite blue dresses I have
found but I didn't keep all my browser
tops open with recall I can just search
for blue dress and it pulls together all
the Dres I have seen now Grandma decided
that she wanted to get the awesome pamu
that we shared in our Discord chat about
3 weeks ago now scrolling through
multiple weeks of chat messages to find
that specific D would have taken an
eternity instead with recall it was easy
I refined my search using the script of
language blue pan suit with a sequin
lace from from Abita and as you can see
recall quickly found the Discord chat my
family and I were using to discuss the
options and here it is the dress that
aita liked recall analyzes the snapshot
and it gives me options for what I can
do next in this case I want to click the
link which takes me back to the right
web page where I can make the purchase
and now Grandma has a gorgeous outfit
for the wedding this is a great example
Carolina that I think is going to show
how recall is so helpful for people but
one thing I'm not sure people noticed is
you used the word peacock but it wasn't
anywhere in what you were searching for
how did recall recognize that and what's
going on behind the covers yeah
absolutely so recall can understand that
peacock is in the family of the blue
hues so it creates that semantic
Association that is used to help me find
the dress recall is also essential for
me at work like many of you my team and
I spend a ton of time collaborating on
presentations and documents for example
over the past few weeks we have been
collaborating on a presentation for a
product the deck was created by my
partners in the marketing team and when
I finally got around to add my content I
could not figure out where to look for
the deck I tried to remember was a deck
shared in the call maybe I had seen it
in an email or maybe I could go and find
it in my recent files in PowerPoint I
don't really have time to spend clicking
around but with recall I no longer have
to go on a searching Expedition I
remembered in the outline we had added a
purple chart with writing a purple WR a
chart with purple writing uh and to find
the PowerPoint all I had to do was
simply use my voice to search for that
clue that I remember in in a few seconds
Reco found the deck for me and notice
that it took me to the exact slide that
I remember saving me even more time now
to put the finishing touches I wanted to
add the final marketing line but
couldn't remember the exact words I did
remember that Nikki shared it in the
planning meeting on teams last Tuesday
with timeline and recall I was able to
scroll back in time and find the
snapshot of that meeting I was then able
to quickly copy that um quote and add it
to the deck and then once again recall
made it easier for me to find the
information just using the clues that I
remember now these are just two of the
examples of how I use recall on a daily
basis so I thought that was rather
impressive because once again this opens
up a huge huge amount of opportunities
because with recall it's basically like
having unlimited memory for your
computer I'm sure we literally all do a
million different things on our computer
and sometimes we often do forget things
and what this allows us to do is this
literally allows us to recall anything
we've ever done on our computers and I
think I think that what we have here is
something that's truly truly impressive
and pretty gamechanging because you can
imagine a scenario where you know you
need to find an important email you need
to find an important document that
you're working on or you need to find
you know something like an article that
you're viewing and you you really
enjoyed it you're literally able to go
back view that and immediately get that
information and you can see here I think
one of the most impressive things about
this wasn't the fact that this was some
kind of basic search history was the
fact that you could literally in natural
language describe what you're looking
for like a blue pants suit with you know
sequin lace from Abita and then you
could immediately find what you're
talking about so the visual matches and
the text matches are really really
important and really really key I think
this is going to be a really unique
feature because it's integrated directly
into your PC so it allows you to just
literally remember anything that you've
done and I think another thing that
would be cool in the future is that you
know maybe you could say okay what were
some of the things I worked on last week
recall could be in the future I'm not
sure if this is going to be a feature
just yet but they could allow it to you
know say okay you worked on this you
worked on that you know you could have
it for you know your timee keeping in
terms of how much you use certain apps I
mean it's something that is I think
opens up like I said whole whole range
of different possibilities like you
could say based on the work that I've
done previously analyze the footage and
see where I'm lacking uh you know maybe
the AI is like maybe have you tried
using this program in your workflow you
could be a lot faster I mean there is
just a lot that I think is going to come
from this that a lot of people don't
realize I think this is just the first
step and I think with these co-pilot PCS
we're going to see a lot more AI
integration that is truly going to
change the game now there was also
another integration that I did want to
show you guys cuz this one I think was
really really really cool too and this
is the co-creator now most of you may
have actually seen something similar to
this because there do exist some online
programs that are quite similar to this
but I think that now this is going to be
integrated into the Surface Pro natively
this is going to change a lot of
people's workflows and make them a lot
quicker today we're releasing a brand
new feature called co-creator that will
allow you to explore the full range of
your imagination without sacrificing the
Integrity or intent of your original
drawing let me show you how fun this is
so this experience is built on diffusion
based algorithms meaning we start with
the random image and go through a series
of denoising or diffusion as that random
image tries to match up with our text
prompt in this case I'm going to use
mountainous Valley of wild
flowers since spring is in full bloom
now the magical thing about this
experience is that the generated output
is not only taking guidance from my text
prompt but also my ink
Strokes so notice how these
two blue blobs have suddenly become a
more refined
Valley now there are five different
machine learning models running locally
in tandem to optimize for highest
quality output over minimum number of
inference steps making the experience
feel truly co-creative the generated
output responding to every single stroke
so this now I actually really love what
we have going on here but I can also
play with how much I want AI to modify
my output so with this creativity slider
the farther I slide to the left the more
closely resemble my original
drawing the farther I slide to the right
the more freedom or creativity I allow
AI to play with now once I found the
perfect balance here I can take this a
step further and actually continue
iterating on top top of my
drawing so I think a monarch butterfly
hovering over this far right
flower would be the perfect
ending and there we go so I think this
actually shows us how AI can be used to
increase the productivity of us on our
creative tasks and whilst yes a lot of
people saying that this is replacing
people I still think that many AI
systems currently lack the I guess you
could say adhering to the actual text
prompts this is something that I find
you know many AI systems struggle with
so for those of you who just want to
create something and for those of you
who aren't you know artistically
talented like myself this is going to be
something that provides you the ability
to create decent art from just basic
strokes and I think something like this
is important because the overarching
trend what we've seen in this industry
is that AI is something that Bridges the
gap between the different skill classes
so this is something that is really
really really impressive uh but it's not
something I hadn't seen before so
co-creator seems to be something that is
really impressive and something that we
will likely see in the future in terms
of how different skill levels are all
being B towards the same level and I'm
kind of excited for this because I think
with this kind of Technology now being
put natively into computers this is
actually going to bring in another wave
of people who previously didn't know
about AI Technologies because whilst yes
if you are someone that's watching this
channel you're likely someone who's
pretty much into AI but there are a lot
of people who just go about their day
today lives and really don't see the
impact of AI that it's going to have in
their lives whether it be generative
narrow and it's important to you know
kind of bring in that second huge wave
of people that are then going to be
surprised by what AI can do now here's
where we have the other announcement for
Microsoft this is their new device it's
another new device and this is their
Surface Pro which they claim to be the
fastest most intelligent Surface Pro
ever and this is really really
impressive because it has a few things
that are going to make this AI on the
device a lot faster so essentially they
have an npu in co-pilot and they work in
conjunction with the AI models running
in the cloud to deliver enhanced
performance and efficiency so basically
how it works and I was actually confused
but the npu is designed to handle
specific AI tasks directly on the device
and this includes real-time operations
that benefit from low latency such as
image processing voice recognition and
local data indexing and you know the
things that are going to be processed by
the npu are things like recall which
allows you to quickly access previously
viewed content co-creator which is where
you can generate images on device and of
course live captions where you can
translate audio in real time and of
course the more complex AI tasks that
require substantial computational power
and data can be offloaded to the cloud
and this actually leverages Microsoft's
AER Cloud infrastructure which hosts D
llms like GPT 4 and allows you to do a
lot of stuff so essentially what we have
here is we have a seamless integration
between the npu and the cloud which
allows for pretty efficient work which
is pretty different to what we have now
and I think this magic mask feature on
my laptop so I can select and make color
complex make complex color corrections
to any object in a single frame and
render that across the whole clip so
it's easy as this I'll select the key
frame and with just a single swipe I'm
able to detect and mass that jellyfish
there and if I play this forward magic
mask is going to attract the jellyfish
through the whole clip so let's
now I'm not a professional filmmaker
myself of course but I've been playing
around with this magic mask feature on
my laptop so I can select and make color
complex make complex color corrections
to any object in a single frame and
render that across the whole clip so
it's easy as this I'll select a key
frame and with just a single swipe I'm
able to detect and mass that jelly fish
there and if I play this forward magic
mask is going to attract the jellyfish
through the whole clip
so let's take a look at the test manager
here you'll notice the mpu is enabled
magic mask is offloaded and my CPU yep
my CPU and GPU resources are almost
completely available for me to work on
any other task without slowing down my
PC and this is running really fast I'm
really impressed with this um let's wait
and see as it finishes and so what I'm
going to do with this is I'm going to
make a quick color Ed it um we're going
to change the color of this jellyfish
and really see what we're able to do
here with the power of this mpu and hit
play and there it is complex editing
made quick and simple with
[Applause]
AI we're so excited about the potential
of copilot plus PCS to give time back to
all creators so they can focus on what
they do best thank
you thanks oen it's amazing to see that
level of performance from our App
Partners across the CPU and the
mpu another new experience that's
powered by the mpu and available across
all co-pilot plus species is live
captions now with live
translations imagine if we could talk to
our family our friends and our
colleagues or enjoy content from across
the world in any language with this new
feature live live captions will turn any
audio that passes through your PC into a
single consistent captions experience in
real time on your screen it's incredible
and makes content more accessible to
more people video calling has become
such a huge part of our Lives both at
home and at work let's watch how live
captions can help Global teams
communicate more naturally
har
[Applause]
for as you saw the captions just
switched automatically and instantly
across three different languages Spanish
Mandarin and Russian this was never
possible before and since it's a Windows
feature it's available across all your
apps consistently any video calling app
any Entertainment app translated locally
on the npu it translates from over 40
languages into English today and we'll
be adding more output languages over
time with the impressive translate as
they talk feature I think this is going
to be something that is revolutionary
for the ability to break down barriers
internationally and to open up a vast
ecosystem of different creat being able
to connect with each other in a way that
is a lot more meaningful now I think
this is going to be something that is of
course improved in the future with
possibly even you know actual realtime
voice translations as well like with
audio but for now I think the live
captions feature is something that is
really really effective and something
that I would love to see on my device
and I think also this short demo here
was very very effective because it
showcased exactly what we could do when
we have different parts of the computer
that are special list for different
tasks and this is going to allow a huge
huge huge increase in terms of
productivity because I mean being able
to literally you know do a very very
intensive AI related task on your
computer without burdening the CPU or
GPU is is really really impressive and
it makes me actually want to get one of
these and I guess that now when we're
moving to different you know
architectures in terms of Hardware I
think that this is going to truly truly
change the game and people are going to
really start to realize the value for a
I okay come on come on
Ver más vídeos relacionados
Google I/O 2024: Everything Revealed in 12 Minutes
Why OpenAI's Announcement Was A Bigger Deal Than People Think
Galaxy S24 Ultra: Official Introduction Film | Samsung
OpenAI's New Model Releases LEAKED | Sam Altman talks about AGI, UBI, GPT-5 and what Agents will be
Microsoft vs. Apple: Satya Nadella Says AI-Focused Copilot+ PCs Beat Macs | WSJ
Microsoft COPILOT - Your New AI Best Friend
5.0 / 5 (0 votes)