Meta Connect 2024: Everything Revealed in 12 Minutes

Summary

Meta Quest 3S和Quest 3是最新推出的高质量混合现实设备，提供高分辨率彩色混合现实和自然手势跟踪。Quest 3系列支持从2D应用到PC远程桌面的全面体验，并且与微软合作升级了远程桌面功能。此外，Meta AI通过开源llama 3.2模型变得更智能，支持图像和文本理解。新功能包括通过自然语言编辑照片和跨应用的自然语音交互。Meta还在实验自动视频配音和唇形同步技术，以扩大内容的全球可及性。

Takeaways

• 🎮 Quest 3s是Meta推出的新型高质量混合现实设备，价格为2.99美元。
• 🌟 Quest 3s拥有与Quest 3相同的定义性特点，包括高分辨率彩色混合现实和Vivid pass-through手部追踪。
• 💾 Quest 3s和Quest 3提供从基础版到512GB和4K显示屏的高端版本，价格为4.99美元。
• 📱 Quest 3s可以运行从2D移动应用到PC远程桌面的各种应用，并支持完全沉浸式体验。
• 👥 Meta与微软合作升级了远程桌面，用户可以轻松连接到任何Windows 11 PC。
• 🌐 Meta推出了名为Hyperscape的逼真虚拟空间，用户可以使用手机扫描房间并重建，或进入他人扫描共享的房间。
• 🎮 Quest 3系列将包括独家游戏《蝙蝠侠：阿卡姆阴影》，随Quest 3或3s购买附赠。
• 🤖 Meta AI通过新的开源llama 3.2模型变得更智能，支持多模态，能理解图像和文本。
• 🖼️ Meta AI的新功能允许用户通过自然语言精确编辑照片。
• 🗣️ Meta正在实验自动视频配音技术，从英语和西班牙语开始，帮助创作者扩大全球观众群。
• 👓 Meta展示了首款全功能原型透明智能眼镜，它结合了最新的显示技术和神经接口。
• 🛠️ Orion作为开发工具，将用于内部开发软件，并与少数外部合作伙伴合作，以确保内容的多样性和体验的优化。

Q & A

Meta Quest 3S的售价是多少？

Meta Quest 3S的售价是2.99美元。

Quest 3和Quest 3S有哪些共同的特点？

Quest 3和Quest 3S都具备高分辨率彩色混合现实、由相同处理器驱动的Vivid Passthrough手部追踪、自然触摸感以及用于精确控制的控制器。

Quest 3系列提供了哪些类型的体验？

Quest 3系列提供了从Horizon OS游戏、社交、健身、观看视频、生产力等多种体验。

Meta Quest 3系列的最高配置有哪些特点？

最高配置的Quest 3系列包括512GB存储和4K显示屏，价格为4.99美元。

Meta如何改进其社交应用以适应混合现实？

Meta重建了所有社交应用，为混合现实2.0提供了全新的Instagram和Facebook体验，允许用户在混合现实中享受内容。

Meta与微软合作升级了哪些功能？

Meta与微软合作升级了远程桌面功能，允许用户轻松连接到任何Windows 11 PC。

Meta AI的新功能有哪些？

Meta AI的新功能包括多模态理解，能够原生理解图像和文本，以及通过自然语言进行精确编辑图片的能力。

Meta Quest 3系列在游戏方面有哪些亮点？

Meta Quest 3系列展示了包括《Batman Arkham Shadow》在内的游戏，这款游戏是Quest 3系列的独占游戏，购买Quest 3或3S的用户可以免费获得。

Meta Quest 3系列的直播演示有哪些风险？

直播演示可能存在技术问题、不可预测性以及潜在的尴尬情况，但同时也更具吸引力、令人难忘。

Meta的Creator AI能做什么？

Creator AI可以作为数字助手，帮助用户响应和与观众互动，发送消息，分享资源，并引导人们进行对话。

Meta在自动视频配音方面有哪些进展？

Meta正在实验自动视频配音功能，从英语和西班牙语开始，这将允许创作者的内容以他们的原声在其他语言中播放，扩大观众范围。

Meta展示的透明框架眼镜有哪些特点？

透明框架眼镜展示了技术细节，重量轻于100克，具有宽视场全息显示，能够在不同光线条件下清晰显示，并允许用户看到物理世界与全息图的叠加。

Meta的全息AR眼镜有哪些技术特点？

Meta的全息AR眼镜采用了微型投影仪和波导技术，通过定制的硅片和传感器控制，能够将全息图投射到不同深度和大小的世界中。

Meta的神经接口技术是如何工作的？

Meta的神经接口技术允许用户通过手腕上的设备发送信号，从而控制眼镜，这是一种新的交互方式，不需要语音或手势。

Meta计划如何改进其AR眼镜？

Meta计划继续改进显示系统，使其更清晰，改进设计使其更小、更时尚，并努力降低制造成本，以便最终推出面向消费者的全息AR眼镜。

Outlines

00:00

💻 Quest 3混合现实设备介绍

该段落介绍了Meta公司新推出的Quest 3混合现实设备。Quest 3以其高分辨率、色彩丰富的混合现实体验而自豪，并且是市场上最好的混合现实设备。它具有高分辨率的彩色混合现实、Vivid pass-through手部追踪功能，以及用于精确操作的控制器。Quest 3能够提供从Horizon OS游戏、社交、健身、观看视频、生产力等全方位的体验。此外，还提到了Quest 3的不同配置，包括基础版和高端版，后者配备了512GB存储和4K显示屏。演示了设备如何实时流式传输内容，并且展示了新的社交应用，如Instagram和Facebook的混合现实体验。还提到了与微软合作升级远程桌面，以及即将推出的游戏《蝙蝠侠：阿卡姆暗影》。

05:01

🎥 AI和创作者工具的创新

该段落讨论了Meta AI的新功能，特别是其多模态能力，能够理解图像和文本。介绍了新的创意工具，如Imagine Edit，它允许用户通过自然语言上传和编辑照片。还提到了语音交互的潜力，以及它如何成为与AI交互的更自然的方式。此外，还展示了实时翻译功能，允许不同语言之间的对话，以及自动视频配音和唇形同步技术。最后，介绍了一种新的透明眼镜原型，它集成了先进的技术，如全息显示和通过手臂上的电池供电。

10:03

🕶️ 全息AR眼镜的未来展望

该段落介绍了一种新的全息AR眼镜原型，它具有宽视场、全息显示和轻薄的设计。眼镜通过微型投影仪和波导技术将全息图投射到用户的视野中。提到了眼镜将支持语音、手部追踪和眼动追踪功能，以及一种基于手腕的神经接口。尽管眼镜目前还处于开发阶段，但Meta计划将其作为开发工具使用，并与合作伙伴一起开发软件，以便为消费者提供最终的产品。

Keywords

💡Meta Quest

Meta Quest是Meta公司推出的虚拟现实头显设备。在视频中，Meta Quest 3s和Quest 3被提及，它们代表了高质量的混合现实体验。Meta Quest 3s是基础版本，而Quest 3则提供更高级别的配置，如512GB存储和4K显示屏。这些设备能够提供从游戏到社交、健身、观看视频和生产力等全方位的体验。

💡混合现实

混合现实（Mixed Reality）是一种结合了物理世界和数字世界的技术，用户可以看到周围的现实世界，并且能够与数字对象进行交互，就像它们真实存在一样。在视频中，Meta Quest 3s和Quest 3提供了高分辨率、彩色的混合现实体验，这是它们的核心卖点之一。

💡Vivid Passthrough

Vivid Passthrough是一种技术，它允许用户在使用虚拟现实设备时看到周围的物理世界。在视频中，提到Quest 3s和Quest 3都由相同的处理器驱动，提供自然的色彩混合现实体验，这表明Vivid Passthrough是Meta Quest设备的一个重要特性。

💡手部追踪

手部追踪是一种允许用户通过移动自己的手来与虚拟对象交互的技术。在视频中，提到Meta Quest设备支持手部追踪，使得交互感觉更自然。这是混合现实体验中提升用户沉浸感的关键技术之一。

💡控制器

控制器是虚拟现实体验中用于精确交互的设备。在视频中，提到Quest 3s和Quest 3配备了控制器，这为用户提供了更精确的操作体验，尤其是在游戏和需要细致操作的场景中。

💡Horizon OS

Horizon OS是Meta公司为虚拟现实体验开发的操作系统。在视频中，提到用户可以通过Horizon OS体验游戏、社交、健身等多种活动，这表明Horizon OS是Meta Quest设备提供丰富体验的软件基础。

💡远程桌面

远程桌面是一种允许用户通过一个设备控制另一个设备的技术。在视频中，提到了与微软合作升级的远程桌面功能，使得用户能够通过Meta Quest设备连接到Windows 11 PC，这扩展了虚拟现实设备的使用场景，使其成为PC的自然延伸。

💡超空间（Hyperscape）

超空间是Meta公司推出的一个功能，它允许用户通过手机扫描一个房间并重建它，或者进入别人扫描并分享的房间。在视频中，提到超空间是将真实世界空间带入元宇宙的一种方式，这展示了混合现实技术在社交和内容共享方面的潜力。

💡多模态（Multimodal）

多模态是指能够同时处理和理解多种类型信息的技术，如文本、图像和声音。在视频中，提到Meta AI的新功能，即通过多模态模型理解图像和文本，这使得AI能够更自然地与用户交互，提供更丰富的体验。

💡想象编辑（Imagine Edit）

想象编辑是Meta公司推出的一项创意工具，它允许用户通过自然语言上传和编辑照片。在视频中，提到了这项工具，它展示了Meta AI在理解用户意图和执行精确编辑任务方面的能力，这是AI技术在创意领域的应用示例。

💡神经接口

神经接口是一种直接连接人脑和外部设备的技术。在视频中，提到了一种基于手腕的神经接口，它允许用户通过思考来控制设备，这是一种未来可能用于控制虚拟现实和增强现实设备的先进技术。

Highlights

新推出的Meta Quest 3s提供高质量的混合现实体验，售价2.99美元。

Quest 3是首款主流的高质量混合现实设备，可以看见周围世界并触摸数字对象。

Quest 3s与Quest 3共享相同的定义性特征，包括高分辨率彩色混合现实。

Quest 3s搭载Vivid pass-through处理器，支持自然手势跟踪和精确控制器。

Quest 3系列提供从Horizon OS游戏到社交、健身、观看视频、生产力等全方位体验。

Quest 3系列不仅性价比高，而且是市面上最好的产品。

演示内容直接从Quest 3系列头显实时流式传输。

内容库大规模扩展，从2D移动应用到PC远程桌面，再到完全沉浸式体验。

所有社交应用都为混合现实2进行了重建，包括全新的Instagram和Facebook体验。

与微软合作升级远程桌面，即将实现通过视线连接到任何Windows 11 PC。

引入了名为Hyperscape的逼真空间到元宇宙中，可以通过手机扫描房间并重建。

展示了即将推出的游戏《Batman Arkham Shadow》，它将独家提供给Quest 3系列。

Meta AI变得更智能，引入了新的开源Llama 3.2模型，支持多模态理解图像和文本。

新增的创意工具Imagine Edit允许用户通过自然语言精确编辑照片。

Meta AI将支持自然语音交互，可能成为与AI互动的最常见方式。

演示了Creator AI，它可以根据用户的响应方式和与观众互动的方式进行训练。

自动视频配音技术将从英语和西班牙语开始，让不同语言的用户能看到更多内容。

展示了透明框架眼镜，庆祝技术的特殊性。

介绍了Orion眼镜，这是首款完全功能的原型，也是最先进的眼镜。

Orion眼镜将使用基于手腕的神经接口，提供全新的显示架构。

Orion眼镜将作为开发工具使用，与合作伙伴共同开发软件。

Transcripts

this is the new meta Quest

3s high quality mixed reality for

$2.99 hell

yeah I've been waiting for this one for

a long time um you know when when we

built Quest 3 we delivered the first

highquality mainstream mixed reality

exper experience where you can see the

world around you you can reach out and

touch digital things as if they're right

there um Quest 3 is the best mixed

reality device that you can buy today

and and I am really proud of it Quest 3s

has the same defining features as Quest

3 high resolution color mixed reality

powered by the same processor Vivid pass

through handtracking feels Natural Touch

plus controllers for precision you get

the full range of experiences from

Horizon OS gaming Social Fitness

watching videos productivity and more so

whether you're looking to get started

with the base Quest 3s or if you want

the top-of-the-line Quest 3 with 512 GB

and 4K displays which is now just

$4.99 the quest 3 family it's not just

the best value it is the best product

available everything you're going to see

up here is streamed from his headset

live this is what he is

seeing yeah it's

sharp all right Kenny what do we got got

all right now the first thing that you

can see is that we have

expanded the quality the the volume of

content

massively yeah sorry to the dude next to

him live demo artifact all right um You

can now run everything from 2D mobile

apps to remote desktop for PC uh to

fully immersive experiences all on one

device you can open screen you can put

them anywhere around you you can have

pretty much as many of them running as

you want and we have also rebuilt all of

our social apps for mixed reality 2

we've got all new Instagram and Facebook

experiences to bring your feeds and your

reels uh with you into mixed reality a

really nice interface uh so you can lean

back enjoy your content on nice big

screens um so and there's there's a

bunch more too are we've been working

with

Microsoft uh to upgrade remote desktop

soon so now you're soon you're going to

be able to easily connect to any Windows

11 PC you just look at the

keyboard and um and you and it'll start

pairing so if you want to work on a

giant virtual display or a bunch of

virtual displays or you know if you want

to work on your projects without

distractions or with other people you

know this is Quest is going to be able

to be a natural extension of your PC

we're also bringing photorealistic

spaces to the

metaverse and we call this

hyperscape and the way this works is you

can use your phone to scan a room and

then recreate it or you can step into a

room that someone else has scanned and

shared now of course there's gaming um

so we got some really good ones coming

showing off what the quest 3 family can

do Batman Arkham Shadow up here um

exclusively on the quest 3 family it's

to be included in any purchase of a

quest 3 or 3s uh that you buy this fall

lots of great experiences a lot more

coming soon today we are making meta AI

even smarter uh with our new open-source

llama 3.2 models and I'm going to get

into more details about llama 3.2 in

just a minute but one of the headlines

is that they're multimodal um so now

meta AI can natively understand images

as well as text

now we're using this new capability to

build some features that that I I

haven't seen anyone else build I mean

this is this is pretty interesting and

novel stuff creative tools that we're

adding to imagine edit it lets you

upload any photo and edit it

precisely um with natural language in

meta AI across the apps I think that

voice is going to be a way more natural

way of interacting with AI than text and

probably I think it has the potential to

be one of if not the most frequent ways

that we all interact with AI um it is

just a lot better so now you're going to

be able to have natural voice

conversations with medai across all the

major apps you got you know Instagram

WhatsApp messenger

Facebook Hey are live demos are

risky live demos can be risky yes they

can be unpredictable prone to technical

issues and potentially embarrassing

however they can also be engaging

memorable I'm going to do it

anyway go for it just be prepared for

anything to happen so you were one of

the first creators to build one of these

Creator AIS with the text only platform

um you know can can you tell us what

what it does and and your experience

with it yeah so like for me I've been

treating my Creator AI kind of like like

my digital assistant so you know I've

trained it on how I respond and how to

engage with my audience and then yeah

can like send DMS can share links to

relevant resources and then uh what else

am I having to do oh yeah it can just it

guides people through conversation it's

great congrats on the new book that you

just released you know what's the main

thing that you're hoping that people

take away from

it thank you so much yeah the main thing

I want people to take away from my book

is the idea that you have the power to

create your own opportunities by

combining curiosity adaptability and

resilience in a rapidly evolving digital

world we are experimenting with

automatic video dubbing on reals

uh starting with English and Spanish um

so this is going to be a way that you

can see more of the content that's out

there no matter what language you speak

and as a Creator this is going to let

you reach a lot more people around the

world no matter what language you speak

so now you're going to be able to take

your content and it's going to be in

your authentic voice um speaking other

languages if you want and we're also

doing automatic lip syncing um so that

we actually looks like you are speaking

the other language so let's check this

one out

my native language is Spanish but check

it out now I can also speak English

through AI automatic dubbing so

glasses they're kind of the perfect form

factor for AI right for letting an AI

assistant see what you see hear what you

hear be able to communicate you with you

privately um so it can help you out with

whatever you're doing throughout the day

I am about to have a conversation with

Brandon Mareno former UFC flyweight

champion of the world where he is

speaking Spanish and I am speaking

English all right Brandon you want to

come out here and show us

this starting live translation from

Spanish to

[Applause]

English

hey hey how are you how is your knee

back

training doing well KN is is doing well

it feels strong I'm ramping up training

um how are you doing are you getting

ready for your next

fight so today we are dropping a limited

edition of clear transparent

frames um you know I think this is neat

because it's it's it's a neat Vibe right

you can see the

technology in them you know for the

first few versions We spent so much

energy designing the glasses to

basically hide the technology um to make

them look like normal stylish glasses

but now I think people are starting to

appreciate what makes them special and I

think having a design that celebrates

that um is pretty awesome this is

a our first fully functioning

prototype and if I do say so the most

advanced glasses the world has ever seen

you know not a headset no WI

less than 100

gr uh they need wide field of view

holographic

displays sharp enough to pick up

details bright enough to see in

different lighting

conditions large enough to display a

cinema screen or multiple monitors for

working wherever you go whether you're

you in a coffee shop or on a plane or

wherever you are

and you need to be able to see through

them and people need to be able to see

them through them too and make eye

contact with you right this isn't pass

through this is the physical world with

holograms overlaid on it it is a

completely new kind of display

architecture with these tiny projectors

and the arms of the glasses that shoot

light into wave guides that have you

know nanoscale 3D structures etched into

into the lenses so they can defract

light and put Holograms at different uh

depths and sizes into the world in front

of you and all of that is directed by

custom silicon and sensors that we

designed and powered by a battery that

fits in the arm of the

glasses it is a absolutely incredible

amount of

technology to be able to miniaturize and

fit into a pair of glass and a small

Puck that goes with it to help power the

whole thing they're going to do voice

and

AI they're going to do hand tracking and

eye tracking so that way you can select

UI Elements by looking at them voice is

great but the thing is sometimes you're

in public and you don't want to say what

you're trying to do with your computer

out loud hand tracking is neat for

controlling different interfaces but you

don't want to like walk down the street

like this right

so I think that you need a device

that allows you to you just send a

signal from your brain to the

device so this isn't just the First full

screen like you know

full Widefield view holographic AR

glasses this is also the first device

that is powered by our wrist-based

neural interface we do still have a few

things that I want us to keep pushing on

before we ship this as a consumer

product um we're going to keep tuning

the display system to make it sharper I

want to keep working on the design to

make it smaller and a bit more

fashionable um and we are going to keep

making we need to keep working on the

the um manufacturing to make it a lot

more affordable

too but we have line of sight to all of

those things so we are going to use

Orion as a dev kit um we are going to

use it mostly internally to build out

the software that we that we need to but

we're also going to work with a handful

of Partners external to make sure that

we get a diversity of content and and um

you know just that we can really dial in

the software and the experience so that

way when we have the next version of

this Hardware it is going to be ready to

be our first consumer full holographic

AR glasses