Meta Connect 2024: Everything Revealed in 12 Minutes

CNET

25 Sept 202411:48

Summary

TLDRThe new Meta Quest 3S offers high-quality mixed reality at an affordable $2.99, featuring high-resolution color mixed reality, natural hand tracking, and a range of experiences from gaming to productivity. The Quest 3 family, starting at $4.99, includes a 512 GB model with 4K displays. New social apps, Microsoft collaboration for remote desktop, and gaming experiences like Batman Arkham Shadow are highlighted. Meta AI is enhanced with multimodal capabilities, allowing natural voice interactions and image understanding. Live demos showcase real-time language translation and automatic video dubbing, while the prototype of advanced AR glasses with neural interface hints at the future of wearable tech.

Takeaways

💾 The new Meta Quest 3S offers high-quality mixed reality at an affordable price of $2.99.
🎮 Quest 3 is considered the best mixed reality device available today, with features like high-resolution color mixed reality and natural touch controls.
📱 The device supports a wide range of experiences including gaming, social, fitness, productivity, and more.
💾 The top-of-the-line Quest 3 model features 512 GB storage and 4K displays, priced at $4.99.
🌐 The content volume has been massively expanded, allowing users to run everything from 2D mobile apps to fully immersive experiences.
🤝 Meta is working with Microsoft to upgrade the remote desktop feature, allowing easy connection to Windows 11 PCs.
🌐 New social apps for mixed reality have been rebuilt, including Instagram and Facebook experiences that bring feeds and reels into mixed reality.
🎮 Gaming on Quest 3 is enhanced with titles like Batman Arkham Shadow, which is included with any Quest 3 or 3S purchase this fall.
🧠 Meta AI is becoming smarter with the introduction of the open-source llama 3.2 models, which are multimodal and understand both images and text.
🗣️ Voice interaction with AI is being promoted as a more natural way of communication, with the ability to have natural voice conversations across major apps.
👓 Meta is also developing advanced AR glasses with holographic displays and a wrist-based neural interface, although they are not yet consumer-ready.

Q & A

What is the new feature of the Meta Quest 3s?
-The Meta Quest 3s offers high-quality mixed reality with features like high resolution color mixed reality, powered by the same processor as Quest 3, Vivid pass-through hand tracking, and natural touch plus controllers for precision.
What is the price range for the Quest 3 family?
-The base Quest 3s is priced at $2.99, while the top-of-the-line Quest 3 with 512 GB and 4K displays is now priced at $4.99.
What kind of experiences does Quest 3s offer?
-Quest 3s offers a full range of experiences including Horizon OS gaming, social, fitness, watching videos, productivity, and more.
How has the content volume expanded on the Quest 3?
-The content volume has massively expanded to include everything from 2D mobile apps to remote desktop for PC and fully immersive experiences.
What new social apps are being introduced for mixed reality?
-New Instagram and Facebook experiences are being introduced to bring your feeds and reels into mixed reality.
What collaboration is mentioned for upgrading the remote desktop experience?
-Meta is working with Microsoft to upgrade the remote desktop experience, allowing easy connection to any Windows 11 PC.
What is the new feature called 'hyperscape'?
-Hyperscape allows users to scan a room with their phone and recreate it in the metaverse, or step into a room that someone else has scanned and shared.
What is the significance of the Meta AI update with the llama 3.2 models?
-The Meta AI update with llama 3.2 models introduces multimodal capabilities, allowing the AI to natively understand images as well as text.
How does the new 'Imagine Edit' feature work?
-Imagine Edit allows users to upload any photo and edit it precisely with natural language using Meta AI across the apps.
What is the new feature for voice interaction with AI?
-The new feature enables natural voice conversations with Meta AI across all major apps, making voice interaction a more natural and potentially frequent way to interact with AI.
What is the significance of the live demo with the Creator AI?
-The live demo showcases the capability of the Creator AI to act as a digital assistant, engaging with an audience, sending DMs, sharing links, and guiding people through conversation.
What is the new feature for content creators regarding language translation?
-Meta is experimenting with automatic video dubbing on reels, starting with English and Spanish, to help creators reach a global audience regardless of language barriers.
What are the key features of the new glasses prototype?
-The new glasses prototype features a wide field of view, holographic displays, are lightweight, have a clear transparent design, and are powered by a wrist-based neural interface.
What is the current status of the holographic AR glasses mentioned in the script?
-The holographic AR glasses are in the prototype stage and are planned to be used as a dev kit internally and with a few external partners to build out the necessary software and experience.

Outlines

00:00

🤖 Introduction to Meta Quest 3

The speaker introduces the new Meta Quest 3, a high-quality mixed reality device available for $2.99. They express pride in the device's capabilities, which include high-resolution color mixed reality, natural hand tracking, and precision controllers. The Quest 3 offers a wide range of experiences from gaming to productivity. The presenter also highlights the device's ability to stream content live and its compatibility with various applications, including social apps rebuilt for mixed reality. They mention partnerships with Microsoft for remote desktop access and the introduction of photorealistic spaces called 'hyperscape'. The device also comes with gaming experiences like Batman Arkham Shadow and the integration of Meta AI with new open-source llama 3.2 models for multimodal understanding of images and text.

05:01

📚 Creator AI and Language Translation

The speaker discusses their use of Creator AI as a digital assistant, which they've trained to engage with their audience and perform tasks like sending direct messages and sharing resources. They mention a new book that emphasizes the importance of curiosity, adaptability, and resilience in the digital world. The script also covers the experiment with automatic video dubbing in English and Spanish to make content more accessible globally. Additionally, there's a live demonstration of real-time translation between Spanish and English, showcasing the potential for AI in language translation. The speaker also talks about the development of advanced glasses that integrate AI and can overlay holograms onto the physical world, with plans to refine the technology before releasing it to consumers.

10:03

🕶️ Advancements in AR Glasses Technology

The speaker provides insights into the development of advanced AR glasses that include hand tracking, eye tracking, and a neural interface controlled by wrist-based signals. They discuss the challenges of miniaturizing technology to fit into glasses and the need for the device to be fashionable and affordable. The script describes the glasses as having holographic displays with a wide field of view, capable of displaying detailed and bright images in various lighting conditions. The speaker also mentions the use of Orion as a dev kit for internal software development and collaboration with external partners to ensure a diverse content ecosystem. The goal is to create the first consumer-ready full holographic AR glasses.

Mindmap

Keywords

💡Meta Quest

Meta Quest refers to a series of virtual reality headsets developed by Meta Platforms, previously known as Facebook. In the script, the new Meta Quest 3 is highlighted as a high-quality mixed reality device, showcasing the company's advancements in VR technology. The device offers a mainstream mixed reality experience where users can interact with digital objects as if they were physically present.

💡Mixed Reality

Mixed Reality (MR) is an umbrella term covering a range of technology that mixes the physical and virtual worlds, combining real-world and digital elements to create new environments and visualizations. The script emphasizes the Meta Quest 3's ability to deliver high-quality MR experiences, allowing users to see and interact with both the physical world and digital content simultaneously.

💡Vivid Passthrough

Vivid Passthrough is a technology that allows virtual reality headsets to display the real world in high resolution and color, enabling users to interact with their environment naturally. The script mentions that the Meta Quest 3 features Vivid Passthrough, enhancing the mixed reality experience by providing clear visuals of the surrounding world.

💡Handtracking

Handtracking is a feature in virtual and mixed reality systems that allows users to control digital interfaces using natural hand gestures, without the need for physical controllers. The script describes handtracking on the Meta Quest 3 as feeling natural, providing a more intuitive way to interact with digital content.

💡Horizon OS

Horizon OS is a virtual reality operating system developed by Meta for their VR devices. It is designed to provide a social and immersive experience. The script mentions Horizon OS in the context of the full range of experiences available on the Quest 3, including gaming, social interaction, fitness, and productivity.

💡Photorealistic Spaces

Photorealistic Spaces refers to the ability to create highly detailed and realistic digital representations of physical spaces. In the script, it is mentioned that Meta is bringing photorealistic spaces to the metaverse through 'hyperscape', allowing users to scan and recreate rooms or enter shared spaces, enhancing the immersion in virtual environments.

💡Meta AI

Meta AI refers to the artificial intelligence technologies developed by Meta Platforms. The script discusses the advancements in Meta AI with the introduction of the open-source llama 3.2 models, which are multimodal and can understand both images and text. This is showcased through features like Imagine Edit, which allows users to edit photos with natural language commands.

💡Multimodal

Multimodal refers to systems that can process and understand information from multiple modes of communication, such as text, images, audio, and video. The script highlights that the new Meta AI models are multimodal, allowing for a more comprehensive understanding and interaction with digital content, including the ability to work with images as well as text.

💡Neural Interface

A Neural Interface is a direct communication pathway between the human brain and an external device. The script discusses a wrist-based neural interface that powers the advanced glasses, suggesting a future where users can control devices with brain signals, offering a more seamless and intuitive interaction.

💡Holographic Displays

Holographic Displays are a type of display technology that creates images by the interference of light waves, resulting in 3D visuals that can be viewed from different angles. The script describes the advanced glasses with holographic displays that can overlay digital content onto the physical world, representing a new kind of display architecture.

💡Live Translation

Live Translation is the real-time conversion of spoken language from one language to another. The script includes a demonstration of live translation from Spanish to English, showcasing the potential for technology to break down language barriers and facilitate communication across different languages.

Highlights

Introduction of the new Meta Quest 3s with high quality mixed reality.

Quest 3 offers mainstream mixed reality experience with high resolution color mixed reality.

Quest 3s is equipped with Vivid pass-through handtracking for a natural touch experience.

The Quest 3 family is considered the best value and product available in the market.

Content expansion with the ability to run everything from 2D mobile apps to fully immersive experiences.

Rebuilt social apps for mixed reality 2, including new Instagram and Facebook experiences.

Cooperation with Microsoft to upgrade remote desktop for easy connection to Windows 11 PCs.

Introduction of hyperscape, allowing users to recreate scanned rooms in the metaverse.

Upcoming gaming experiences showcasing the capabilities of the Quest 3 family.

Meta AI is made smarter with the new open-source llama 3.2 models.

Llama 3.2 models are multimodal, understanding both images and text.

Introduction of creative tools in imagine edit that allow photo editing with natural language.

The potential of voice interaction with AI being more natural than text.

Live demo of a Creator AI assisting in audience engagement and resource sharing.

Experimentation with automatic video dubbing on reals for language inclusivity.

Live translation from Spanish to English during a conversation with a UFC champion.

Introduction of limited edition clear transparent frames to celebrate the technology.

Presentation of the first fully functioning prototype of advanced glasses with holographic displays.

The glasses are powered by a wrist-based neural interface for a more natural control.

Plans for internal development and external partnerships to refine the software and experience.

The next version of the hardware is aimed to be the first consumer full holographic AR glasses.