Apple WWDC 2024: AI, iOS 18 and More in Less than Six Minutes | WSJ

Summary

Apple unveils its AI system, 'Apple intelligence,' integrating it across iOS, iPadOS, and macOS. The update includes a smarter Siri with enhanced privacy, typing capabilities, and personal data understanding. New features such as generative AI for writing tools, emoji creation, and image search are introduced. Voice transcription and prioritized notifications round out the enhancements, initially available on the latest iPhone 15 Pro models, iPads, and Macs with M chips, and in English only.

Takeaways

• 🧠 **Apple Intelligence**: Apple's AI system is being integrated into iPhone, Mac, and iPad operating systems.
• 🗣️ **Siri Enhancements**: Siri is evolving to be smarter, understanding personal information while maintaining privacy through on-device or cloud compute systems.
• 💬 **Conversational Siri**: Siri can now be interacted with via typing, making it more chatbot-like and capable of understanding screen content.
• 🔍 **Siri's New Abilities**: Siri can find information across apps, even if the user doesn't remember where it's located, and autofill forms with personal data.
• 🤖 **Siri and Chat GPT**: Siri will collaborate with Chat GPT for tasks it can't handle, offering access to paid features for subscribers.
• 📝 **Writing Tools**: Generative AI will be used for summarizing, rewriting, and proofreading text across various Apple apps.
• 🎨 **Generative Imagery**: Apple introduces 'gen Emoji', allowing users to create custom emojis through descriptions.
• 🖼️ **Image Playground**: A new app for quick image creation with various styles, integrated into messages and pages.
• 🔍 **Improved Photo Search**: Apple intelligence enhances photo search capabilities with more specific criteria and a cleanup tool for background removal.
• 📝 **Voice Transcription**: Apple is introducing on-device AI transcription in the Notes app, with summarization capabilities and similar features for call recording.
• 🔔 **Smart Notifications**: AI will prioritize notifications, with the most important appearing at the top for quick attention.
• 📅 **Release Timeline**: Apple intelligence features will roll out in iOS 18, iPad OS 18, and Mac OS Sequoia, initially in beta this summer, with full availability in English later in the year.
• 🚫 **Hardware Requirements**: Some new AI features are exclusive to the latest iPhone 15 Pro models and devices with M chips or later.

Q & A

What is Apple intelligence mentioned in the script?

Apple intelligence refers to Apple's AI system that is integrated throughout its iPhone, Mac, and iPad operating systems.

How often was the term 'Apple intelligence' mentioned in the conference?

The term 'Apple intelligence' was mentioned approximately 60 times during the announcements.

What is the main goal of the improvements to Siri mentioned in the script?

The main goal is to make Siri smarter and more capable with personal information while keeping user data private through on-device models or Apple's private Cloud compute system.

How does the new Siri enhance user interaction?

The new Siri can understand users better, even when they make mistakes in their speech, and allows typing to interact more like a chatbot.

What is the significance of Siri being able to understand what appears on the user's screen?

This feature allows Siri to find information even if the user doesn't remember where it is, such as in a message, email, or note, and can work across apps.

What is the role of chat GPT in the new Siri updates?

When Siri doesn't know how to perform a task, it can ask for permission to use chat GPT, and users can connect their chat GPT accounts to access paid features.

What writing tools does Apple intelligence offer in its new update?

Apple intelligence offers generative AI models that can summarize, rewrite, and proofread text in various apps, and also suggest responses with different tones.

What is 'gen Emoji' and how does it work?

'Gen Emoji' is a feature that allows users to generate new emojis by adding a description, which can then be used as reactions or stickers in messages.

What is the 'image playground' app mentioned in the script?

The 'image playground' app is a new feature that lets users create images in seconds with various styles like animations, sketches, or illustrations by providing a description.

How will Apple intelligence improve photo search and editing?

Apple intelligence will allow more specific photo searches and includes a cleanup tool to remove objects or people in the background of photos, similar to Google's Magic erase feature.

What is the new voice transcription feature in the notes app?

The new voice transcription feature in the notes app uses AI to transcribe spoken words into text and also offers a summary of the transcript.

How will Apple prioritize notifications with its AI system?

Apple will use AI to prioritize notifications, placing the most important ones at the top of the stack for users to see at a glance.

When will Apple intelligence start appearing in Apple's operating systems?

Apple intelligence will start appearing in iOS 18, iPad OS 18, and Mac OS Sequoia, first in beta versions in the summer, and more broadly in the fall.

What are the hardware requirements for the new AI features introduced by Apple?

The new AI features require the latest iPhone 15 Pro lineup and iPads, as well as Macs running the M chip or later.

What are the two main limitations mentioned for the new Apple intelligence features?

The two main limitations are that not all features will be available right away, with some coming by the end of the year or even next year, and that the features are initially only available in English.

Outlines

00:00

🤖 Apple's AI Integration and Siri Enhancements

Apple has unveiled its AI system, 'Apple intelligence,' during its worldwide developers conference. This system is set to be integrated across iPhone, Mac, and iPad operating systems. The focus is on enhancing Siri's capabilities, with a new emphasis on privacy and on-device processing. Siri is now more adept at understanding user commands, even when they are misstated, and can perform tasks like typing to Siri, similar to a chatbot. Additionally, Siri can now search across apps and documents, and even fill out forms by extracting information from personal data like a driver's license. Apple also announced a partnership with chat GPT, allowing Siri to use it when it doesn't know how to perform a task, with the option to connect to a user's chat GPT account for access to paid features.

05:02

📝 Writing Tools and Generative AI Models

Apple is introducing writing tools powered by generative AI models that can summarize, rewrite, and proofread text in various apps like Notes, Mail, and Pages. Users can now adjust the tone of their responses to be more friendly, professional, or concise. Apple will also suggest different tones for messages and emails, a feature that has been seen in Google and Samsung's offerings but is now being implemented by Apple as a welcome upgrade.

😜 Introducing Gen Emoji and Image Playground

Apple is introducing 'gen Emoji,' a feature that combines generative image creation with emoji and mimoji. Users can generate new emojis by providing a description, which can then be used as reactions in Messages or as stickers. Additionally, a new app called 'Image Playground' allows for the quick creation of images in various styles such as animations, sketches, or illustrations. Users can type a description to create an image in their desired style, and Apple's AI will assist in searching through photos with more specificity, including a cleanup tool to remove unwanted objects or people from the background.

📝 Voice Transcription and Notification Prioritization

Apple is catching up with competitors by introducing on-device AI transcription in the Notes app, with the ability to summarize transcripts. A similar feature is coming to the Phone app for recording calls. Additionally, Apple will use AI to prioritize notifications, ensuring the most important ones appear at the top. This feature is designed to help users quickly identify what requires their attention.

📅 Availability and Hardware Requirements

Apple intelligence will first appear in beta versions of iOS 18, iPad OS 18, and Mac OS Sequoia this summer, with a broader release planned for the fall. However, not all features will be available immediately, with some, like the chat GPT integration, expected to arrive by the end of the year. Furthermore, Apple has noted that these new AI features require the latest iPhone 15 Pro lineup and iPads, as well as Macs running the M chip or later, indicating a need for new hardware to fully utilize the advancements in Apple's AI capabilities.

Keywords

💡Apple Intelligence

Apple Intelligence refers to the AI system developed by Apple that is set to be integrated throughout its operating systems, including iPhone, Mac, and iPad. It is central to the video's theme, highlighting Apple's advancements in AI technology. The script mentions its introduction at the Worldwide Developers Conference, emphasizing its role in enhancing user experience through privacy-focused data handling and improved functionalities.

💡Siri

Siri is Apple's voice-activated assistant, which is undergoing improvements to become smarter and more capable of understanding and processing personal information while maintaining privacy. In the video, Siri's enhancements are a key focus, with new features such as typing capabilities and the ability to understand and act on information across different apps and services, exemplified by the script's mention of Siri finding and inputting personal data into forms.

💡On-device models

On-device models are a type of AI processing that occurs directly on the user's device rather than on remote servers, which helps in keeping personal data private. The script discusses Apple's commitment to using on-device models for Siri, ensuring that user data is processed locally and remains secure.

💡Private Cloud Compute

Private Cloud Compute is a system that Apple is introducing to further enhance data privacy. It allows for AI processing in the cloud while still maintaining the privacy of the user's data. The script mentions this system in the context of Apple's efforts to keep user data private, suggesting a balance between AI capabilities and data security.

💡Chat GPT

Chat GPT is a language model that Siri can integrate with to handle tasks it doesn't know how to perform. The script introduces this feature as a new buddy for Siri, allowing it to access additional capabilities and content, even offering integration with paid features for users who have a Chat GPT account.

💡Generative AI models

Generative AI models are AI systems capable of creating new content, such as text, images, or audio, based on input data. In the video, these models are highlighted for their ability to summarize, rewrite, and proofread text in various Apple applications, showcasing the script's emphasis on the creative and functional aspects of AI in enhancing user productivity.

💡Gen Emoji

Gen Emoji is a feature that allows users to generate custom emojis by providing a description, blending generative image creation with emoji and mimoji technology. The script presents this as an innovative aspect of Apple's AI capabilities, demonstrating the personalization and creativity enabled by AI in user communication.

💡Image Playground

Image Playground is an app mentioned in the script that enables quick creation of images through descriptions, offering various styles like animations, sketches, or illustrations. It represents Apple's push towards integrating AI into creative processes, allowing users to visualize their ideas with ease and precision.

💡Voice Transcription

Voice Transcription is the process of converting spoken language into written text using AI. The script discusses Apple's introduction of this feature in its Notes app and the ability to summarize transcripts, aligning with the video's theme of enhancing user experience through AI-powered functionalities.

💡Priority Notifications

Priority Notifications is a feature that uses AI to sort and prioritize notifications, ensuring that the most important ones are easily accessible. The script positions this feature as part of Apple's strategy to streamline user interactions with their devices, highlighting the video's focus on AI's role in organizing and simplifying digital experiences.

💡iOS 18, iPad OS 18, Mac OS Sequoia

These are the upcoming operating systems for Apple's devices, mentioned in the script as the platforms on which Apple Intelligence will first appear in beta versions. They represent the next generation of Apple's software, integrating AI advancements and setting the stage for the company's future AI-driven features.

Highlights

Apple introduces its AI system, Apple intelligence, to be integrated across iPhone, Mac, and iPad operating systems.

Apple intelligence is mentioned around 60 times during the conference, emphasizing its significance.

Siri, Apple's assistant, is set to become smarter with improved personal data handling and privacy.

New Siri features include better comprehension of user queries, even when misspoken.

Siri gains the ability to understand and act on typed commands, resembling a chatbot.

Siri will be able to find and interact with content across different apps and services.

Apple introduces the concept of using Siri to autofill forms with personal information.

Siri will collaborate with chat GPT for tasks it cannot perform, without requiring a subscription.

Apple plans to integrate with other large language models in the future.

New writing tools with generative AI capabilities for summarizing, rewriting, and proofreading text.

Apple's writing tools will allow users to change the tone of their responses in various apps.

Introduction of gen Emoji, a feature combining generative image creation with emoji and mimoji.

Image playground app allows for quick creation of images through text descriptions.

Apple intelligence enhances photo search capabilities and introduces a cleanup tool for photo editing.

On-device AI transcription feature coming to Notes app and phone app for call recording.

AI prioritizes notifications, displaying the most important ones at the top.

Apple intelligence will first appear in iOS 18, iPad OS 18, and Mac OS Sequoia.

Some features of Apple intelligence will only be available on the latest iPhone 15 Pro models and devices with M chip or later.

Not all features of Apple intelligence will be available immediately; some may arrive by the end of the year or next year.

Transcripts

Apple intelligence Apple intelligence

Apple intelligence if you're wondering

what Apple's been talking about here at

its worldwide developers conference let

me sum it up for you introducing Apple

intelligence you see what they did there

yes Apple intelligence is the company's

AI system that will be integrated

throughout its iPhone Mac and iPad

operating systems and yes they said

Apple intelligence around 60 times some

of these intelligence announcements are

just table Stakes Samsung and Google

already have them others break new

ground here's my AI analysis number one

Siri yes ever since Siri's birth Apple's

been promising that it can do things for

you just by you asking not exactly yet

maybe now is Siri's moment the assistant

is getting smarter at what it can do

with your personal information the aim

is to keep your data private either by

using on device models or using Apple's

new private Cloud compute system Apple

says the new Siri can understand you

better even when you flub what you're

saying what does the weather look like

for tomorrow at Mir Beach oh wait I

meant Mir

Woods the forecast is calling for clear

skies in the morning near mer Woods

National Monument the big thing is that

you can now type to Siri making it much

more like a chatbot but unlike those

other chatbots Apple's making the point

that Siri knows all about you and your

personal info it eventually will be able

to understand what appears on your

screen you can ask Siri to find

something even if you don't remember

it's in a message email or note and it

can work across app or for times when

you're filling out a form and need to

input your driver's license Siri will be

able to find a photo of your license

extract your ID number and type it into

the form for you number two Siri and

chat GPT Siri has a new buddy when you

ask Siri to do something that it doesn't

know how to do it will ask your

permission to use chat GPT and you don't

have to be a chat GPT subscriber either

and if you have a chat GPT account you

can connect it and access the paid

features Apple says it plans to

integrate with other large language

models later number three writing tools

these were expected generative AI models

will be able to summarize rewrite and

proofread text in Notes mail pages and

more well now you can change the tone of

that response to your colleague to make

it sound more friendly professional or

concise Apple will also suggest

responses and different tones for

messages and emails Google and Samsung

have been doing this but it's a very

welcome upgrade here too number four

Images we're introducing gen Emoji yes

that's a thing it's generative image

creation mixed with emoji and mimoji you

can generate these new emojis by adding

a description you can then use them as a

tapback reaction in messages inline or

stickers just provide a description and

you'll see your gen mooji appear right

before your eyes along with more options

to choose from we were promised flying

cars and well we got AI that can make

emojis of flying cars I mean yeah of

course I'm really excited about this one

then there's a new app called image

playground that lets you create images

in seconds it's also built into apps

like messages and pages and you choose

from animations sketch or illustration

Styles when you have a really specific

idea in mind you could just type a

description to add it to your

playground and you can easily adjust

which style you want to use and speaking

of images Apple intelligence will make

it easier for you to search through

photos with more specificity and similar

to Google's Magic erase feature there's

a cleanup tool that can help you remove

objects or people in the background of

your photos number five voice

transcription can I get a finally on

this one Samsung and Google both have on

device AI transcription apple is just

getting with the program it's coming to

the notes app and you can use AI to

summarize the transcript for you too

there's a similar feature coming to the

phone app so you can record phone calls

and do the same don't worry participants

are automatically notified so no one is

surprised number six notifications

taking another page from Google and say

Samsung Apple will use AI to prioritize

your notifications the most important

will appear at the top your priority

notifications appear at the top of the

stack letting you know what to pay

attention to At a Glance Apple

intelligence will start appearing in iOS

18 iPad OS 18 and Mac OS Sequoia first

in the beta versions this summer and

then more broadly to everyone this fall

and it's only available in English at

first but there are two big bummers not

all the features will be available right

away Apple says the Chachi p integration

will arrive before the end of the year

and some features might not even come

until next year and all this requires

new hardware luckily this is an iPhone

15 Pro Max but sorry everyone else Apple

says these new AI features require the

latest iPhone 15 Pro lineup and iPads

and Macs running the M chip or later so

there you have it Apple's first step

into the AI Wars the Siri improvements

and the mix of on device and Cloud

processing are unique the rest is a lot

of what we've already seen from Google

and S Samsung but that's Apple's usual

move wait it out and make it better and

now we wait to test to see if it is

actually better hey Siri are you smarter

yet I'm not a person so I only know what

I've been programmed to understand well

that's going to change