Can AI Catch What Doctors Miss? | Eric Topol | TED

Summary

The script explores groundbreaking advancements in biomedical and healthcare technology through the lens of Scripps Research's experiences, highlighting the transformative impact of AI, particularly AlphaFold by DeepMind, in protein structure prediction. It discusses how AI has revolutionized diagnostic accuracy, from identifying diseases through retinal images to improving medical imaging and pathology. The narrative also touches on the potential of transformer models and GPT-4 in enhancing healthcare delivery, reducing diagnostic errors, and liberating medical professionals from administrative tasks. The speaker shares compelling patient stories where AI significantly improved diagnostic outcomes, underscoring a future where AI empowers deeper patient-doctor connections and precision medicine.

Takeaways

• 💻 AlphaFold, developed by DeepMind, has significantly accelerated the process of determining the 3D structures of proteins from taking years to just minutes.
• 📈 The American Nobel Prize recognized the contributions of Demis Hassabis, John Jumper, and their team for AlphaFold, despite their admitted lack of understanding of the underlying transformer model.
• 🛠 AlphaFold's success has inspired advancements in predicting structures of not only proteins but also RNA, antibodies, and novel proteins not found in nature.
• 📱 Diagnostic errors in medicine are a major issue, with a Johns Hopkins study indicating they contribute to 800,000 American deaths or serious disabilities annually, a problem AI could help address.
• 👁 AI has shown remarkable accuracy in medical imaging, such as determining gender from retinal images with 97% accuracy, outperforming human experts in many areas.
• 🧪 Deep learning models have demonstrated the ability to detect health issues from retinal scans that are not visible to the human eye, predicting diseases such as Alzheimer's years in advance.
• 📊 Transformer models, like GPT-4, are setting new standards in AI's capability to process and understand complex data, including language, images, and speech.
• 📚 The implementation of AI in healthcare promises 'keyboard liberation,' improving patient-doctor interactions by reducing the clerical burden on healthcare providers.
• 📷 Moorfields Eye Hospital's use of a foundation model to predict various health outcomes from retinal images demonstrates the potential for comprehensive diagnostics from single AI models.
• 📝 ChatGPT and similar AI technologies have successfully diagnosed complex medical conditions, underscoring their potential to augment or even outperform human clinical judgment in challenging cases.

Q & A

What breakthrough allowed protein structure prediction to go from taking years to just minutes?

The breakthrough was the work of AlphaFold, a derivative of DeepMind, which takes the one-dimensional amino acid sequence of a protein and can predict its 3D structure at the atomic level in just minutes. This used to take scientists years to determine experimentally.

What impact could more accurate protein structure prediction have on fields like RNA, antibodies, and genome analysis?

More accurate protein structure prediction models like AlphaFold have inspired similar models for predicting RNA structures, antibody structures, and even detecting mutations across the genome. This could massively accelerate research and discovery in these fields.

Why does the speaker suggest the AlphaFold team should get an asterisk on their award?

The speaker notes that the AlphaFold team of 30 scientists does not fully understand how their transformer model works. So he jokingly suggests they should get an asterisk on their award since the AI itself deserves some credit.

How could AI help reduce diagnostic medical errors?

AI has shown it can analyze medical images as well as or better than experts, catching things human doctors miss. With enough accurate labeled training data, AI diagnostic tools could significantly reduce errors and improve health outcomes.

What remarkable diagnoses has AI made that stumped teams of human doctors?

In one case, ChatGPT correctly diagnosed a boy with occulta spina bifida when 17 previous doctors missed it over 3 years. In another, it diagnosed limbic encephalitis in a patient incorrectly told she had long COVID. AI tools show real promise in augmenting human diagnostic capabilities.

How could transformer models like GPT help transform medicine?

Transformer models can process huge datasets spanning text, images and speech. Applied to healthcare, they could help automate keyboard/data entry tasks, analyze patient history to assist diagnosis, generate notes from doctor-patient conversations, provide decision support, and more.

What is keyboard liberation in healthcare and why does it matter?

Keyboard liberation refers to freeing doctors from having to do data entry and clerical work, letting them focus on patients. With AI handling notes, prescriptions, referrals etc. based on conversations, it improves doctor-patient relationships and outcomes.

How was the Moorfields eye disease prediction model different from previous disease detection AIs?

Unlike previous models trained on images for single diseases, this open-source model was trained on 1.6 million eye images to predict likelihoods for 8 different eye conditions with one unified model. It demonstrates the expanded capabilities from transformer architectures.

What future positive impacts does the speaker foresee AI having on medicine?

The speaker is excited about keyboard liberation freeing up doctor time, improved doctor-patient relationships as a result, AI assistance making diagnosis faster and more accurate, and transformed medical education so students learn up-to-date best practices augmented by AI.

What advice does the speaker give to a medical student about embracing the coming age of AI in medicine?

He tells the student doctor that he's lucky to be practicing in an era with keyboard liberation, more time with patients, AI diagnostic assistance, and better patient care. But he notes AI still needs extensive validation before being deployed widely.

Outlines

00:00

🧬 The Revolution of Protein Structure Prediction with AI

The speaker shares their experience at Scripps Research, highlighting the rapid advancements in protein structure prediction due to AlphaFold, a derivative of DeepMind. This AI model has significantly reduced the time required to define the 3D structure of proteins from years to minutes, revolutionizing the field. It has spurred the development of various models for predicting protein structures, RNA, antibodies, and even designing novel proteins. However, the speaker raises a thought-provoking question regarding the recognition of AI contributions, pondering if AI should receive acknowledgment in awards when the underlying mechanisms of models like transformers are not fully understood by their creators.

05:03

🔬 AI's Impact on Medical Diagnostics and Precision Medicine

The speaker discusses the potential of AI to address diagnostic errors in medicine, a significant issue highlighted by studies indicating that such errors contribute to a considerable number of deaths and disabilities annually. By leveraging AI, the medical community can achieve not just precision but accuracy in diagnostics. The speaker provides examples of AI's ability to discern details undetectable by human eyes in medical imaging, such as distinguishing gender from retinal images or identifying diseases from chest X-rays with remarkable accuracy. These advancements underscore the potential of supervised learning and AI in enhancing diagnostic processes and medical outcomes.

10:04

👁️ Breakthroughs in Health Care and Medicine Through AI

The narrative concludes with the introduction of a groundbreaking foundation model in medicine, capable of predicting various health outcomes from retinal images. This model represents a significant leap forward, consolidating multiple studies into a single, open-source model. The speaker shares compelling patient stories where AI, specifically ChatGPT, accurately diagnosed conditions missed by numerous doctors, showcasing AI's potential to revolutionize healthcare. The conversation shifts towards the future of medicine, emphasizing the transformative impact of AI on the doctor-patient relationship, diagnostics, and overall healthcare efficiency. The speaker reflects on the exciting prospects for future generations of medical professionals, highlighting the importance of validation to ensure that the benefits of AI in healthcare substantially outweigh any risks.

Keywords

💡AlphaFold

AlphaFold is an AI program developed by DeepMind that predicts the 3D structure of proteins based on their amino acid sequence. This breakthrough has significantly accelerated biological research, reducing a process that could take years to mere minutes. In the video, AlphaFold is credited for its revolutionary impact on structural biology, exemplified by its ability to rapidly and accurately model protein structures, which previously was a time-consuming task for researchers.

💡Transformer model

The transformer model is a type of deep learning architecture primarily used for understanding language. It has a unique mechanism called 'attention', allowing it to weigh the importance of different words in a sentence. The video raises a philosophical question about the recognition of AI contributions, citing the creators of AlphaFold, who utilized transformer models, yet do not fully understand how these models make their predictions, suggesting an asterisk (*) might be added to their award to acknowledge the AI's role.

💡Diagnostic medical errors

Diagnostic medical errors refer to mistakes made in diagnosing a patient's condition. The video highlights this as a significant issue in healthcare, with a large number of patients affected annually. It suggests AI's potential to reduce these errors, thereby improving patient outcomes and safety.

💡Precision medicine

Precision medicine is a medical approach that tailors treatment to the individual characteristics of each patient. The video criticizes the repetitive nature of errors in the medical field, humorously noting that making the same mistake consistently is not the goal of precision medicine. Instead, it advocates for both precision and accuracy, suggesting AI could play a key role in achieving this.

💡Supervised learning

Supervised learning is a type of machine learning where the model is trained on a labeled dataset, meaning each training example is paired with an output label. The video discusses supervised learning in the context of training AI to interpret medical images, such as retinal scans, to diagnose diseases or predict health conditions that are not visible to human eyes.

💡Machine vision

Machine vision refers to the ability of a computer to 'see' and interpret images. The script discusses several instances where machine vision, through the use of AI, surpasses human capabilities in medical imaging, such as detecting polyps during colonoscopy or identifying diseases from X-rays and scans, highlighting the potential of AI to enhance diagnostic accuracy.

💡Self-supervised learning

Self-supervised learning is a machine learning technique where the system learns to understand data by itself, without the need for labeled data. The video mentions this as a solution to a significant bottleneck in medicine—the lack of expert-labeled images for training AI—allowing for more scalable and efficient learning from medical images.

💡GPT-4

GPT-4 is a state-of-the-art language model developed by OpenAI, known for its multimodal capabilities, integrating language, images, and speech. The video discusses its potential in medicine, emphasizing its role in processing vast amounts of information and its application in generating synthetic notes and assisting with diagnoses, leading towards 'keyboard liberation' for clinicians.

💡Convolutional neural networks

Convolutional neural networks (CNNs) are a type of deep learning algorithm specifically designed to process pixel data and are widely used in image recognition and processing tasks. The video references CNNs in the context of medical imaging, where they have been utilized to detect diseases and health conditions from various scans and tests, showcasing their effectiveness in augmenting human diagnostic capabilities.

💡Keyboard liberation

Keyboard liberation refers to the use of AI to automate the documentation process in healthcare, freeing clinicians from the burdensome task of manual data entry. The video envisions a future where AI-generated synthetic notes and automated clerical tasks enable doctors to focus more on patient care, enhancing the patient-doctor relationship by providing clinicians more time to engage with their patients.

Highlights

AlphaFold's impact on reducing protein structure prediction time from years to minutes.

Recognition of AlphaFold's creators by the American Nobel Prize.

AlphaFold's influence on the development of other prediction models for proteins, RNA, and antibodies.

Discussion on whether AI should receive credit in awards when creators do not fully understand the underlying models.

The significant issue of diagnostic medical errors leading to deaths and disabilities.

AI's potential to reduce diagnostic errors and advance precision medicine.

AI's remarkable accuracy in determining patient sex from retinal images.

AI's capabilities surpassing human experts in medical imaging diagnosis.

Machine vision's superior performance in detecting polyps during colonoscopies.

Deep learning models uncovering medical conditions from retina images beyond human capability.

AI's unexpected ability to diagnose systemic diseases from electrocardiograms and chest X-rays.

Transformation from deep learning to transformer models in AI, enhancing data interpretation.

Introduction of GPT-4 and its multimodal capabilities, revolutionizing medical data analysis.

AI's role in liberating healthcare professionals from administrative tasks.

Examples of AI diagnosing rare medical conditions successfully where human doctors failed.

GPT-4's competitive diagnostic performance against expert clinicians.

The future of medicine with AI integration, offering more time for patient-doctor interaction.

Transcripts

I've had the real fortune of working at Scripps Research

for the last 17 years.

It's the largest nonprofit biomedical institution in the country.

And I've watched some of my colleagues,

who have spent two to three years

to define the crystal 3-D structure of a protein.

Well, now that can be done in two or three minutes.

And that's because of the work of AlphaFold,

which is a derivative of DeepMind, Demis Hassabis and John Jumper,

recognized by the American Nobel Prize in September.

What's interesting, this work,

which is taking the amino acid sequence in one dimension

and predicting the three-dimensional protein at atomic level,

[has] now inspired many other of these protein structure prediction models,

as well as RNA and antibodies,

and even being able to pick up all the missense mutations in the genome,

and even being able to come up wit proteins

that have never been invented before, that don't exist in nature.

Now, the only thing I think about this is it was a transformer model,

we'll talk about that in a moment,

in this award, since Demis and John

and their team of 30 scientists

don't understand how the transformer model works,

shouldn't the AI get an asterisk as part of that award?

I'm going to switch from life science,

which has been the singular biggest contribution just reviewed,

to medicine.

And in the medical community,

the thing that we don't talk much about are diagnostic medical errors.

And according to the National Academy of Medicine,

all of us will experience at least one in our lifetime.

And we know from a recent Johns Hopkins study

that these errors have led to 800,000 Americans dead

or seriously disabled each year.

So this is a big problem.

And the question is, can AI help us?

And you keep hearing about the term “precision medicine.”

Well, if you keep making the same mistake over and over again, that's very precise.

(Laughter)

We don't need that,

we need accuracy and precision medicine.

So can we get there?

Well, this is a picture of the retina.

And this was the first major hint,

training 100,000 images with supervised learning.

Could the machine see things that people couldn't see?

And so the question was, to the retinal experts,

is this from a man or a woman?

And the chance of getting it accurate was 50 percent.

(Laughter)

But the AI got it right, 97 percent.

So that training,

the features are not even fully defined of how that was possible.

Well that gets then to all of medical images.

This is just representative, the chest X-ray.

And in fact with the chest X-ray,

the ability here for the AI to pick up,

the radiologists, expert radiologists missing the nodule,

which turned out to be picked up by the AI as cancerous,

and this is, of course, representative of all of medical scans,

whether it’s CT scans, MRI, ultrasound.

That through supervised learning of large, labeled, annotated data sets,

we can see AI do at least as well, if not better,

than expert physicians.

And 21 randomized trials of picking up polyps --

machine vision during colonoscopy -- have all shown

that polyps are picked up better

with the aid of machine vision than by the gastroenterologist alone,

especially as the day goes on, later in the day, interestingly.

We don't know whether picking up all these additional polyps

changes the natural history of cancers,

but it tells you about machine eyes,

the power of machine eyes.

Now that was interesting.

But now still with deep learning models, not transformer models,

we've seen and learned that the ability

for computer vision to pick up things that human eyes can't see

is quite remarkable.

Here's the retina.

Picking up the control of diabetes and blood pressure.

Kidney disease.

Liver and gallbladder disease.

The heart calcium score,

which you would normally get through a scan of the heart.

Alzheimer's disease before any clinical symptoms have been manifest.

Predicting heart attacks and strokes.

Hyperlipidemia.

And seven years before any symptoms of Parkinson's disease,

to pick that up.

Now this is interesting because in the future,

we'll be taking pictures of our retina at checkups.

This is the gateway to almost every system in the body.

It's really striking.

And we'll come back to this because each one of these studies

was done with tens or hundreds [of] thousands of images

with supervised learning,

and they’re all separate studies by different investigators.

Now, as a cardiologist, I love to read cardiograms.

I've been doing it for over 30 years.

But I couldn't see these things.

Like, the age and the sex of the patient,

or the ejection fraction of the heart,

making difficult diagnoses that are frequently missed.

The anemia of the patient, that is, the hemoglobin to the decimal point.

Predicting whether a person,

who's never had atrial fibrillation or stroke

from the ECG,

whether that's going to likely occur.

Diabetes, a diagnosis of diabetes and prediabetes, from the cardiogram.

The filling pressure of the heart.

Hypothyroidism

and kidney disease.

Imagine getting an electrocardiogram to tell you about all these other things,

not really so much about the heart.

Then there's the chest X-ray.

Who would have guessed that we could accurately determine

the race of the patient,

no less the ethical implications of that,

from a chest X-ray through machine eyes?

And interestingly, picking up the diagnosis of diabetes,

as well as how well the diabetes is controlled,

through the chest X-ray.

And of course, so many different parameters about the heart,

which we could never,

radiologists or cardiologists, could never be able to come up

with what machine vision can do.

Pathologists often argue about a slide,

about what does it really show?

But with this ability of machine eyes,

the driver genomic mutations of the cancer can be defined,

no less the structural copy number variants

that are accounting or present in that tumor.

Also, where is that tumor coming from?

For many patients, we don’t know.

But it can be determined through AI.

And also the prognosis of the patient,

just from the slide,

by all of the training.

Again, this is all just convolutional neural networks,

not transformer models.

So when we go from the deep neural networks to transformer models,

this classic pre-print,

one of the most cited pre-prints ever,

"Attention is All You Need,"

the ability to now be able to look at many more items,

whether it be language or images,

and be able to put this in context,

setting up a transformational progress in many fields.

The prototype is, the outgrowth of this is GPT-4.

With over a trillion connections.

Our human brain has 100 trillion connections or parameters.

But one trillion,

just think of all the information, knowledge,

that's packed into those one trillion.

And interestingly, this is now multimodal with language, with images,

with speech.

And it involves a massive amount of graphic processing units.

And it's with self-supervised learning,

which is a big bottleneck in medicine

because we can't get experts to label images.

This can be done with self-supervised learning.

So what does this set up in medicine?

It sets up, for example, keyboard liberation.

The one thing that both doctors, clinicians

and patients would like to see.

Everyone hates being data clerks as clinicians,

and patients would like to see their doctor

when they finally have the visit they've waited for a long time.

So the ability to change the face-to-face contact

is just one step along the way.

By having the liberation from keyboards with synthetic notes

that are driven, derived from the conversation,

and then all the downstream normal data clerk functions that are done,

often off-hours.

Now we're seeing in health systems across the United States

where people, physicians are saving many hours of time

and heading towards ultimately keyboard liberation.

We recently published, with the group at Moorfields Eye Institute,

led by Pearse Keane,

the first foundation model in medicine from the retina.

And remember those eight different things that were all done by separate studies?

This was all done with one model.

This is with 1.6 million retinal images

predicting all these different outcome likelihoods.

And this is all open-source,

which is of course really important that others can build on these models.

Now I just want to review a couple of really interesting patients.

Andrew, who is now six years old.

He had three years of relentlessly increasing pain, arrested growth.

His gait suffered with a dragging of his left foot,

he had severe headaches.

He went to 17 doctors over three years.

His mother then entered all his symptoms into ChatGPT.

It made the diagnosis of occulta spina bifida,

which meant he had a tethered spinal cord that was missed by all 17 doctors

over three years.

He had surgery to release the cord.

He's now perfectly healthy.

(Applause)

This is a patient that was sent to me,

who was suffering with, she was told, long COVID.

She saw many different physicians, neurologists,

and her sister entered all her symptoms after getting nowhere,

no treatment for long COVID,

there is no treatment validated,

and her sister put all her symptoms into ChatGPT.

It found out it actually was not long COVID,

she had limbic encephalitis, which is treatable.

She was treated, and now she's doing extremely well.

But these are not just anecdotes anymore.

70 very difficult cases

that are the clinical pathologic conferences

at the New England Journal of Medicine

were compared to GPT-4,

and the chatbot did as well

or better than the expert master clinicians

in making the diagnosis.

So I just want to close with a recent conversation with my fellow.

Medicine is still an apprenticeship,

and Andrew Cho is 30 years old,

in his second year of cardiology fellowship.

We see all patients together in the clinic.

And at the end of clinic the other day,

I sat down and said to him,

"Andrew, you are so lucky.

You're going to be practicing medicine in an era of keyboard liberation.

You're going to be connecting with patients

the way we haven't done for decades."

That is the ability to have the note

and the work from the conversation

to derive things like pre-authorization,

billing, prescriptions, future appointments --

all the things that we do,

including nudges to the patient.

For example, did you get your blood pressure checks

and what did they show

and all that coming back to you.

But much more than that,

to help with making diagnoses.

And the gift of time

that having all the data of a patient

that's all teed up before even seeing the patient.

And all this support changes the future of the patient-doctor relationship,

bringing in the gift of time.

So this is really exciting.

I said to Andrew, everything has to be validated, of course,

that the benefit greatly outweighs any risk.

But it is really a remarkable time for the future of health care,

it's so damn exciting.

Thank you.

(Applause)