ADHD & Digital Medicine (Apps & Games)

Summary

In this video, Dr. Russ Barkley discusses the role of digital medicine, particularly the Endeavor video game, in managing ADHD. He highlights the limited evidence base for such apps and games, cautioning against their use as a primary treatment. Barkley critiques the FDA's lower standards for approving digital devices compared to drugs and emphasizes the need for independent research to validate claims of effectiveness. He concludes that while digital medicine shows promise, it is not yet ready for prime time as an evidence-based treatment for ADHD.

Takeaways

• 😀 Digital medicine, including apps and video games like Endeavor, is being used for ADHD management, but the evidence for their effectiveness is limited.
• 📱 Companies have created games and apps for smartphones that claim to improve attention, impulse control, and working memory through practice.
• 💡 Some websites offer a variety of cognitive games for a monthly fee, but they cannot claim to treat diagnosed mental or medical disorders, only to improve cognitive abilities that might be impaired.
• 🚫 The FDA approval for a device like a digital medicine app is not as stringent as for a drug, focusing more on safety than efficacy.
• 🔍 A review found that there are hundreds of apps for ADHD or related cognitive deficits, but most lack research to support their claims.
• 🎮 Endeavor is a video game designed to improve ADHD symptoms, with a version called Endeavor RX that has FDA approval and another called Endeavor OTC.
• 📊 A 2020 study on Endeavor RX showed improvement in lab tests measuring attention but did not demonstrate real-world behavioral changes.
• 🤔 The concept of 'near transfer' refers to improvement in lab measures similar to the game practice, which is not as important as 'far transfer' that extends into real life.
• 🧐 A subsequent trial of Endeavor showed improvement in impairment ratings but lacked a placebo control and did not convincingly demonstrate far transfer.
• 💡 The speaker does not recommend apps or games for ADHD as first-line treatments due to the weak evidence base and suggests buyer beware.
• 👎 The speaker expresses concern about relabeling games as digital therapeutics, as it may be misleading and implies medical treatment without sufficient evidence.

Q & A

What is the main topic of the video by Russ Barkley?

The main topic is the role of digital medicine, particularly focusing on the Endeavor video game's effectiveness for the management of ADHD.

What has been the trend since the invention of smartphones in relation to mental health apps?

Since the invention of smartphones, companies have been creating apps, usually games, that individuals can practice on, with the belief that this practice leads to improved attention, impulse control, and working memory.

What is the difference between the claims websites like lumosity.com can make and those they are not allowed to make?

Websites like lumosity.com are not allowed to claim that they treat mental health disorders or medical disorders. However, they can claim to improve cognitive abilities that might be impaired in various diagnosed mental or medical disorders.

What is the FDA's stance on approving apps for mental health treatment?

The FDA has a less strict set of standards for approving devices like apps compared to drugs. The main concern is safety, and there must be some evidence of efficacy.

What is the current evidence base for apps and games designed for ADHD?

The evidence base is extremely limited, with almost no research on most of these apps. The few that have research are often funded by the companies that manufacture and market the apps.

What is the Endeavor RX and how does it differ from Endeavor OTC?

Endeavor RX is a version of the video game that requires a prescription and is FDA approved. Endeavor OTC, on the other hand, is over-the-counter and does not require a prescription.

What is the primary concern Russ Barkley has about the Endeavor RX study published in 2020?

The primary concern is that the study relies on a lab test (TOVA) for measuring attention and inhibition, which may not correlate significantly with real-world behavior changes in ADHD symptoms.

What is meant by 'near transfer' and 'far transfer' in the context of digital medicine?

Near transfer refers to improvements on lab measures similar to what is practiced in the game. Far transfer refers to whether the results of the treatment extend into real-life settings, such as home, school, work, and social functioning.

What does Russ Barkley suggest about the evidence for the Endeavor program's effectiveness in real life?

Russ Barkley suggests that the evidence for the Endeavor program's effectiveness in real life is thin and not convincing, as the studies lack rigorous examination of far transfer using randomized trials with placebo controls.

What is the general recommendation from Russ Barkley regarding the use of digital medicine apps for ADHD?

Russ Barkley does not recommend digital medicine apps or games as frontline treatments for ADHD due to the weak evidence base and suggests caution, advising consumers to 'buyer beware' until more independent research is available.

What is the issue with relabeling a game as a digital therapeutic according to Russ Barkley?

Relabeling a game as a digital therapeutic can be misleading and deceptive to patients, as it implies a medical or digital treatment of the disorder without sufficient evidence to support such claims.

Outlines

00:00

😀 Digital Medicine and ADHD Apps

Dr. Russ Barkley discusses the role of digital medicine, particularly focusing on the Endeavor video game designed for ADHD management. He explains that while apps and games claim to improve attention, impulse control, and working memory, their efficacy is often overstated due to lax FDA approval standards for devices compared to drugs. Barkley emphasizes that the evidence base for these apps is weak, with most lacking research support, and he personally does not recommend them due to the limited evidence.

05:01

🔍 Scrutinizing the Evidence for Endeavor RX

This section critiques the evidence supporting the Endeavor RX program, a digital medicine application for ADHD. Barkley examines a 2020 study, noting its methodological strengths such as random assignment and double-blind measures but criticizes its reliance on lab tests that poorly correlate with real-world behavior changes. He argues that improvements on the TOVA test, the study's primary outcome, do not necessarily translate to real-life benefits. Barkley also introduces the concepts of near and far transfer, expressing skepticism about the generalizability of the game's effects to everyday life.

10:02

🤔 The Limitations of Current ADHD Digital Treatments

Barkley continues his discussion by highlighting the lack of convincing evidence for digital treatments like the Endeavor program. He describes a subsequent trial that showed improvements on an impairment rating scale but lacks a placebo control, casting doubt on the game's effectiveness. Barkley points out common issues in studies funded by manufacturers, such as the absence of independent replications and the failure to demonstrate far transfer to real-life settings, concluding that these games are not ready for prime time as evidence-based treatments.

15:03

🛑 A Word of Caution on Digital Medicine for ADHD

In the final paragraph, Barkley advises against considering digital medicine apps like Endeavor as frontline treatments for ADHD due to the thin evidence base. He references a review by Andrew Schuman, which also expresses caution regarding health apps, suggesting they may be recommended as ancillary treatments at best. Barkley personally does not recommend these apps and games for ADHD, advocating for a wait-and-see approach until more rigorous research is conducted. He also addresses his discomfort with the relabeling of games as digital therapeutics, advocating for truth in advertising.

Keywords

💡Digital Medicine

Digital medicine refers to the use of digital technologies, such as apps and video games, to improve health outcomes. In the video, it is discussed in the context of managing ADHD symptoms, where the Endeavor video game is presented as an example of this approach. The script mentions that while digital medicine is a growing field, the evidence for its effectiveness, especially in treating ADHD, is limited and requires more rigorous research.

💡ADHD

Attention Deficit Hyperactivity Disorder (ADHD) is a neurodevelopmental disorder characterized by symptoms such as inattention, impulsivity, and hyperactivity. The video discusses the use of digital medicine, specifically the Endeavor video game, as a potential management tool for ADHD. The script raises concerns about the efficacy of such tools in improving real-world behavior associated with ADHD.

💡Cognitive Abilities

Cognitive abilities encompass various mental processes such as attention, memory, and problem-solving. The video script mentions that digital medicine, including apps and games, claims to improve cognitive abilities that might be impaired in diagnosed mental or medical disorders, such as ADHD. However, the script also highlights the lack of evidence supporting these claims.

💡FDA Approval

The U.S. Food and Drug Administration (FDA) approval is a regulatory process that ensures the safety and efficacy of medical products. In the script, it is mentioned that some companies have sought FDA approval for their digital medicine products, but the standards for approving a device are less stringent than for drugs. This is important because consumers may mistakenly believe that FDA approval signifies a high level of evidence for effectiveness.

💡Endeavor RX

Endeavor RX is a specific video game designed to improve symptoms of ADHD in children. It has received FDA approval and is mentioned in the script as an example of a digital medicine product. The video discusses the limited evidence supporting its effectiveness and the need for more research to establish it as a reliable treatment for ADHD.

💡Evidence-Based Treatment

Evidence-based treatment refers to healthcare practices that are supported by scientific research demonstrating their effectiveness. The script emphasizes the importance of evidence-based treatments for ADHD and expresses skepticism about the current evidence supporting the use of digital medicine, like the Endeavor game, as a first-line treatment.

💡Near Transfer

Near transfer is a concept in learning where skills or knowledge gained in one context are applied to another context that is similar. The script discusses near transfer in the context of digital medicine, noting that while the Endeavor RX game may show improvements in lab settings similar to the game, this does not necessarily translate to real-world improvements in ADHD symptoms.

💡Far Transfer

Far transfer refers to the application of learned skills or knowledge to contexts that are significantly different from the original learning situation. The video script argues that for digital medicine to be effective, it should demonstrate far transfer, meaning that improvements seen in a game or app should extend to real-life settings and behaviors associated with ADHD.

💡Randomized Controlled Trial

A randomized controlled trial (RCT) is a type of scientific experiment that is considered the gold standard for testing the effectiveness of medical treatments. The script mentions an RCT conducted on the Endeavor RX program, highlighting the need for such rigorous studies to provide reliable evidence for the effectiveness of digital medicine in treating ADHD.

💡Practice Effects

Practice effects refer to improvements in performance on a test or task due to repeated exposure to the same or similar tasks, rather than due to the intervention being studied. The script discusses how practice effects can influence the results of studies on digital medicine, potentially leading to overestimations of the effectiveness of a treatment like the Endeavor game.

💡Placebo Control

A placebo control is a group in an experiment that receives a fake or inactive treatment to compare against the group receiving the actual treatment. This helps to determine if any observed effects are due to the treatment or other factors. The script points out the absence of a placebo control in some studies of digital medicine, which is a limitation in assessing the true effectiveness of such treatments.

Highlights

Digital medicine, including video games like Endeavor, is being used for ADHD management, aiming to improve attention, impulse control, and working memory.

Smartphone apps and online games claim to enhance cognitive abilities, but cannot advertise as treatments for diagnosed mental health disorders.

FDA approval for digital medicine devices is less stringent than for drugs, focusing more on safety than efficacy.

There are over 300 apps for ADHD or related cognitive deficits, but most lack substantial research support.

Studies on apps like Endeavor often lack real-world evidence, relying on lab tests that may not correlate with actual behavior changes.

Endeavor RX, an FDA-approved version for children, and Endeavor OTC are examples of digital medicine, but their efficacy is still under scrutiny.

A 2020 study on Endeavor RX showed improvement in lab tests but did not demonstrate real-world behavioral changes.

The importance of distinguishing between 'near transfer' in lab settings and 'far transfer' to real-life situations is highlighted.

A subsequent trial of Endeavor in 2021 showed some improvement in impairment ratings but lacked a placebo control for comparison.

Practice effects on rating scales can lead to improvements even without intervention, questioning the validity of some study results.

The need for independent replications of studies on digital medicine apps and games is emphasized for better reliability.

Dr. Barkley does not recommend apps or games as frontline treatments for ADHD due to the weak evidence base.

A review by Andrew Schuman on mental health apps suggests caution, recommending apps only as ancillary treatments with thin evidence.

The relabeling of games as 'digital therapeutics' can be misleading and is a point of contention for some professionals.

More research is needed on digital medicine apps to establish their efficacy and role in ADHD treatment.

Consumers are advised to be cautious, as many companies capitalize on the lack of consumer knowledge about FDA approval standards for devices.

Transcripts

hey friends Russ Barkley here welcome

back in this short video I want to talk

about the role of digital medicine

generally and specifically the Endeavor

video game for management of

ADHD ever since the invention of

smartphones we have seen companies move

into the space where they create uh apps

usually games where individuals can

practice on the game and It is believed

that this practice leads to improved

attention improved impulse control uh

and improved working memory now depends

on what app you have as to which or all

three of those might be included in the

app uh besides these apps that can be

used on Smart devices you can also go

out on the internet to several websites

that allow you to for a monthly fee

practice on a variety of these games now

now the thing about these websites such

as lumosity.com just to name one uh is

that they are not allowed to make claims

that they treat mental health disorders

that is diagnosed disorders or even

medical disorders specifically but they

can claim to improve cognitive abilities

that might be impaired in various

diagnosed mental or medical disorders uh

and it's kind of the same with these

apps although they're not as well

regulated uh some of the companies that

make these apps have gone to the FDA to

get their approval for their device

understand however that the guidelines

for that FDA approval for a device are

not as strict or as high a set of

standards as if you were asking the FDA

to approve to approve uh a drug uh and I

I think that's important because

oftentimes companies trade on the fact

that consumers don't know about that

difference and when they see that uh a

game or an app has FDA approval uh they

think that it has met a very high bar

for standards of evidence for

Effectiveness and safety uh and in fact

that's not the case the bar is pretty

low there has to be some evidence of

some efficacy but it's mainly issues of

safety that the f da is most concerned

about here with a device that is so uh

we've got all these apps out there at

last count one review I looked at a

while ago counted 300 apps for either

ADHD specifically or for ADHD related

cognitive deficits more generally such

as working memory for instance um and

the the problem is as this review found

is that there's almost no research on

most of the these apps uh and the few

that have research might have a paper or

two often funded by the company that's

manufacturing and um marketing the app

uh and so what's a consumer to to do

with these things right now the evidence

base is so weak personally I don't

recommend any of these apps or games for

ADHD or even any of the websites just

because of the extremely limited

evidence base for them out there uh I'm

going to uh use the Endeavor gaming

program as an example of digital

medicine uh not because they're any

better or any worse than any others but

because they're the one that was in my

newsfeed twice this week when I was

looking at the trade media uh Endeavor

is a video game played on a smart device

that uh is supposedly improve symptoms

of inattention

impulsivity uh other related symptoms of

ADHD and there is a version for children

called Endeavor RX uh that's the one

that has the FDA approval and then there

is an overthe counter called Endeavor

OTC that doesn't require a prescription

uh but both of these require a fee uh

and in the case of endeavor RX uh also a

prescription

in order to access the game uh so very

clever there because it makes it sound

like this is kind of a drug it's digital

medicine it's FDA approved and therefore

it must meet a high bar of evidence and

safety and don't be misled by that

that's that's not the case it's just a

device and devices uh can pass through

FDA approval much much more easily than

drugs as I've already said so uh

Endeavor is this video game that you

play That's supposed to improve these

symptoms as I've mentioned uh now what

is the evidence for this uh obviously

these trade articles uh interview

company representatives and they claim

that it does have evidence for

improving uh in atttention uh improving

impairment uh and other symptoms related

to ADHD but but is that really the case

uh let's take a closer look one of the

first papers

on the Endeavor RX program uh which is

manufactured by the company uh achil uh

is this article that appeared in 2020

that was a uh randomized trial of their

software for children ages 8 to 12 uh

with ADH so uh it's a reasonably good

study it involve random assignments uh

the improvements uh or that is the

measures were collected in a double

blind fashion the groups went through

treatment in parallel fashion uh and so

it's a good controlled trial with

pediatric patients uh and all of these

patients had to have diagnosed ADHD

confirmed with clinical

diagnosis uh now what's what's the

problem with this paper which claimed to

have found Improvement uh in lab test

measuring attention uh and inhibition

the lab test they used was the Tova the

tested variables of attention uh and

they assessed individuals at several

time points across the trial the primary

outcome measure for the trial uh was

changes in the Tova score uh and in this

case the attention performance score

from pre to post uh intervention and

then they also looked at safety side

effects and so on but um so you know in

in terms of the rigor of the trial is

pretty good however the problem here is

that it's using a lab test of ADHD and

you've heard me say in other videos

particularly my one on the role of

neuropsychological testing for

diagnosing ADHD that lab measures are

not very good for diagnosing

ADHD or for monitoring change in

treatment that reflect changes in real

world Behavior now why is that because

these tests don't correlate

significantly with rating scales of

realworld ADHD symptoms or of executive

functioning so we don't know what

they're predicting with regard to what's

changing out there in real life so you

could get a change on a lab test that

would not necessarily be reflected in

any any changes out there in real life

as assessed by parents teachers or in

the case of adults uh self- ratings uh

and so it to me uh it kind of

obscures the issue when all you give is

a lab test so yes they did find

Improvement um as assessed on the Tova

measure of attention however if you look

into the study results you look deeply

into the methods they did collect

ratings of ADHD symptoms uh and other

aspects of daily life functioning and if

you look at the results they didn't find

anything what they found was

improvements on the TOA and since that

was their primary measure they get to

claim that this was a successful trial

but what we want to know as clinicians

and as clients is did it change their

life uh and at this point the study does

not

suggest real world changes now here's

where we need to talk about something

researchers talk about when you design a

game or an intervention like this you

can look at what is called near transfer

that is do you improve on measures in

the lab that are similar to what you're

practicing in the game in this sense the

Endeavor RX provant shows some near

trans

but that's not real important as I've

said because measures in the lab often

don't correlate with real life so the

next thing we want to know is far

transfer does the results of the

treatment program extend into real life

natural settings home school work social

functioning and so on and we didn't see

that in this trial so that's very

typical

of studies of these apps for instance

cogmed was a very popular app over a

decade ago for training working memory

in people with ADHD uh it's even more

popular in Europe than it was in the us

at the time but a number of studies were

done on

cogmed that found that all you got you

got better at the game you got better at

measures in the lab similar to the game

but there was no convincing evidence

that it extended into real life

situations so I don't hear an awful lot

about cogmet anymore uh at least here in

the US uh but I do hear more about this

particular digital medicine application

and that's the Endeavor program So based

on that one trial uh I for one would say

well it seems to have some promise at

least on lab measures but we need to see

that there's benefits in real life and

we didn't see it there uh subsequently

another trial was published in the

journal npj digital medicine uh that

involved this game uh and in this case

it was given to kids who were on

medication and kids who were not on

medication and in this time they did use

a impairment rating scale and they

looked at before treatment and four

weeks into treatment

uh in playing this game and what they

found was that there was Improvement in

the impairment ratings over that time in

both the treated the stimulant treated

and the non-treated group um so that's

interesting but again to a scientist

like myself it it's kind of a a so what

finding because we know that if all you

do is measure something with a rating

scale or a test

before treatment and after the treatment

you often see

Improvement and it may have nothing to

do with the treatment it may have to do

with contact with professionals it may

have to do with increased attention from

professionals or in this case parents

that are encouraging their children to

play the game uh it may have to do with

practice effects on these rating scales

we've known for decades that if you give

a rating scale twice to anybody even if

you don't do anything the second score

is often significantly lower than the

initial score so there's practice

effects on the rating scales and so for

that reason researchers like studies

like this to give the scale once before

treatment give it again just before

treatment and then after treatment and

then you've gotten rid of the majority

of practice effects though not entirely

there continue to be improvements on

rating scales with repeated applications

of the scale even when there's no

intervention but the largest jump occurs

between time one and time two now the

second thing you need to do in a study

like this is you need to have an active

Placebo condition which this second

study didn't have it's just comparing

the patients before and after uh it it

does show that the game add something to

people who are on medication but again

that's not convincing because there's

no comparison Placebo

intervention where we might have found

the same effect with a placebo game

being given to these individuals that

doesn't Target attention and uh other

aspects of of ADHD so again there's a

couple trials out there they're kind of

interesting to a researcher they're not

real rigorous when it comes to looking

for far transfer using randomized trials

with Placebo

controls and both ratings and if you

want lab measures as well but the

naturalistic measures are much more

important now I know that's gotten into

the weeds a little more than you like uh

but it does exemplify what's happening

in the space called digital medicine

where games and apps are being used and

are being advertised for use in

improving ad PhD in patients and what

what do we have uh most apps and games

have no evidence whatsoever other than

the claims of the manufacturer even

where there are studies as in the case

of the Endeavor program by achil these

studies were funded by the manufacturer

the marketing company for the achil app

uh and so one has to at least say CAU

iously buyer beware until we get

independent

replications of these apps games and

specifically the Endeavor program these

are not something that I would recommend

as Frontline first line treatments for

ADHD uh again as I said about neuro

feedback earlier this week uh if you've

tried the other things and you're still

not satisfied with the other treatments

that are evidence-based and you've got

Expendable income and you want to play a

game um okay fine nobody's going to talk

you out of doing that but if you're

claiming that these games and apps are

equivalent to the science-based

interventions for ADHD uh then I'm sorry

there's just no evidence-base to support

that kind of definitive conclusion about

these games or apps uh now there was

recently in a mental health newsletter a

nice review by Andrew Schuman on mental

health health apps and should Mental

Health Providers recommend them and they

also talk here about the Endeavor app uh

and the conclusion of this article uh is

uh

that the first of all we have to be

assured that the patient downloads and

uses the app routinely second it's

helpful if benefits are seen right away

because it encourages people to continue

with the game but beyond that as he

points out the evidence base for these

apps uh is is pretty thin so uh you can

recommend them if you wish as ancillary

treatments if I can even call them

treatments but I certainly don't

recommend them to patients myself uh or

uh even as you know fun things to play

that might have some benefits at this

point now just as it aside I have a lot

of trouble with

relabeling a game as a digital

therapeutic it is just a game I think

the relabeling uh is a bit uh confusing

to patients a bit deceptive it implies

that there's something about this game

uh that leads to some kind of medical or

or digital treatment of the of the

disorder uh so so watch out for that

again it's just a personal preference of

mine I I you know to me it should be

truth and advertising do games help

people with ADHD and and don't relabel

this as a digital medicine uh which some

of these uh promoters have so uh so

there you have it right now I think this

is not ready for prime time it's not an

evidence-based treatment the evidence

for most apps is wanting the evidence

for this particular game or app is

pretty thin as well and to a scientist

like me not especially convincing so

time will tell we need a lot more

research on these apps as Schuman talks

about in his uh article editorial if you

will here in contemporary Pediatrics so

that's the status of the field everybody

as always buy or beware thanks for

joining me don't forget to subscribe if

you're not a subscriber and if you like

this material please recommend us to

others uh otherwise I hope you found

this video informative thanks again

everybody see you next time be well