History's Worst Software Error
Summary
TLDRThe script recounts the tragic history of the Therac-25, a radiation therapy machine that caused severe injuries and deaths due to software errors. It highlights the importance of rigorous software testing and ethical decision-making in medical technology. The Therac-25's malfunctions, including a deadly bug known as 'Malfunction 54', led to the machine's withdrawal and the dissolution of the manufacturer's medical division.
Takeaways
- 😔 Katie Yarborough was the first victim of the Therac-25, a radiation therapy machine with fatal software flaws.
- 🔬 Therac-25 was a state-of-the-art linear accelerator designed for cancer treatment, but it was primarily controlled by software.
- 💡 The concept of radiotherapy involves using high-energy radiation to target and destroy cancer cells.
- 🛠️ Therac-25 was developed by AECL Medical and was smaller and more software-dependent than its predecessors.
- 🚫 A critical oversight in the 1983 safety analysis excluded any evaluation of the Therac-25's software.
- 🔍 The software, based on the older Therac-20 model and written by a single individual, had significant errors that were not addressed.
- 🚨 Multiple patients were severely injured or killed due to software malfunctions that caused overdoses of radiation.
- 🤖 Therac-25's software had a critical flaw known as 'Malfunction 54', which allowed unfiltered radiation beams to hit patients.
- 🛑 AECL initially denied the possibility of overdoses and did not acknowledge the software issues until forced by evidence and lawsuits.
- 🔄 Therac-25's design relied on software without hardware fail-safes, leading to a series of tragic accidents.
- 📚 The Therac-25 case is now a cautionary tale in ethics and computer science, highlighting the importance of rigorous software testing and safety measures.
Q & A
What was the name of the woman who experienced the first known software-related accident in the medical field?
-Katie Yarborough.
What was the medical device involved in the accidents described in the script?
-The device involved was the Therac-25, a linear accelerator used for cancer treatment.
What was the year when Katie Yarborough's accident with the Therac-25 occurred?
-Katie Yarborough's accident occurred in 1985.
How did the Therac-25 work in terms of delivering radiation to patients?
-The Therac-25 worked by using a double pass accelerator to direct high-energy electrons and/or X-rays into patients' lymph nodes.
What was the estimated radiation dose Katie Yarborough received instead of the prescribed 200 rads?
-Katie Yarborough received an estimated dose of 20,000 rads, which was hundreds of times more than the prescribed amount.
What was the name of the company that developed the Therac-25?
-The Therac-25 was developed by AECL Medical, a division of Atomic Energy of Canada Limited.
What was the main issue with the Therac-25's software that led to the accidents?
-The main issue with the Therac-25's software was the lack of proper safety checks and the potential for errors like arithmetic overflow, which allowed unfiltered beams of radiation to strike patients.
What was the 'Malfunction 54' error that was repeatedly mentioned in the script?
-Malfunction 54 was an undefined error in the Therac-25's software that, when encountered, allowed the machine to deliver a powerful, unfiltered beam of radiation to patients, causing severe injuries or death.
What was the corrective action plan (CAP) that AECL eventually submitted to the FDA after the accidents?
-The corrective action plan included 23 software changes and six hardware safety features, including a dose per pulse monitor to shut down dangerous doses even if all software safety checks failed.
What was the final outcome for AECL Medical after the Therac-25 accidents?
-AECL Medical dissolved their medical division in 1988, and lawsuits from the families of the victims were settled out of court.
How is the Therac-25 incident viewed today in the context of medical technology and ethics?
-Today, the Therac-25 incident is considered a staple of ethics and computer science classes as a case study of what can go wrong when new technology is trusted implicitly and ethical decision-making fails.
Outlines
😨 The Tragic Beginning of the Therac-25 Story
The script begins with the tragic story of Katie Yarborough, a 61-year-old manicurist who underwent her 12th cancer treatment using the Therac-25 at Kennestone Regional Oncology Center. Instead of painlessly receiving 200 rads of radiation, she experienced a severe burn. Despite the technician's reassurances, Yarborough suffered extensive damage, including the need for a mastectomy and paralysis of her left arm. The incident marked the beginning of what would be recognized as some of the worst software-related accidents in history. The script then explains the concept of radiotherapy and its evolution into a sophisticated field using advanced imaging and targeted radiation beams. The Therac-25, developed by AECL Medical, was a revolutionary machine that used software to streamline radiation delivery. However, the safety analysis conducted in 1983 overlooked the software's potential flaws, which were inherited from the older Therac-20 model and written by an unidentified coding hobbyist.
🤕 The Escalation of Therac-25 Malfunctions and Their Consequences
The script continues to describe the escalating series of malfunctions with the Therac-25, which resulted in severe radiation overdoses for patients. It recounts the experiences of a 40-year-old woman with cervical cancer who suffered a similar fate to Yarborough, and the subsequent investigation by AECL, which initially dismissed the possibility of an overdose. The narrative details how the Therac-25's software was error-prone, with operators encountering multiple error messages daily and pressing 'proceed' without fully understanding the implications. The script also describes the tragic cases of other patients, including Voyn Ray Cox and Vernon Kidd, who died as a result of the machine's malfunctions. It highlights the challenges in identifying the root cause of the malfunctions, particularly 'Malfunction 54,' and the slow response from AECL to address the issues.
🔍 Unraveling the Mystery of Malfunction 54
This section delves into the investigation of Malfunction 54, which was responsible for delivering lethal doses of radiation to patients. The script describes how the Therac-25's software and hardware were designed without adequate safety checks, allowing for the possibility of unfiltered radiation beams to strike patients. It explains the eight-second delay in the machine's magnets moving into position and how operator actions within this timeframe could lead to critical errors. The script recounts the efforts of Dr. Fritz Hager and a technician to recreate Malfunction 54, ultimately discovering the issue and prompting the FDA to declare the Therac-25 defective and demand corrective action from AECL.
🚫 The Continuing Tragedy and the Final Resolution
Despite identifying and attempting to correct Malfunction 54, the script reveals that accidents with the Therac-25 continued, resulting in further fatalities. It discusses the discovery of another software error related to an 'overflow' condition that led to the death of Glenn Dodd. The FDA once again declared the Therac-25 defective and demanded all units be taken out of service. The script outlines the final corrective action plan submitted by AECL, which included significant software and hardware changes, and the eventual dissolution of AECL's medical division in 1988. The Therac-25 is now remembered as a cautionary tale in ethics and computer science classes, highlighting the dangers of trusting new technology without proper oversight and testing.
🏥 The Legacy of the Therac-25 and Lessons Learned
The final paragraph reflects on the legacy of the Therac-25 and the lessons learned from its tragic history. The script emphasizes the importance of ethical decision-making in technology development and the need for rigorous testing and documentation of medical software. It also notes the changes in regulatory requirements by the FDA for new medical products, which now include independent investigation of software documentation. The script concludes with a quote from the lawyer for the first Therac victim, highlighting the irony of such sophisticated machines causing harm due to a lack of common sense in their operation and safety measures.
Mindmap
Keywords
💡Therac-25
💡Radiation Therapy
💡Software Malfunction
💡Linear Accelerator
💡High-Energy Radiation
💡Cancer Treatment
💡Safety Analysis
💡Arithmetic Overflow
💡Ethical Decision Making
💡Medical Malpractice
💡Corrective Action Plan (CAP)
Highlights
Katie Yarborough's tragic experience with the Therac-25, marking one of the worst software-caused accidents in history.
Therac-25, a state-of-the-art linear accelerator, was used for cancer treatment by directing high-energy electrons or X-rays into patients' lymph nodes.
Katie Yarborough's severe injury from an overdose of radiation, leading to the removal of her breast and paralysis of her arm.
Therac-25's innovative double pass accelerator design by AECL Medical, which streamlined the process by sending beams through a target twice.
Therac-25 was primarily run by software, which was a significant shift from previous hardware-dependent accelerators.
Safety analysis of Therac-25 omitted software interrogation, which was based on the older Therac-20 model and written by a single unidentified coder.
The discovery of an exit dose on Yarborough, indicating an electron beam had been used, which was inconsistent with her prescribed dose.
AECL's initial denial of any overdose possibility and their request for the medical physicist to stop making unproven claims.
Subsequent similar accidents with Therac-25, indicating a systemic issue with the machine's software or operation.
The identification of 'Malfunction 54' as a critical software error that allowed unfiltered radiation beams to strike patients.
Operators' testimony of frequent error messages and their practice of proceeding without fully understanding the implications.
Therac-25's reliance on software without hardware interlocks, which removed physical safety mechanisms.
The FDA's declaration of Therac-25 as defective and the demand for a corrective action plan.
AECL's eventual acknowledgment of the software issues and the implementation of 23 software changes and six hardware safety features.
The end of Therac-25's use in hospitals and the dissolution of AECL's medical division due to the high-profile accidents.
Therac-25 now serves as a case study in ethics and computer science classes, emphasizing the importance of responsible technology use and ethical decision-making.
The FDA's current requirement for documentation on all software for new medical products, highlighting the lessons learned from Therac-25.
Transcripts
katie yarborough woke up on a warm clear
june day in 1985 and prepared for her
12th cancer treatment
the 61 year old manicurist got dressed
and drove herself to the kennestone
regional oncology center in marietta
georgia where a state-of-the-art linear
accelerator called the therac-25
would direct high-energy electrons and
or x-rays into her lymph nodes as it had
done for patients in the area thousands
of times before
the therac would need only a few seconds
to painlessly deliver around 200 rads to
her upper left chest
but that day
something went wrong
yarborough felt a red-hot sensation
instead of nothing
you burned me
she told the technician who quickly
assured her that this wasn't possible
over the next few weeks she would need
one breast fully removed and her left
arm would become completely paralyzed
but her useless arm didn't stop her from
living her life or from driving
she died five years later when her car
was struck by a truck
on a georgia highway
katie yarborough was the first victim of
what would be later called some of the
worst software caused accidents in
history
this
is the true story
of the therak-25
how do you treat an insidious and
deep-seated disease like cancer and its
many forms without invasive and
dangerous surgeries
one answer in use for over a hundred
years now is radiation or radiotherapy
the concept is simpler than its name
suggests
radiation in the form of high energy
particles and photons can ionize or
otherwise change atoms and molecules
in a sensitive structure like dna enough
of this damage can lead to the death of
a cell
a disease like cancer progresses through
the unchecked division of cells so why
not try blasting these mutants with
radiation that can by its very nature
pass invisibly through body tissue
what began in earnest only after world
war ii and the first nuclear reactors is
now a highly sophisticated field that
uses three-dimensional body imaging and
targeted beams of radiation from linear
particle accelerators to prevent
halt and otherwise destroy cancer cells
in the best cases radiotherapy is
considered an effective weapon in 4 out
of 10 cancers
no scalpel required
in 1976 aecl medical a division of
atomic energy of canada limited
developed a revolutionary double pass
accelerator which streamlined linear
accelerator designs by using
electromagnets to send beams through a
target twice instead of once
the therak-25 was one such double pass
machine 7 feet high and 12 feet wide
smaller than previous accelerators also
unlike the accelerators of old the
therac-25 was run principally by
software instead of hardware lines of
code instead of interdependent physical
mechanisms
in 1983 aecl performed a safety analysis
on the new machine and started selling
the therak25 to excited customers
this state-of-the-art device was in high
demand
however
left out of that 1983 analysis was any
interrogation of the software that ran
these complicated devices of the code
based on the older therak-20 model and
written by a single person
a coding hobbyist
who left the company in 1986.
he remains unidentified to this day
two weeks after katie yarborough told
her technician that she felt a burning
sensation during her cancer treatment
there was a red mark the size of a dime
on her chest
and directly opposite that mark a larger
disc on her back
tim still the medical physicist at
kennestone examined her
that looks like the exit dose made by an
electron beam
he said
it looked nothing like what could be
created by her prescribed 200 rad dose
the physicist later estimated what
actually hit yarborough was closer to 20
000 rats hundreds of times more than
what you'd receive standing inside a
failed reactor at fukushima daiichi
but dr still wasn't able to recreate a
beam of that strength with the machine
himself so he contacted a professional
organization to tell them what had
happened he quickly got a call from the
aecl in response telling him to stop
making these claims without any proof
they assured him that such an overdose
simply wasn't possible
over the next few weeks the dime-sized
red circle on yarborough's chest became
a hole
skin grafts failed as any new tissue
simply rotted away
her left breast recently cancer-free
had to be entirely removed
her left arm
was now immobile
many sources report it was though a
slow-motion gunshot wound had gone
through her chest and out of her back
yarburo would hire a lawyer and sue the
hospital and aecl in the october of 1985
but she wouldn't live to find out the
reason why nanoscopic bullets
had done this to her
seven weeks later concerningly similar
to katie yarbrough a 40 year old woman
with cervical cancer arrived for her
most recent therak-25 treatment at the
hamilton regional cancer center in
ontario canada
she too was hit with a slow-motion
bullet complaining of tingling electric
shocks during treatment
it would be later estimated that what
the therak operator had mistakenly
irradiated her hip width several times
was a total
of 17 000 rats
a larger dose than what harry dogly and
junior or louis slaughten received
from the demon core
the aecl was informed immediately and
had an engineer dispatch to examine the
unit
the micro switches that controlled the
position of the unit's turntable were
deemed faulty and a software change to
constantly check the turntable position
was introduced
aecl would later claim in a september
letter to customers that this change had
increased the safety of the therac-25 by
five orders of magnitude
but
the cervical cancer patient died a month
before this pronouncement
on november 3rd
her official cause of death was her
cervical cancer
though an autopsy revealed that if she
had lived her hip
obliterated by high energy radiation
would have to have been entirely
replaced
five days later a letter from the
canadian radiation protection bureau
begged aeco for hardware fail-safes and
additional software changes but nothing
came of it
a month after the heavily irradiated
cervical cancer patient died it happened
again
a therak-25 unit at the yakima valley
memorial hospital in washington state
supposed to be now 9 million percent
safer hit another cervical cancer
patient in the hip with more radiation
than what cecil kelly endured when a
whirlpool of plutonium
went prompt critical in his face
thankfully the woman ultimately suffered
only minor disability and scarring
the more impactful outcome was that
doctors and therak-25 operators in the
u.s and canada were now talking to each
other
something was going on that the aecl
obviously wasn't addressing or didn't
care to
two months later aecl declared that
after careful consideration we are of
the opinion that this damage could not
have been produced by a malfunction of
the therac-25 or by any operator error
end quote
however
over the next 12 months
therap-25 malfunction and operator error
would kill three cancer patients
the therac-25 software likely had around
100 000 lines of code
small by today's standards but
complicated nonetheless
and error-prone
operators would later testify that they
encountered as many as 4 serious error
messages a day
many of those errors would simply read
malfunction with a number from 1 to 64.
these numbers were not explained not by
aecl not in any
manual operators also admitted that they
became accustomed to this ambiguity
rather than fearful of it
they could and did simply press p to
proceed
without knowing whether or not an error
code was benign or potentially deadly
it was part of the job to keep an
expensive and sought after machine like
the therak running
malfunction 54 one of these mysterious
undefined errors would turn out to be
the one you couldn't skip past
but it would take three catastrophes
before anyone figured out what they were
allowing to happen
[Music]
voyn ray cox lay beneath the therak-25
unit at the east texas cancer center in
tyler texas for his ninth cancer
treatment a technician set his dose at
180 rads
then she noticed a mistake
she had selected x for x-ray instead of
e for electron beam
she quickly moved the cursor made the
change and activated the machine
malfunction 54.
used to this by now she hit proceed
anyway
mr cox then felt a powerful shock
according to reporting done by barbara
wade rose cox tried to get up but
because the intercom for the room just
happened to be broken that day the
technician didn't see him struggling
or hear him screaming
so she hit him again
another shock ripped through mr cox
the technician only stopped when she
heard cox slamming the door she was
behind with his fists
he was examined by physicians
and sent home
told to return if anything changed
a few weeks later he returned to the
hospital
spitting up blood
[Music]
after the accident no one can reproduce
malfunction 54.
aecl told the hospital that an overdose
was impossible suggesting maybe it was
indeed an electric shock that produced
the sensations mr cox felt but
ruled that out too
the company claimed it knew of no other
similar accidents so the use of the
therak unit resumed 17 days later
voin ray cox died the following august
after receiving a calculated dose higher
than the worst dose a liquidator would
receive when the chernobyl nuclear power
plant exploded
just a month later
only four days after the therac at the
east texas cancer center was back online
66 year old bus driver vernon kidd
walked through the lobby on the way to
his scheduled treatment
he was to have a therak-25 aimed at the
skin cancer on his face
malfunction 54
treatment proceeded
a loud noise brought the technician back
into the room to find mr kidd writhing
in pain
confused
he said it had felt like something hit
him on the side of the face
he saw a flash of light
heard an intense
sizzling sound
the therak-25 unit at the center was
shut down until the cause could be
determined
verdan kidd died a month later from
radiation-induced damage to his brain
and brain stem
his death
four months before the death of voyn ray
cox was the first recorded fatality from
radiation treatment
in medical history
the therac technician on site that day
and physicist dr fritz hager stayed the
weekend after the kid accident
attempting to recreate malfunction 54
the malfunction that aecl said wasn't
possible
they changed the machine's modes moved
the cursor quickly up and down and typed
in different treatment instructions for
hours upon hours and then suddenly
they did it
dr hager telephoned to aecl immediately
the fda already investigating the
accidents declared the therak-25
defective and demanded a corrective
action plan or cap from the company
a letter soon went out from the aecl to
all therac-25 users
quote effective immediately and until
further notice the key used for moving
the cursor back through the prescription
sequence must not be used for editing or
any other purpose
end quote it was something that everyone
had missed and it was finally going to
be fixed
but
even after everyone knew what
malfunction 54 was and how to fix it
the accidents continued
you don't think of software as something
being able to fail
once working code is in a computer how
could it bend like a steel beam or break
like a pane of glass
but like any machine there's a
difference between how it's supposed to
be used in theory and how it's actually
used in practice
the therac-25 used magnets to filter and
control powerful beams of radiation
magnets that after an input was received
physically took eight seconds to move
everything into position
fine in theory
what dr hager had figured out was that
if an operator set radiation levels and
then made a change to those levels
within the eight seconds it took for the
magnets to move
the change was not detected the magnets
were already in motion
this could and did allow powerful
unfiltered beams of radiation to strike
patients
this animation reproduced in spanish
shows the sequence of events needed to
produce malfunction 54.
in theory an operator would make a
change and then wait for the magnets in
the machine to move
but an experienced operator working with
the therac every single day encountering
multiple error messages a shift is in
practice more than likely of making a
change like changing a beam from x-ray
to electron within 8 seconds
there was no code in place to check
whether the prescribed input on the
monitor match what the machine was
actually set up to do
the therac-25's reliance on software and
not hardware interlocks like previous
models
also meant that input errors didn't have
mechanical fail-safes that wouldn't
listen to the mistakes that ended up
fatally irradiating people
with malfunction 54 finally identified
aecl sent the first corrective action
plan to the fda part of which were
changes to the therax software to tell
the machine where the cursor actually
was
the cap was revised twice by the fda
over the next few months and tharac25s
were back in use before the end of the
year
six weeks later
a therak unit killed again
on january 17th 1987 glenn dodd 65
walked into the yakima valley memorial
hospital for treatment of a carcinoma
his disease was to be flooded with 86
rats
the therak instead bombarded the man's
chest with ten thousand
he died from acute radiation poisoning
three months later
the staff at yakima reportedly stopped
using the machine altogether after this
accident they were paranoid they thought
this had been fixed
this was safe
what did they miss
what was going on
eventually it was discovered that the
therac25 had another invisible software
error
inside the code was a so-called
housekeeper task that would constantly
check whether or not the machine's
turntable was in the correct position
make adjustments if necessary and then
revert to zero
anything other than a zero in the code
therefore was an error and the machine
would not proceed with treatment again
good in theory
however like your car's odometer this
code ticked up checks only until a
certain value in this case one byte of
memory
256
after that it would tick over to zero
out of necessity
if a technician entered an erroneous
treatment at this precise moment
just an instant before the next check
the computer would read a zero
no errors
and proceed
this is called arithmetic overflow
and it's what killed glenn dodd
in february 1987 the fda again declared
the therak 25 defective recommended that
all units be taken out of service until
corrective action could be taken
finally the accident stopped
after months of revisions aecl told
customers that the fda had accepted a
final corrective action plan it included
23 software changes and six hardware
safety features
the largest of which was a dose per
pulse monitor affixed to the machine
that would shut down dangerous doses
even if all the software safety checks
failed
however
unsurprisingly
after high profile and unprecedented
accidents
the therak was no more
in 1988 aecl dissolved their medical
division and lawsuits from families were
settled out of court
at the time of this recording with a
cursory search i was unable to find any
hospital actively using a therap-25
even if they are which isn't unlikely
the machines are probably under a
different name or company
even without any further incidents to
point to today
no hospital wants to draw active
comparison
to disaster
today the therac25 is more a staple of
ethics and computer science class
required readings than it is of medicine
a unique case study of what can go wrong
when new technology is trusted
implicitly and when ethical decision
making malfunctions aecl assumed that
the software for the therac-25 written
by a single unidentified hobbyist and
imported from the older therak-20 model
did not have residual software errors to
be tested
it didn't consider how the machine was
being used in practice never envisioning
something like malfunction 54
the company repeatedly denied knowing of
any accidents and believed that
overdoses were impossible
it made proclamations like five orders
of magnitude increases in safety that
were physically impossible
today the fda requires documentation on
all software for new medical products
which the therac-25 didn't have
that can be investigated independently
ten years after the deadliest software
errors in history reporter barbara wade
rose asked bill bird the lawyer for the
first therac victim katie yarborough to
comment on the events
quote
the thing that amazes me
is that the people who develop these
machines are surely some of the most
brilliant people in the world
this machine was unbelievably
sophisticated
and yet
nobody would have gotten hurt
if somebody had just used common sense
until next time
[Music]
[Music]
[Music]
[Music]
you
Weitere ähnliche Videos ansehen
THERAC-25: O PIOR erro de SOFTWARE da HISTÓRIA
A Brief History of: The killer Therac-25 Radiotherapy machine (Short Documentary)
Medical Errors: The Silent Killer in Medicine | Carol Gunn | TEDxFargo
Mengetahui Pengujian Perangkat Lunak || Materi Informatika Kelas 12
Space Shuttle Challenger Disaster: Major Malfunction | Retro Report | The New York Times
Why is it so dangerous to step on a rusty nail? - Louise Thwaites
5.0 / 5 (0 votes)