How A Steam Bug Deleted Someone’s Entire PC

Summary

In 2015, Linux Steam users reported that starting Steam recursively deleted all files owned by their user account. Investigating the Steam startup script revealed a bug where a variable returning the Steam root directory path instead returned nothing under certain conditions, causing 'rm -rf' to wipe system files. Though the exact sequence of user actions triggering this remains unclear, the bug was fixed by adding checks before deleting directories and preventing the script from continuing on failures.

Takeaways

• 😀 This is the first episode of a video series about interesting software issues
• 🤔 The Linux Steam client had a bug that caused it to delete a user's entire filesystem
• 😮 Running 'steam.sh' directly with bash caused STEAMROOT to be an empty string
• 🤨 This empty STEAMROOT variable triggered the 'reset_steam' function
• 😠 'reset_steam' dangerously used 'rm -rf' on STEAMROOT, deleting everything
• 😕 Multiple users experienced total data loss from this bug
• 🧐 The root cause was incorrect handling of the $0 variable in 'steam.sh'
• 🤔 There were debates about the best way to fix this dangerous bug
• 😊 Multiple improvements were made to 'steam.sh' to avoid deleting user data
• ✅ The Steam deletion bug has not resurfaced in many years and is considered fixed

Q & A

What caused Steam to delete user files on Linux?

A bug in the Steam Linux client caused it to run 'rm -rf' on the root directory if the STEAMROOT variable was set to an empty string. This happened if steam.sh was run directly rather than through the steam binary.

How did the 'rm -rf' command end up targeting the root directory?

The STEAMROOT variable contained an empty string instead of the path to the Steam directory. When substitution occurred in the 'rm -rf $STEAMROOT/*' command, it expanded to 'rm -rf /*', targeting the root directory.

What triggered the reset_steam() function that contained the 'rm -rf' command?

The reset_steam() function runs if the INSTALLED_BOOTSTRAP variable is empty. This variable should be set by the STEAMEXE executable, but wasn't run since STEAMROOT was invalid.

How did running 'steam.sh' cause the STEAMROOT variable to be empty?

When run directly, 'steam.sh' fails to get the proper $0 path variable, causing a cd command to fail and STEAMROOT assignment to output nothing.

Did Keyvin actually run the steam.sh script?

It's unclear. Keyvin claimed he only ran the steam binary, but others speculate he may have run steam.sh and forgotten.

What was Keyvin trying to do prior to launching Steam?

Keyvin was trying to move the Steam installation to another drive. He symlinked the original Steam folder to the new location.

How could the bug have been triggered by Keyvin's setup?

If the symlink was created incorrectly or accidentally deleted, it could cause STEAMROOT path resolution to fail when starting Steam.

What were some of the proposed fixes for the bug?

Using readlink/dirname for STEAMROOT, adding checks for valid paths, making steam.sh exit on failures, writing scripts in other languages, and removing use of rm -rf wildcard.

Has the deletion bug issue resurfaced since it was reported?

No, the issue has not reappeared over the past 8+ years since it was originally reported and fixed.

What was the overall impact of this Steam bug?

It caused total data loss for at least two Linux users. Beyond that, the impact seems to have been limited.

Outlines

00:00

😮 How a Steam bug accidentally deleted users' files

Paragraph 1 describes how in 2015, a bug in the Steam Linux client caused it to recursively delete all files when launched in certain ways. It explains the Linux directory structure, how Steam is installed, the steam.sh script, and how passing it incorrectly to Bash caused variable issues leading it to wipe files.

05:03

😲 Narrowing down the exact trigger of the wipe bug

Paragraph 2 analyzes the conditions in steam.sh that trigger the reset_steam() function that wipes files. It explains how running steam.sh directly with Bash causes key variables to be empty, meeting the if statement conditions that run reset_steam().

10:05

😅 Speculation on how the original poster triggered the wipe

Paragraph 3 speculates on how Keyvin, the original poster, might have accidentally triggered the bug even though he only claimed to run the Steam client normally. It suggests race conditions around symlinks as one unlikely theory.

Keywords

💡Steam

Steam is a video game digital distribution service and platform developed by Valve Corporation. It is used by gamers on Linux, MacOS, and Windows to buy, play and organize video games. The bug discussed in the video caused Steam to recursively delete files on a Linux user's computer when run in a certain way.

💡symlink

A symlink, also known as a symbolic link, is a type of file in Linux that contains a reference to another file or directory. It allows you to make a file or directory visible in multiple locations. In the video, a symlink was created to move the Steam installation to another drive, which contributed to the bug.

💡rm -rf

In Linux, 'rm -rf' is a command that recursively deletes files and directories. The 'rm' command removes files and directories, while the '-r' option makes it recursive and '-f' forces deletion without prompting. This dangerous command was unintentionally run by the buggy Steam script, wiping users' drives.

💡Linux filesystem

The Linux filesystem refers to the logical way Linux organizes files and directories on a drive or disk. Understanding the basic structure of the Linux filesystem, like the root directory and home directories, is key to following how the Steam bug was able to delete arbitrary files.

💡environment variable

Environment variables are dynamic values that can affect the behavior of programs and scripts on Linux/Unix systems. In the Steam bug, environment variables were improperly set, causing the Steam script to malfunction and delete files.

💡shell script

A shell script is a simple program written for the Linux/Unix shell. The steam.sh shell script contained the buggy code that lead to the recursive deletion when Steam was started in a particular way.

💡race condition

A race condition arises when the timing or order of events impacts program behavior. In one hypothetical theory, a race condition caused the Steam symlink to be deleted at just the wrong time, triggering the bug.

💡NTFS

NTFS is a proprietary Microsoft Windows filesystem that is also readable on Linux. One theory was that the Steam files being on an NTFS drive contributed to the bug, but this was later ruled out.

💡executable bit

The executable permission bit determines whether a file can be executed and run as a program in Linux. One theory was that the NTFS drive failed to properly set the executable bit on Steam, but this was debunked.

💡bash

Bash is a common Linux shell program used to execute commands and scripts. Running the Steam script directly in bash in a certain way reliably reproduced the recursive delete bug.

Highlights

Steam for Linux accidentally deleted user's entire computer by recursively deleting root directory

Bug caused by steam.sh script changing to invalid working directory and then trying to delete contents of steam root directory

Symlinks used to move Steam folder, but Steam couldn't find new location

steam.sh deletes everything recursively with no checks, very dangerous

STEAMROOT variable gets empty string instead of path due to script being run incorrectly

Script fails when run directly with bash instead of as executable

Bug triggers because bootstrap variable not set after failed launch

Possible NTFS drive issues with executable bit permissions

Multiple recommendations to improve script - use Python/Perl, add checks before deletes, etc.

Race condition theory - symlink deleted after creation causing issues

Fixes implemented - exit on error, verify paths, no recursive deletes

Debate on best practices for scripts and deleting files

Some skepticism about details of original report

Issue caused lots of discussion but hasn't resurfaced since

Funny theory about lasagna dinner before steam install issues

Transcripts

Welcome to the first episode of Issues Insights, a series where we look at some of the most

interesting issues out there, from Github to Gitlab, to proprietary issue trackers,

even to your favorite mailing lists - we’ll cover them all.

We expect to produce at least one episode of this series every 10 years, so buckle up

as we dive into today’s issue from Valve - the Linux version of Steam unintentionally

rm rf-ing or recursively force deleting someone’s entire computer!

Steam for Linux has changed a lot since 2015, but here is how it worked back in the day:

you first install steam, after which you can run the binary file called “steam”

found in the directory /usr/bin to start the client.

This binary does not contain the full steam application

but invokes files in the installation directory called STEAMROOT in each user’s home.

Before getting too deep, let’s take a high level view of the Linux file system, first there

is the root directory, which as we know contains everything.

Then, we see that /usr/bin contains user programs like steam, compared to /bin which usually

contains programs essential for the operating system to function.

The home directory, abbreviated as a ~ and located at /home/name works as you’d

expect and is where each user stores all their stuff.

In each user's home is the steam root directory where Steam installs itself and stores its games.

It also has an important script called steam.sh, which is what the steam binary invokes to

configure and launch steam.

After setting everything up, this script will then invoke the main executable called STEAMEXE.

All of three of these entry points can technically be used to start steam directly, just for

different purposes and with certain limitations.

Anyways, on January 14, 2015, Keyvin opened an issue on the steam-for-linux Github repo

stating the following:

>I moved the steam folder to a drive mounted somewhere else, and symlinked the original

>directory to the new location.

Symlinks or symbolic links are just special files which point to the file or directory

that they are linked to.

These are kind of like shortcuts, except symlinks are completely transparent to the application

layer as the operating system automatically resolves it to the target.

So you can kind of see what Keyvin was trying to do here - move the steam folder to a drive

with better performance or storage capacity, and create this symlink so applications can

still find that folder using the original path.

>Then I launched Steam.

>It did not launch.

>It offered to let me browse and still could not find it when I pointed to the new location.

>Steam crashed.

>I restarted it.

>It re-installed itself and everything looked great.

>Until I looked and saw that steam had apparently deleted everything owned by my user recursively

>from the root directory.

>Including my 3tb external drive I back everything up to.

Doofy then follows up with a reply, having encountered the same issue.

>This is terrible.

>I just lost my home directory.

>All I did was start steam.sh with the STEAM_DEBUG flag enabled.

If you recall, steam.sh is the helper script located in steamroot that can be used to start steam.

After a bit of digging in this script, they found a line which runs rm -rf for all the

contents of the steam root directory.

I think we’re all familiar with what the rm command and its options -rf do.

One line above this command was the comment “Scary” indicating this was a very scary operation.

Soon enough, TcM1911 chimed in with a hypothesis: this steamroot variable must have somehow

returned an empty string, causing the rm -rf argument to be the root directory.

Taking a closer look at STEAMROOT, this enclosing dollar sign and parentheses is a command substitution

where the output of the commands run inside will substitute the expression and thus be

assigned to STEAMROOT.

What is the output?

Well it’s going to be what is echoed or printed, in this case, the current working

directory signified by this PWD environment variable.

The working directory of a script is where it is run from, not where the script is located.

If you execute a script from the /tmp directory, the working directory returned by PWD will

be /tmp regardless of where the script is located.

So the strategy here is to change directory or cd to the directory of the script, and

then print the working directory.

We have another OS provided environment variable zero which refers to the path of the script

being executed.

Then, this percent signifies you want to remove the proceeding pattern from the end of the

pre-ceding string.

The pattern here is a forward slash, followed by a wildcard which matches anything.

So from the preceding string which is the path of the script, we start from the end

and delete the first slash we see, followed by everything after it.

In this case, we delete the name of the file, leaving us with… the STEAMROOT directory.

But this appears to work.

What causes it to return an empty string?

You see, if any part of this command fails prior to the echo, the command substitution

will output nothing and therefore nothing will be assigned to STEAMROOT so it will remain

Rcxdude notes that the command definitely fails in such a way when running the script

directly with bash like so.

Why is this the case?

Normally, when you run a script in Linux by invoking its path, the kernel will detect

that it is an interpreter script, since it starts with a shabang specifying which interpreter to use.

Then, it will invoke the interpreter with the script path as the first argument.

This first argument is what the environment variable zero we talked about previously refers to.

However, bash and similar shells allow you to run a script by passing a plain filename

into the command as long as it’s in the current directory.

This will start a new bash shell and run the script using the filename as the first argument,

which results in a dollar zero variable that is an invalid path.

Here is the phenomenon in action: we run the same script but in the second case, it only

prints a file name, not a valid path.

This causes the directory change command to error out and return no output, setting Steam

root as an empty string, causing rm -rf to delete the root directory’s contents.

At this point, most were satisfied, and began suggesting fixes, discussing how terrible

the code was, and wondering how it was possible for Keyvin to be so cool and collected.

But wait, this recursive deletion is in reset_steam(), why was that called in the first place?

reset_steam() is only invoked under two circumstances.

First, the user can intentionally pass the --reset flag into the script.

Given the bug report did not mention the reset flag, let’s take a closer look at the other case.

All of these conditions need to be met, as indicated by the “and” operator between each condition.

Only one of them is important, the rest is just housekeeping, so let’s get the obvious

ones out of the way first.

Initial launch just verifies there isn’t already a steam process running, so this is easily true.

Next, current status must not be set to this magic exitcode, which is just a variable to

ensure this doesn’t get stuck in an infinite loop.

Next, this steam starting file must exist, and it should exist given the steamconfig

folder has remained untouched, which would include the starting file.

Lastly, it confirms that the steam script is not out of date, which should be expected.

So the key condition which triggers reset_steam() is that this INSTALLED_BOOTSTRAP environment

variable has to be an empty string, the default value for an unset variable.

Before the reset_steam check, steam.sh should run the main executable STEAMEXE which sets

this variable to 1 after installing some bootstrap software.

The developers intended for reset_steam() to trigger if STEAMEXE failed to do this, indicating

something went wrong.

However in this case, nothing was wrong with STEAMEXE, it is just executed with a path

that is based on STEAMROOT, which we’ve already established is set incorrectly as

an empty string due to the other bug.

So steam.sh fails to run the main executable at all since it has the wrong path, the INSTALLED_BOOTSTRAP

environment variable is not exported and remains blank, all the conditions of the

if-statement are met, and then reset_steam() is called.

So now we know how to trigger the bug: run steam.sh inline with a shell like bash.

This explains what happened to Doofy, who clearly ran steam.sh, but how does this play

into Keyvin’s original account, who only mentioned launching the main steam binary?

Well, a valve employee had a hypothesis: Keyvin first moves steam to an NTFS drive, which

being made for Windows doesn’t play nice with the Linux filesystem and would strip

the executable bit in the version of Ubuntu used here.

This causes the steam binary to inevitably fail and crash as it cannot execute anything in the drive.

Then, he gets really annoyed and runs “bash steam sh”, and most importantly, he gets

amnesia and forgets he ever ran such a command.

Keyvin replies that he only ran steam normally, and he wasn’t even sure if that line

was the culprit, which makes sense given from his perspective he never ran steam.sh.

Furthermore, he recalls he may have mounted the drive with options to allow execution

of all files, which is reasonable given that ValveSoftware themselves have tutorials to

mount NTFS drives in this way.

And of course he is still very calm and collected.

However, the exact play-by-play actions cannot be confirmed since the computer with all of

its configurations was wiped.

So what now?

I suppose all we can do is guess, as I have my own never-seen-before-before theory to

how this happened:

So it all started on a stormy Wednesday evening, when Keyvin finished cooking up his lasagna

and heads to the computer room to work on his new gaming setup.

After setting everything up, he attempts to run steam.

But steam fails to run.

Not because the NTFS drive did not add the executable bit, but because Keyvin created

the symlink incorrectly in home/local, rather than home/local/share, where steamroot is

supposed to be.

This is a fairly easy typo to make, as he made the same mistake on the Github issue he created.

He realizes his mistake, and fixes the symlink, creating it in the proper folder, and re-runs steam.

While steam is still starting up, he absent-mindedly goes to clean up the incorrect symlink

because why not, and forgets that he ever did so.

But unbeknownst to him, he actually accidentally unlinks the correct symlink instead.

In a stunning race condition, he manages to do this in the literal microseconds after

steam.sh is invoked but before the assignment of STEAMROOT, and since the symlinked path

no longer exists, the directory change fails, and STEAMROOT remains an empty string.

And we’ve already established that if STEAMROOT is an empty string, the bug will trigger.

To fix the bug, various alterations to the STEAMROOT command such as using readlink or

dirname can guarantee a valid path is returned, and checks can be added to verify steamroot

is a real path.

Furthermore, bash has built in options which make the shell exit when a command fails,

so when the directory change or STEAMEXE execution fails, the script immediately ends instead

of proceeding with buggy behavior.

These are what ended up being the actual fix, but many disagreed that this was enough.

Some suggested to write all scripts in a real language like Python or Perl, or check for

typical Linux directories, lookup files from an internal database and only remove known files,

not writing untreated amateur hour code, never use rm -rf, use an empty file as a marker,

don’t use the wildcard, you should just lock this thread,

just don't delete user data, by the way here is my public domain script that has my preferred fix,

Valve has no idea how to linux, has this problem been fixed?

It’s still happening.

Question mark?

You know, we could talk for centuries about the ideal way to fix this, but sometimes you

don’t need to achieve perfection.

Anyways, it’s been like 8 years and the issue hasn’t resurfaced for anyone except this

guy, but he didn’t provide further evidence so we can mark this issue resolved.

See you in the next issue of issues insights.