Fuzzing (fuzz testing) 101: Lessons from cyber security expert Dr. David Brumley

Summary

Dr. David Brumley, a professor at Carnegie Mellon University and CEO of For All Secure, explains fuzzing, a technique used to improve software security and development. Fuzzing involves feeding random inputs to a program to uncover bugs and vulnerabilities, much like monkeys randomly typing on keyboards. The process has evolved from black box fuzzers to the current third generation, which uses instrumentation-guided fuzzing to intelligently explore and identify security issues. Brumley highlights the benefits of fuzzing for both security and reliability in software development.

Takeaways

• 📚 Fuzzing is a decades-old process that is not widely known outside of cybersecurity circles but is crucial for improving security processes and software development.
• 🏆 Dr. David Brumley, a professor at Carnegie Mellon University and CEO of For All Secure, is a pioneer in fuzzing technology, having built the winning entry for the DARPA Cyber Grand Challenge.
• 🔍 Fuzzing involves providing random input to applications to discover vulnerabilities, much like monkeys typing on a keyboard, but with the aim of uncovering security issues rather than creating literary works.
• 🤖 The analogy of a program being a maze with a robot navigating it helps explain how fuzzing works, with inputs as directions that the robot follows through the program's logic.
• 🐛 Fuzzing automates the process of input generation and execution to find bugs, which is more efficient than manual unit testing by developers.
• 🚀 Fuzzing has evolved to its third generation, moving from random input generation to more sophisticated techniques that learn from execution paths to find new vulnerabilities.
• 🔬 Static analysis and fuzzing are contrasted, with static analysis examining code for patterns without execution, while fuzzing actively runs the program with generated inputs to find issues.
• 🛡️ Google's use of fuzzing has led to the discovery of 25,000 bugs in Google Chrome and open-source libraries over three years, demonstrating the power of automated testing for security.
• 🛠️ Beyond security, fuzzing benefits developers by providing test cases that can improve the reliability of software and speed up the development lifecycle.
• 🔑 There are different types of fuzzers, including black box, grammar-based, and instrumentation-guided fuzzers, each with its approach to generating inputs and finding bugs.
• 🌐 Companies can start using fuzzing by adopting these techniques, which are favored by major development shops like Google and Microsoft for their effectiveness in finding vulnerabilities.

Q & A

What is fuzzing and how long has it been around?

Fuzzing is a technique used to discover security issues in software by providing random inputs to the program to see if it can cause a crash or uncover vulnerabilities. It has been around for about 25 years, originally coined by Professor Bart Miller.

What did Professor Bart Miller and his graduate students discover when they gave random inputs to Unix, Microsoft, and Apple applications?

They discovered that about a third of these applications would crash when given random inputs, revealing serious security issues.

Can you explain the analogy used by Dr. David Brumley to describe how fuzzing works?

Dr. Brumley used the analogy of a program being like a maze and the input being directions for a robot navigating through it. Fuzzing automates the process of giving the robot different paths to explore to find bugs or crashes in the program.

What is the difference between fuzzing and static analysis in terms of software testing?

Static analysis involves examining the program's code without running it, looking for patterns that might indicate problems. Fuzzing, on the other hand, involves actually running the program with various inputs to see if it behaves unexpectedly or crashes.

How does fuzzing benefit the software development process beyond security?

Fuzzing can help improve the reliability of software by executing various paths and uncovering potential bugs. It can also speed up the software development life cycle by generating test cases automatically, reducing the need for manual testing.

What are the three generations of fuzzing techniques mentioned in the script?

The first generation is black box fuzzing, which generates random inputs. The second generation is grammar-based fuzzing, which uses templates to generate structured inputs. The third generation is instrumentation-guided fuzzing, which learns from the program's execution to generate new inputs.

How does instrumentation-guided fuzzing differ from the earlier generations of fuzzing?

Instrumentation-guided fuzzing generates inputs and observes the program's execution path, learning from it to inform the generation of the next input. This approach combines the benefits of structured exploration with the ability to discover new paths, unlike earlier methods.

What is the significance of the DARPA Cyber Grand Challenge mentioned in the script?

The DARPA Cyber Grand Challenge is a competition that Dr. David Brumley's team won using their fuzzing technology. This achievement highlights the effectiveness and advancement of fuzzing techniques in the field of cybersecurity.

How has Google utilized fuzzing in their projects?

Google has a project where they use fuzzing to automatically find bugs in Google Chrome and many open-source libraries they use. Over the last three years, they have found 25,000 bugs with zero false positives using this method.

What advice does Dr. David Brumley give for companies looking to start using fuzzing?

Dr. Brumley suggests that companies should consider using the third generation of fuzzing techniques, specifically instrumentation-guided fuzzing, as it offers a balance between structured exploration and the ability to uncover new vulnerabilities.