KM - FInal Presentation

Summary

Kevin McMurdy’s project, 'Guardrails,' focuses on evaluating and securing large language model (LLM)-empowered applications, such as chatbots. With a background in pentesting, Kevin developed a tool to test LLMs for vulnerabilities, particularly around unwanted responses. The tool utilizes test cases with transformations like base64 encoding or Morse code to assess how well LLMs are configured to prevent harmful outputs. The project aims to automate security evaluations, ensuring LLMs don’t provide dangerous or inappropriate information, while staying adaptable to new prompt bypass techniques. It’s a vital resource for developers and security professionals seeking to safeguard AI-powered tools.

Takeaways

• 😀 Guardrails is a project focused on evaluating and securing large language model (LLM) empowered offerings to prevent harmful or unintended outputs.
• 😀 The creator, Kevin McMurdy, has a background in pentesting and embedded software engineering, and recently shifted to evaluating security for AI and LLM systems.
• 😀 The primary goal of Guardrails is to ensure that LLM-powered services like chatbots don't provide dangerous or inappropriate responses (e.g., instructions on making thermite).
• 😀 The project involves creating test suites that contain multiple test cases aimed at evaluating LLM responses to various prompt smuggling techniques.
• 😀 Prompt smuggling techniques such as Base64 encoding, Morse code, and reversals are used to assess how LLMs handle obfuscated prompts.
• 😀 After running tests, responses are manually reviewed to ensure that harmful or irrelevant content (e.g., dangerous advice) is not provided by the LLM.
• 😀 The system currently supports registering endpoints, which are LLM instances that are evaluated for their security responses to different test cases.
• 😀 The evaluation process is designed to be repeatable and scalable, allowing for multiple test cases and iterations to assess the effectiveness of guardrails.
• 😀 There is a focus on automated evaluation criteria, where responses can eventually be analyzed programmatically to detect inappropriate content.
• 😀 The project is modular, allowing for continuous updates and improvements, including new prompt obfuscation techniques and security measures.
• 😀 Security tools such as the Post Inspector plugin are used to capture traffic and evaluate LLM responses, though there are security concerns with these tools that need to be addressed in future iterations.

Q & A

What is the primary focus of Kevin McMurdy's project?

Kevin McMurdy's project, titled 'Guardrails,' is focused on evaluating large language model (LLM) empowered offerings, specifically from a security standpoint. The project aims to identify and address potential vulnerabilities and concerns related to LLMs in various use cases.

What was Kevin's professional background before working on this project?

Before working on this project, Kevin McMurdy was a pentester for eight years and an embedded software engineer. This background in security and engineering laid the foundation for his current focus on evaluating LLM-based solutions.

Why does Kevin refuse to call these models 'AI'?

Kevin refuses to call these models 'AI' because he believes the term is misused and doesn’t accurately reflect the nature of large language models, which are more like advanced algorithms rather than true artificial intelligence.

What challenge is Kevin trying to address with his project?

Kevin is addressing the challenge of ensuring that large language models are correctly configured and secure, specifically preventing them from answering inappropriate or potentially harmful questions, such as giving instructions on dangerous topics.

What is 'fuzzy prompts,' and how does it work?

'Fuzzy prompts' is the working name for Kevin's solution that helps evaluate LLM empowered offerings. It involves creating a suite of test cases, applying different prompt-smuggling techniques, and testing how the model responds. The goal is to ensure that LLMs don't give inappropriate answers by using transformations of test prompts.

What is prompt smuggling, and why is it important in this context?

Prompt smuggling refers to techniques used to sneak inappropriate or hidden instructions into prompts to test if the model can be tricked into answering questions it shouldn't. It is important because it helps identify vulnerabilities in the configuration of LLMs, which could potentially lead to unintended responses.

How does Kevin manage and organize test cases for his evaluations?

Kevin organizes test cases into test suites, where each suite contains a collection of different test cases. These test cases are designed to assess various aspects of LLMs, including potential vulnerabilities and biases. He also applies different transformations, like base64 encoding or Morse code, to test prompt-smuggling techniques.

What role do legal concerns play in Kevin's project?

Legal concerns play a significant role in Kevin's project. As he collaborates with his legal team, he ensures that test cases address issues related to bias, gender, race, and other potentially problematic content in LLM responses. The goal is to ensure that LLMs do not generate harmful or biased content.

What is the significance of the 'dialogue' feature in Kevin's project?

The 'dialogue' feature allows Kevin to track interactions and test cases over time. It records the prompts and responses from the LLM, enabling Kevin to evaluate if any malicious or inappropriate responses are triggered through various permutations or prompt manipulations.

What does Kevin mean by 'terminal times' and how does it impact the project?

When Kevin mentions 'terminal times,' he is referring to the time constraints he faces during demonstrations of his project. This limitation affects how much time he has to showcase the full functionality of the tool, sometimes leading to quick demonstrations and truncated explanations.