2.1 Developing Hypotheses - MAD20 Threat Hunting & Detection Engineering Course

MAD20Tech

25 Apr 202407:45

Summary

TLDRThis module delves into developing hypotheses and abstract analytics in the threat hunting methodology. It emphasizes the importance of formulating testable hypotheses based on TTP insights and evidence, guiding data collection and analytic development. A good hypothesis should be specific, evidence-driven, and falsifiable, helping to focus research and reason about behavior naturally. The process involves iterative refinement to address nuances and false positives, ultimately aiding in identifying malicious activity.

Takeaways

🔎 The module focuses on developing and refining hypotheses and abstract analytics to explore for evidence of malicious activity.
📝 A hypothesis is defined as a supposition or proposed explanation made on limited evidence as a starting point for further investigation.
📋 A well-formed hypothesis should be specific, evidence-driven, testable, and falsifiable to guide data collection and analysis.
🧐 Hypotheses are crafted using TTP insights and existing knowledge of adversary behavior to make claims about potential malicious activity.
🔍 The development of hypotheses helps in focusing the research, data collection, and analytic development for a deeper understanding of the environment.
🤔 A hypothesis should be framed in a way that allows for testing to gain additional evidence and should consider what evidence would support or refute it.
🚫 A hypothesis should be falsifiable, meaning it can be disproven through testing, avoiding statements that are indistinguishable from benign usage.
🛠 Hypothesis creation is an iterative process that involves continual updating and refinement based on evidence and evaluation of falsifiability.
📖 Writing a hypothesis in plain language helps facilitate reasoning and understanding without being constrained by specific query syntax.
🔑 Hypotheses should be specific enough to avoid false positives and should incorporate key elements of the suspected malicious behavior.
🔄 The process of hypothesis refinement involves considering benign scenarios and addressing them to focus on identifying actual malicious usage.

Q & A

What is the main focus of module two in the threat hunting methodology?
-Module two focuses on developing and refining hypotheses and abstract analytics to explore hunting for evidence that indicates a malicious actor may be present.
What is the definition of a hypothesis according to the Oxford dictionary?
-A hypothesis is defined as a supposition or proposed explanation made on the basis of limited evidence as a starting point for further investigation.
What are the criteria that a good hypothesis should meet?
-A good hypothesis should be specific enough to be useful, evidence-driven, testable to gain additional evidence, and falsifiable, meaning it can be disproven through testing.
Why is it important to create a hypothesis that is specific?
-A specific hypothesis helps to focus the problem, making it easier to scope data collection and analysis, and avoiding vagueness that could lead to inadequate answers.
How does a hypothesis help in the threat hunting process?
-A hypothesis provides clarity in thinking about what is being looked for, helps reason about behavior naturally, and bridges narrative information about behavior to concrete analytics.
What is the purpose of creating a hypothesis in the context of threat hunting?
-Creating a hypothesis helps to provide focus for research, data collection, and analytic development, allowing for a deeper understanding of what an analytic does and what can trigger false positives.
Why should a hypothesis be falsifiable in scientific terms?
-A falsifiable hypothesis is one that can be disproven through testing, which is essential for scientific rigor and to avoid making claims that cannot be objectively evaluated.
What is an example of a hypothesis that is not falsifiable?
-An example of a non-falsifiable hypothesis is 'a malicious actor will use extreme stealth to operate in a way that will be indistinguishable from benign usage,' as there would be no evidence to examine if the claim were correct.
How does the process of hypothesis refinement help in threat hunting?
-Hypothesis refinement helps to account for nuances not captured during initial development and focuses on malicious usage, improving the accuracy and effectiveness of the hypothesis.
What should be the language of a hypothesis in the methodology stage?
-A hypothesis should be written in plain, human-understandable language to facilitate reasoning and understanding without the constraints of specific query syntax and to allow for sharing of thoughts and ideas.
Can you provide an example of how to refine a hypothesis based on the script?
-An initial hypothesis like 'if a task is scheduled, an adversary is establishing persistence' can be refined to 'if a task is scheduled by a non-admin user, an adversary is establishing persistence' to account for benign task scheduling by administrators.

Outlines

00:00

🔎 Developing Hypotheses and Abstract Analytics

This paragraph introduces Module 2 of the threat hunting methodology, focusing on the development and refinement of hypotheses and abstract analytics. It emphasizes the importance of using TTP insights to form testable hypotheses about potential malicious activity. The paragraph outlines the criteria for a well-formed hypothesis: it must be specific, evidence-driven, testable, and falsifiable. The process of hypothesis creation is described as iterative, allowing for continuous refinement based on evidence. The purpose of creating hypotheses in threat hunting is to clarify thinking, reason about behavior naturally, and bridge the gap between narrative information and concrete analytics. The paragraph also provides an example of how to refine a hypothesis to reduce the likelihood of false positives and ensure it remains falsifiable.

05:03

🚔 Hypothesis Refinement and Cybersecurity Application

The second paragraph delves deeper into the process of hypothesis refinement using the analogy of a burglar breaking into a home by kicking open locked doors. It illustrates how a hypothesis can be made more specific and less prone to false positives by incorporating key elements of the malicious technique. The paragraph also discusses the importance of gathering evidence to support or refute the hypothesis and the need to consider benign scenarios that could mimic malicious activity. A cyber-related example is provided, where the hypothesis evolves from a general statement about task scheduling to a more specific one that considers the user role in task creation. The paragraph concludes by reiterating the importance of a solid hypothesis being specific, evidence-driven, and falsifiable to guide effective research in cybersecurity.

Mindmap

Keywords

💡Hypothesis

A hypothesis is a proposed explanation made on the basis of limited evidence as a starting point for further investigation. In the context of the video, a hypothesis serves as a supposition about potential malicious activity that can be tested and refined through data collection and analysis. The script emphasizes the importance of creating specific, evidence-driven, and falsifiable hypotheses to guide threat hunting operations and to focus research efforts effectively.

💡Threat Hunting

Threat hunting refers to the proactive search for potential security threats that may not be detected by traditional security measures. In the video, it is the process of developing and refining hypotheses and abstract analytics to explore for evidence indicating the presence of a malicious actor. The methodology described involves using TTP insights to formulate testable claims about malicious activities within an environment.

💡Abstract Analytics

Abstract analytics in the video refers to a conceptual approach to analyzing data that is not tied to specific query syntax or platform. It is about understanding patterns and behaviors that could indicate malicious activity. The script discusses the importance of formulating abstract analytics to guide the development of more concrete analytic tools and to facilitate reasoning about behavior in a way that is not constrained by specific technologies.

💡TTP (Tactics, Techniques, and Procedures)

TTP stands for Tactics, Techniques, and Procedures, which are the methods used by adversaries to carry out cyber attacks. The script mentions using TTP insights to develop hypotheses about what behaviors or patterns might indicate malicious activity. Understanding TTPs helps in crafting more accurate and relevant hypotheses for threat hunting.

💡Evidence-Driven

Being evidence-driven means basing one's hypothesis or conclusions on the available data or evidence. The video emphasizes the importance of using existing techniques, knowledge of adversary behavior in TTPs, and findings from research and investigation to support the development of a hypothesis. This approach ensures that hypotheses are grounded in reality and can be tested and refined.

💡Falsifiable

A falsifiable hypothesis is one that can be disproven through testing. The script explains that a good scientific hypothesis should be falsifiable, meaning there should be a way to gather evidence that could potentially show the hypothesis to be incorrect. This is crucial for the scientific process of hypothesis testing in threat hunting.

💡Data Collection

Data collection is the process of gathering and assembling information that is relevant to the hypothesis being tested. In the video, data collection is highlighted as a critical step following the development of a hypothesis, as it provides the necessary information to analyze and test the hypothesis for indications of malicious activity.

💡Analytic Development

Analytic development involves creating tools and methods for analyzing data to detect patterns or anomalies that could indicate a security threat. The video discusses how hypotheses guide the development of these analytics, which are then used to test the hypotheses and gather further evidence.

💡Malicious Actor

A malicious actor in the context of the video refers to an individual or entity that is suspected of carrying out unauthorized or harmful activities within a computer system or network. The script discusses the process of developing hypotheses to test for the presence of such actors and the evidence that might indicate their activities.

💡Iterative Process

An iterative process is one that is repeated several times, with each repetition allowing for refinement and improvement based on the results of the previous cycles. In the video, the creation and refinement of hypotheses is described as an iterative process that involves continuous updating based on new evidence and insights gathered during threat hunting.

💡False Positives

False positives occur when a security system incorrectly identifies a benign activity as malicious. The video script discusses the importance of refining hypotheses to minimize false positives, ensuring that the analytic tools developed are as accurate as possible in identifying real threats.

Highlights

Module 2 focuses on developing hypotheses and abstract analytics to explore for evidence of malicious actors.

Hypotheses guide data collection, analytic development and future hunting operations.

A hypothesis should be specific enough to be useful and help focus the problem.

Evidence from techniques, adversary behavior, and research should drive hypothesis development and refinement.

A good hypothesis is testable and can be proven or disproven through evidence.

Falsifiability is key - a hypothesis should be able to be disproven through testing.

Creating a hypothesis helps clarify thinking, reason about behavior, and bridge narrative to concrete analytics.

Hypothesis creation is an iterative process of continual updating and refinement based on evidence.

Evaluating falsifiability helps expose potential false alarm scenarios not captured initially.

A hypothesis should be written in plain language to facilitate reasoning and understanding.

Starting the hypothesis development process involves choosing a behavior and identifying evidence of malicious activity.

The example of burglars kicking open doors illustrates the need for specificity and falsifiability in hypotheses.

Continuous sensing and monitoring are required to gather evidence to support or refute a hypothesis.

Refined hypotheses should address identified nuances and focus on capturing malicious usage.

In the cyber example, the hypothesis evolves from 'if a task is scheduled' to 'if a non-admin user schedules a task'.

A solid hypothesis should be specific, evidence-driven, and falsifiable to effectively guide research.