2.2 Hypothesis Considerations - MAD20 Threat Hunting & Detection Engineering Course

Summary

This lesson delves into the influence of bias in threat hunting, teaching how to recognize and mitigate its impact on intelligence reporting. It emphasizes the importance of being aware of cognitive biases like visibility and victim bias, and how defenders can introduce availability and anchoring biases. The lesson guides on formulating hypotheses, choosing attack techniques wisely, leveraging existing data, and engaging with the community to refine analytic approaches and avoid redundancy.

Takeaways

• 🔍 Bias in threat hunting can occur and must be recognized to minimize its impact on intelligence reporting.
• 🧠 Cognitive biases, such as visibility and victim bias, can skew the perception of the full scope of attacks.
• 👀 Availability and anchoring biases can lead to a narrow focus on familiar or currently accessible data, potentially overlooking other important information.
• 📝 Documenting and sharing assumptions with the team is crucial for validating and revisiting them during the analytic development process.
• 🔑 When generating hypotheses, be specific about known facts, inferences, chosen hypotheses, discarded options, and the environment being defended.
• 💡 Focus analytic efforts on techniques that are not commonly covered, have a significant impact if used, and leverage existing data collection for efficient implementation.
• 🚀 Consider techniques that are not typically employed by users and system administrators to avoid high false alarm rates.
• 🔎 Engage with the community and existing resources to avoid redundant work and to uncover potential flaws in your approach.
• 🔗 Investigate if there are precursor, follow-on, or correlated techniques to the one being analyzed, as grouping them can improve precision and recall.
• 🛠 Define the scope of the behavior under examination based on platforms, implementations, and functionality to focus research effectively.
• ⚖️ Be prepared to revisit and adjust the scope as needed to ensure the full range of the technique is covered according to the environmental terrain.

Q & A

What is the main focus of lesson 2.2 in the provided transcript?

The main focus of lesson 2.2 is to discuss how bias can occur in threat hunting, how to recognize it, and considerations for choosing an attack technique on which to focus the hypothesis.

What is the significance of understanding cognitive biases in the context of threat intelligence reporting?

Understanding cognitive biases is crucial to minimize their impact in threat intelligence reporting, as they can lead to skewed analysis and false impressions of the full scope of attacks or activities.

Can you give an example of a cognitive bias mentioned in the transcript?

One example mentioned is visibility bias, which occurs when threat intelligence produced by an organization is only focused on the subset of adversarial activity that they can detect.

What is the importance of being aware of inherent biases in models used for threat hunting?

Being aware of inherent biases in models is important to ensure accurate and comprehensive threat analysis, as these biases can influence the focus and interpretation of data.

How can victim bias affect the threat intelligence reports?

Victim bias can affect reports by focusing more on high-profile victims and skewing the data based on what is allowed to be published, which may not represent the full range of threats.

What is the impact of novelty bias in the context of threat hunting?

Novelty bias can lead to more coverage and attention being given to new or flashy adversary groups, potentially overshadowing long-standing threats that may be more prevalent or significant.

Why is it important for threat hunters to document and share their assumptions with their team?

Documenting and sharing assumptions is important for validation and to revisit them through the analytic development process, ensuring a more accurate and objective threat hunting approach.

What is the advice given for focusing analytic efforts when choosing a technique in threat hunting?

The advice is to focus on techniques that are not already commonly used by adversaries, would create a significant impact if used successfully, and capitalize on existing data collection, documentation, or analytics.

Why is it beneficial to check for existing analytics, mitigations, or other defensive ideas online before conducting research?

Checking for existing work can save time and effort, help avoid redundant work, and may highlight gaps that can be focused on, leveraging the knowledge and findings of other security researchers.

What is the purpose of engaging with the community in the context of threat hunting?

Engaging with the community helps improve work by sharing new discoveries, getting feedback on approaches, and uncovering flaws early on, which can save time and prevent potential issues.

Why is it necessary to define the scope of the behavior when preparing to conduct research on a technique?

Defining the scope helps to focus research on relevant systems and behaviors, ensuring that the analysis is accurate and tailored to the specific environment and requirements of the threat hunting process.