2.2 Hypothesis Considerations - MAD20 Threat Hunting & Detection Engineering Course
Summary
TLDRThis lesson delves into the influence of bias in threat hunting, teaching how to recognize and mitigate its impact on intelligence reporting. It emphasizes the importance of being aware of cognitive biases like visibility and victim bias, and how defenders can introduce availability and anchoring biases. The lesson guides on formulating hypotheses, choosing attack techniques wisely, leveraging existing data, and engaging with the community to refine analytic approaches and avoid redundancy.
Takeaways
- 🔍 Bias in threat hunting can occur and must be recognized to minimize its impact on intelligence reporting.
- 🧠 Cognitive biases, such as visibility and victim bias, can skew the perception of the full scope of attacks.
- 👀 Availability and anchoring biases can lead to a narrow focus on familiar or currently accessible data, potentially overlooking other important information.
- 📝 Documenting and sharing assumptions with the team is crucial for validating and revisiting them during the analytic development process.
- 🔑 When generating hypotheses, be specific about known facts, inferences, chosen hypotheses, discarded options, and the environment being defended.
- 💡 Focus analytic efforts on techniques that are not commonly covered, have a significant impact if used, and leverage existing data collection for efficient implementation.
- 🚀 Consider techniques that are not typically employed by users and system administrators to avoid high false alarm rates.
- 🔎 Engage with the community and existing resources to avoid redundant work and to uncover potential flaws in your approach.
- 🔗 Investigate if there are precursor, follow-on, or correlated techniques to the one being analyzed, as grouping them can improve precision and recall.
- 🛠 Define the scope of the behavior under examination based on platforms, implementations, and functionality to focus research effectively.
- ⚖️ Be prepared to revisit and adjust the scope as needed to ensure the full range of the technique is covered according to the environmental terrain.
Q & A
What is the main focus of lesson 2.2 in the provided transcript?
-The main focus of lesson 2.2 is to discuss how bias can occur in threat hunting, how to recognize it, and considerations for choosing an attack technique on which to focus the hypothesis.
What is the significance of understanding cognitive biases in the context of threat intelligence reporting?
-Understanding cognitive biases is crucial to minimize their impact in threat intelligence reporting, as they can lead to skewed analysis and false impressions of the full scope of attacks or activities.
Can you give an example of a cognitive bias mentioned in the transcript?
-One example mentioned is visibility bias, which occurs when threat intelligence produced by an organization is only focused on the subset of adversarial activity that they can detect.
What is the importance of being aware of inherent biases in models used for threat hunting?
-Being aware of inherent biases in models is important to ensure accurate and comprehensive threat analysis, as these biases can influence the focus and interpretation of data.
How can victim bias affect the threat intelligence reports?
-Victim bias can affect reports by focusing more on high-profile victims and skewing the data based on what is allowed to be published, which may not represent the full range of threats.
What is the impact of novelty bias in the context of threat hunting?
-Novelty bias can lead to more coverage and attention being given to new or flashy adversary groups, potentially overshadowing long-standing threats that may be more prevalent or significant.
Why is it important for threat hunters to document and share their assumptions with their team?
-Documenting and sharing assumptions is important for validation and to revisit them through the analytic development process, ensuring a more accurate and objective threat hunting approach.
What is the advice given for focusing analytic efforts when choosing a technique in threat hunting?
-The advice is to focus on techniques that are not already commonly used by adversaries, would create a significant impact if used successfully, and capitalize on existing data collection, documentation, or analytics.
Why is it beneficial to check for existing analytics, mitigations, or other defensive ideas online before conducting research?
-Checking for existing work can save time and effort, help avoid redundant work, and may highlight gaps that can be focused on, leveraging the knowledge and findings of other security researchers.
What is the purpose of engaging with the community in the context of threat hunting?
-Engaging with the community helps improve work by sharing new discoveries, getting feedback on approaches, and uncovering flaws early on, which can save time and prevent potential issues.
Why is it necessary to define the scope of the behavior when preparing to conduct research on a technique?
-Defining the scope helps to focus research on relevant systems and behaviors, ensuring that the analysis is accurate and tailored to the specific environment and requirements of the threat hunting process.
Outlines
🕵️♂️ Bias in Threat Hunting and Hypothesis Considerations
This paragraph discusses the occurrence of bias in threat hunting and the importance of recognizing and addressing it. It mentions the MITRE ATT&CK Defender Cyber Threat Intelligence course, which covers cognitive biases that can affect threat intelligence reporting. The script emphasizes the need to be aware of inherent biases such as visibility bias, victim bias, and novelty bias. It also points out that defenders can introduce biases like availability bias and anchoring bias. The importance of documenting and sharing assumptions with the team is highlighted, as well as the need for specificity when generating hypotheses. The paragraph concludes with advice on choosing a technique to focus on, considering the return on investment, and leveraging existing data and analytics to minimize false alarms.
🔍 Technique Selection and Hypothesis Scoping
The second paragraph delves into the process of selecting and scoping techniques for threat hunting. It advises focusing on techniques that are not commonly covered, have a significant impact if used, and can be implemented easily without triggering many false alarms. The paragraph also suggests considering techniques that are not typically used by system administrators. It encourages researchers to check for existing analytics, mitigations, and defensive ideas to avoid redundant work and to engage with the community to improve their work. The importance of defining the scope of the behavior to be examined is highlighted, including considering factors like platforms, implementations, and functionality. The paragraph concludes by emphasizing the need to be aware of biases when developing hypotheses and the significance of technique choice and hypothesis scoping for setting up for success in the long run.
Mindmap
Keywords
💡Bias
💡Threat Hunting
💡Cognitive Biases
💡Visibility Bias
💡Victim Bias
💡Novelty Bias
💡Availability Bias
💡Anchoring Bias
💡Hypothesis
💡Technique
💡Investment Return
💡False Alarms
💡Community Engagement
💡Scope
Highlights
Bias can occur in threat hunting and recognizing it is crucial.
Cognitive biases can be present in threat intelligence and user actions.
The goal is not to memorize all cognitive biases but to understand their impact.
Visibility bias gives a false impression of the full scope of attacks.
Victim bias skews focus towards high-profile victims.
Novelty bias leads to more attention on new adversary groups.
Defenders may introduce bias through availability, focusing only on accessible data.
Anchoring bias can cause defenders to miss useful information from other sources.
Threat hunters should document and share assumptions with their team for validation.
When generating hypotheses, be specific about what is known and inferred.
Focus analytic efforts on techniques not commonly used by adversaries.
Choose techniques that capitalize on existing data collection for easier implementation.
Avoid techniques that trigger too many false alarms in the system.
Engage with the community to improve work and avoid redundant efforts.
Consider precursor, follow-on, or correlated techniques for a comprehensive approach.
Define the scope of the behavior to examine within the context of desired factors.
Scoping helps in focusing research towards relevant systems and behaviors.
Be aware of biases when developing hypotheses and conducting research.
Technique choice and hypothesis scoping are crucial for setting up for success.
Transcripts
hello and welcome to lesson 2.2
hypothesis
considerations in this lesson we will
discuss how bias can occur in threat
hunting as well as how to recognize it
we'll also discuss considerations for
choosing an attack technique on which to
focus your
hypothesis if you've already taken the
miter attack Defender cyber threat
intelligence course then you'll recall
that it discusses cognitive biases
namely the bias present in the threat
intelligence itself as well as bias that
we as users can
introduce while there are dozens of
known types of cognitive biases our goal
in this section is not to memorize them
all but to go through some examples to
keep in mind as we discuss ways to deal
with bias in order to minimize its
impact in threat intelligence reporting
and even in models such as attack
inherent biases can be present and it is
important to be aware of how they may
present
themselves one example is visibility
bias
which occurs when the threat
intelligence produced by an organization
is only focused on the subset of
adversarial activity that they can
detect which may give a false impression
of the full scope of the attack or
activity other examples of bias that can
occur are victim bias where reports tend
to focus on more high-profile victims
and can be skewed based on what they
actually allow to be published as well
as novelty bias where for example a
flashy new adversary group may receive
more coverage and attention than a
long-standing
one there are also several ways that
bias can be introduced by the defender
themselves as an example availability
bias can be introduced by a threat
Hunter who is relying only on the data
that they currently have access to in
order to prioritize techniques or
narrowly focusing on adversarial
behaviors and techniques that they are
already familiar with which could give a
false sense of the importance or urgency
of the threats at
hand another example is an anchoring
bias which can cause the defender to
lose out on a lot of useful information
provided by other data sources because
they're solely focusing on those that
have already been discussed or reported
on there are many more types of bias
that can occur in this environment and I
would encourage you to continue to learn
about them and how they could apply to
threat
hunting as threat Hunters we need to
understand when we are making
assumptions explicitly document and
share them with our team to validate
them and revisit them through our
analytic development process this is
especially important when determining
what activity to hunt for and for
generating
hypothesis be specific about what you
factually know from threat intelligence
what you're inferring why you chose a
particular hypothesis what other
hypotheses you discarded or deprioritize
and what you believe about the
environment you are
defending when choosing a technique
there are many things to consider but
what it essentially boils down to is is
getting a good return on your investment
we advise you to focus analytic efforts
first on techniques that aren't already
covered commonly used by adversaries or
would create a big impact if
successfully used on your
systems also select techniques that
capitalize on existing data collection
documentation or analytics order that
you anticipate will be relatively easy
to implement and not trigger too many
false alarms in your system for example
techniques that typical users and system
administrators don't employ you'll have
to think through and find a good balance
between these characteristics to
determine how to best Focus your
efforts once you've chosen a technique
consider the following questions as you
prepare to conduct your
research keep in mind that you aren't
alone in this work many security
researchers have investigated techniques
and published their findings and ideas
so read up on what others have done so
you don't end up doing redundant work
check for any other existing analytics
mitigations or other defensive ideas
online as associated with this Behavior
attack car Sigma the threat Hunter
Playbook and countless others are freely
available and often contain excellent
information and specific analytics and
mitigations for these malicious
behaviors searching those first can help
save you a lot of time and effort and
may help highlight a gap that you can
focus your time
on engaging with the community on your
ideas is also a great way to help
improve your work if you've discovered
something new you can share with others
if there's a flaw with the approach
engagement can help uncover it early and
save you trouble down the
road in this course we focus on a single
technique at a time for Simplicity you
should consider if there are precursor
followon or correlated techniques to the
one you're investigating and think about
grouping them together during your
analytic
approach there may be two techniques
that in isolation have a high false
alarm rate but when seen together more
likely indicate malicious activity the
converse may also be true in both cases
grouping related techniques can help
with precision and
recall in addition to techniques that an
adversary may use in conjunction with
each other it's also worth examining
other means through which an adversary
can accomplish their goal in other words
their plan
B what other techniques exist in the
same
tactic another key item at this point is
to define the scope of the behavior we
want to examine which we can do in the
context of factors we'd like to support
such as platforms implementations and
functionality limiting your scope to one
or more platforms will help to focus our
research towards relevant systems which
should be dictated by the environmental
terrain scoping based on implementation
method is also useful at this stage as
we may for example wish to exclude
invocations that rely on deprecated
commands or other methods not relevant
to our
systems finally intended functionality
is also a good scope scoping Factor at
this point as it can help determine what
types of behavior to include or exclude
in your research for example whether or
not to support remote
execution as you continue in this
process you may have to revisit this
step and narrow or expand your scope as
needed in order to ensure you're finding
the correct behaviors that cover the
full range of the technique in
accordance with your
terrain to summarize it's important to
be aware of biases When developing
hypotheses and while you're conducting
your research technique choice and
hypothesis scoping are also important
aspects of this process that will help
set you up for Success later on down the
road
تصفح المزيد من مقاطع الفيديو ذات الصلة
12 Cognitive Biases Explained - How to Think Better and More Logically Removing Bias
Kenapa Kita Bisa Kemakan Marketing?
8 BIAS COGNITIVI che influenzano le tue DECISIONI
Seni Berpikir Jernih (99 Jenis Sesat Pikir) | Ringkasan Buku
13 BIAS COGNITIVI che ci rendono IRRAZIONALI 🧠
BIAS cognitivi: cosa sono (con esempi)
5.0 / 5 (0 votes)