2.1 Developing Hypotheses - MAD20 Threat Hunting & Detection Engineering Course
Summary
TLDRThis module delves into developing hypotheses and abstract analytics in the threat hunting methodology. It emphasizes the importance of formulating testable hypotheses based on TTP insights and evidence, guiding data collection and analytic development. A good hypothesis should be specific, evidence-driven, and falsifiable, helping to focus research and reason about behavior naturally. The process involves iterative refinement to address nuances and false positives, ultimately aiding in identifying malicious activity.
Takeaways
- 🔎 The module focuses on developing and refining hypotheses and abstract analytics to explore for evidence of malicious activity.
- 📝 A hypothesis is defined as a supposition or proposed explanation made on limited evidence as a starting point for further investigation.
- 📋 A well-formed hypothesis should be specific, evidence-driven, testable, and falsifiable to guide data collection and analysis.
- 🧐 Hypotheses are crafted using TTP insights and existing knowledge of adversary behavior to make claims about potential malicious activity.
- 🔍 The development of hypotheses helps in focusing the research, data collection, and analytic development for a deeper understanding of the environment.
- 🤔 A hypothesis should be framed in a way that allows for testing to gain additional evidence and should consider what evidence would support or refute it.
- 🚫 A hypothesis should be falsifiable, meaning it can be disproven through testing, avoiding statements that are indistinguishable from benign usage.
- 🛠 Hypothesis creation is an iterative process that involves continual updating and refinement based on evidence and evaluation of falsifiability.
- 📖 Writing a hypothesis in plain language helps facilitate reasoning and understanding without being constrained by specific query syntax.
- 🔑 Hypotheses should be specific enough to avoid false positives and should incorporate key elements of the suspected malicious behavior.
- 🔄 The process of hypothesis refinement involves considering benign scenarios and addressing them to focus on identifying actual malicious usage.
Q & A
What is the main focus of module two in the threat hunting methodology?
-Module two focuses on developing and refining hypotheses and abstract analytics to explore hunting for evidence that indicates a malicious actor may be present.
What is the definition of a hypothesis according to the Oxford dictionary?
-A hypothesis is defined as a supposition or proposed explanation made on the basis of limited evidence as a starting point for further investigation.
What are the criteria that a good hypothesis should meet?
-A good hypothesis should be specific enough to be useful, evidence-driven, testable to gain additional evidence, and falsifiable, meaning it can be disproven through testing.
Why is it important to create a hypothesis that is specific?
-A specific hypothesis helps to focus the problem, making it easier to scope data collection and analysis, and avoiding vagueness that could lead to inadequate answers.
How does a hypothesis help in the threat hunting process?
-A hypothesis provides clarity in thinking about what is being looked for, helps reason about behavior naturally, and bridges narrative information about behavior to concrete analytics.
What is the purpose of creating a hypothesis in the context of threat hunting?
-Creating a hypothesis helps to provide focus for research, data collection, and analytic development, allowing for a deeper understanding of what an analytic does and what can trigger false positives.
Why should a hypothesis be falsifiable in scientific terms?
-A falsifiable hypothesis is one that can be disproven through testing, which is essential for scientific rigor and to avoid making claims that cannot be objectively evaluated.
What is an example of a hypothesis that is not falsifiable?
-An example of a non-falsifiable hypothesis is 'a malicious actor will use extreme stealth to operate in a way that will be indistinguishable from benign usage,' as there would be no evidence to examine if the claim were correct.
How does the process of hypothesis refinement help in threat hunting?
-Hypothesis refinement helps to account for nuances not captured during initial development and focuses on malicious usage, improving the accuracy and effectiveness of the hypothesis.
What should be the language of a hypothesis in the methodology stage?
-A hypothesis should be written in plain, human-understandable language to facilitate reasoning and understanding without the constraints of specific query syntax and to allow for sharing of thoughts and ideas.
Can you provide an example of how to refine a hypothesis based on the script?
-An initial hypothesis like 'if a task is scheduled, an adversary is establishing persistence' can be refined to 'if a task is scheduled by a non-admin user, an adversary is establishing persistence' to account for benign task scheduling by administrators.
Outlines
🔎 Developing Hypotheses and Abstract Analytics
This paragraph introduces Module 2 of the threat hunting methodology, focusing on the development and refinement of hypotheses and abstract analytics. It emphasizes the importance of using TTP insights to form testable hypotheses about potential malicious activity. The paragraph outlines the criteria for a well-formed hypothesis: it must be specific, evidence-driven, testable, and falsifiable. The process of hypothesis creation is described as iterative, allowing for continuous refinement based on evidence. The purpose of creating hypotheses in threat hunting is to clarify thinking, reason about behavior naturally, and bridge the gap between narrative information and concrete analytics. The paragraph also provides an example of how to refine a hypothesis to reduce the likelihood of false positives and ensure it remains falsifiable.
🚔 Hypothesis Refinement and Cybersecurity Application
The second paragraph delves deeper into the process of hypothesis refinement using the analogy of a burglar breaking into a home by kicking open locked doors. It illustrates how a hypothesis can be made more specific and less prone to false positives by incorporating key elements of the malicious technique. The paragraph also discusses the importance of gathering evidence to support or refute the hypothesis and the need to consider benign scenarios that could mimic malicious activity. A cyber-related example is provided, where the hypothesis evolves from a general statement about task scheduling to a more specific one that considers the user role in task creation. The paragraph concludes by reiterating the importance of a solid hypothesis being specific, evidence-driven, and falsifiable to guide effective research in cybersecurity.
Mindmap
Keywords
💡Hypothesis
💡Threat Hunting
💡Abstract Analytics
💡TTP (Tactics, Techniques, and Procedures)
💡Evidence-Driven
💡Falsifiable
💡Data Collection
💡Analytic Development
💡Malicious Actor
💡Iterative Process
💡False Positives
Highlights
Module 2 focuses on developing hypotheses and abstract analytics to explore for evidence of malicious actors.
Hypotheses guide data collection, analytic development and future hunting operations.
A hypothesis should be specific enough to be useful and help focus the problem.
Evidence from techniques, adversary behavior, and research should drive hypothesis development and refinement.
A good hypothesis is testable and can be proven or disproven through evidence.
Falsifiability is key - a hypothesis should be able to be disproven through testing.
Creating a hypothesis helps clarify thinking, reason about behavior, and bridge narrative to concrete analytics.
Hypothesis creation is an iterative process of continual updating and refinement based on evidence.
Evaluating falsifiability helps expose potential false alarm scenarios not captured initially.
A hypothesis should be written in plain language to facilitate reasoning and understanding.
Starting the hypothesis development process involves choosing a behavior and identifying evidence of malicious activity.
The example of burglars kicking open doors illustrates the need for specificity and falsifiability in hypotheses.
Continuous sensing and monitoring are required to gather evidence to support or refute a hypothesis.
Refined hypotheses should address identified nuances and focus on capturing malicious usage.
In the cyber example, the hypothesis evolves from 'if a task is scheduled' to 'if a non-admin user schedules a task'.
A solid hypothesis should be specific, evidence-driven, and falsifiable to effectively guide research.
Transcripts
hello and welcome to module two
developing hypotheses and Abstract
analytics this module will cover step
two of the threat hunting
methodology in it we will develop and
refine hypotheses and Abstract analytics
to explore hunting for evidence that
indicates a malicious actor may be
present we will also discuss the purpose
of and how to formulate abstract
analytics as well as how to leverage
external resources to help with this
effort during this step in the
methodology ology we will use TTP
insights to develop hypotheses that we
can test during our hunt in order to
make claims about malicious activity in
an
environment the hypotheses developed in
this step will guide our data collection
requirements analytic development and
future hunting
operations later on in the methodology
we'll use the collected data and
concrete analytics to test these
hypotheses hello and welcome to lesson
2.1 developing hypotheses
in this lesson we will describe the
purpose of and characteristics of a
well-formed
hypothesis so what is a
hypothesis the Oxford dictionary defines
a hypothesis as a supposition or
proposed explanation made on the basis
of limited evidence as a starting point
for further
investigation in other words a
hypothesis describes unproven but
suspected ideas about why something may
be
happening
a good hypothesis needs to meet certain
criteria the first of which is being
specific enough to be
useful a hypothesis that is too vague
doesn't help Focus the problem enough to
be adequately answerable for example
scoping what data to collect and what
time frame to cover amongst many other
factors being more specific helps to
hone in on a more focused statement to
drive research analysis and data
collection a good hypothesis should also
be evidence-driven throughout the
process of crafting a hypothesis you
should use as much evidence as possible
such as existing techniques and
knowledge of adversary behavior in ttps
as well as findings from your own
research and Hands-On
investigation evidence should also Drive
hypothesis refinement to account for
nuances not captured during initial
development your hypothesis should be
framed in a way that can be tested to
gain additional evidence here it is
important to think about what type of
evidence would support your initial
claim as well as what evidence would ref
it finally a good scientific hypothesis
should be falsifiable meaning it is able
to be disproven through
testing an example that is not
falsifiable would be a malicious actor
will use extreme stealth to operate in a
way that will be indistinguishable from
benign
usage given the way the statement is
written there would be no evidence to
examine if it were in fact correct and
thus it cannot be proven
false so why should we care about taking
the time to create hypothesis while
threat
hunting well a good hypothesis helps
clarify your thinking about what you're
looking for it also helps you reason
about behavior in a natural way without
getting bogged down in query syntax and
helps to bridge narrative information
about Behavior to concrete
analytics a good hypothesis will provide
Focus for research data collection and
analytic development that allows for a
deeper understanding of what an analytic
does what it means when an alert fires
and what can trigger false
positives hypothesis creation is truly
an iterative process that allows for
continual updating and refinement based
on
evidence during this process thinking
through and evaluating the statement's
falsifiability helps to expose potential
false alarm scenarios that were not
captured during initial
development these types of scenarios
help to capture those nuances and drive
hypothesis refinement in a way that
focuses on malicious
usage at this this stage in the
methodology a hypothesis should be
written in plain human understandable
language as it helps facilitate
reasoning and understanding in an
abstract way that avoids the constraints
of any specific query
syntax it also allows for sharing of
thoughts and ideas anal loss for
hypotheses to endure across changes in
implementation such as query language or
platform to begin this process start by
choosing a behavior and develop a
hypothesis around what evidence would
indicate that a malicious Act is
exhibiting this Behavior now let's walk
through some
examples in this first example we
observe that burglers sometimes enter
homes by kicking open locked doors to
steal
property this may lead us to develop the
hypothesis if the door opens a burglar
is breaking
in as we can see this hypothesis is much
too vague as it leaves lots of room for
false positives for example if a
homeowner enters they may also open the
door
a better hypothesis would be if the door
opens while still locked a burglar is
breaking
in this statement is more specific as it
incorporates key elements of the
malicious technique of kicking open
locked
doors it's important to note that
Gathering the evidence to either support
or refute this claim will require
continuous sensing to determine if the
door is open and if it is
locked this statement is also
falsifiable in that evidence can be
collected to show non-malicious opening
of the door without it being unlocked
for example emergency Personnel such as
a firefighter may open the door without
unlocking it in response to a fire alarm
we would need to think through some more
benign scenarios such as that one and
try to address it in future
iterations going into our last iteration
we have refined our hypothesis to read
if the door opens while locked but no
911 call has been made and no fire alarm
is active then a burglar is breaking in
this statement is still specific and
attempts to address the nuances that we
previously
identified it is also still falsifiable
as evidence can still be generated to
disprove the claim such as someone
calling 911 while a burglar is in fact
still breaking
in this statement however is much less
likely to be false compared to earlier
statements now to move on to a cyber
related example we have observed that
adversaries maintain persistence on a
compromised host by scheduling tasks for
example setting up malicious software to
run at startup or some other specified
time we begin with the hypothesis that
if a task is scheduled an adversary is
establishing
persistence this statement is somewhat
specific in that it incorporates key
elements of the behavior for example
scheduling tasks although it will also
require a continuous monitoring to
determine if a task is being or has been
scheduled it is also Al falsifiable and
that evidence can be obtained of benign
task scheduling such as by a system
administrator a refined hypothesis that
takes this fact into account is if a
task is scheduled by a non-admin user an
adversary is establishing
persistence again the statement is still
falsifiable but less likely to be false
than the previous one it also will not
catch instances of a malicious task
being scheduled by an administrator
which may be acceptable at this point
but a weakness of the hypothesis to keep
in mind as we move
forward in summary a solid hypothesis
should be specific evidence-driven and
falsifiable and it is important to have
a strong hypothesis as it will guide the
rest of your research
5.0 / 5 (0 votes)