Who Touched My GCP Project? Understanding the Principal Part in Cloud Audit Logs - Gabriel Fried
Summary
TLDRIn this insightful talk, Gabrielle Freed, a senior security researcher, delves into Google Cloud Platform (GCP) audit logs, focusing on the principal part and its significance in cybersecurity. He explains the importance of understanding identities—users, workload identities, and service accounts—in the context of GCP. Freed discusses the concept of impersonation, delegation chains, and the unique challenges in identifying service agents. He also touches on the practical aspects of investigating incidents within GCP environments, providing valuable insights for securing cloud-based operations.
Takeaways
- 🔒 The importance of understanding who accessed or modified GCP projects is emphasized for reducing time loss and avoiding unnecessary financial losses in cybersecurity incidents.
- 👤 The script introduces three main identity types in GCP: users, workload identities, and service accounts, each serving different purposes and access methods within the GCP environment.
- 🔑 Service accounts in GCP are used for service-to-service operations and can be identified by the 'gs://service-account.com' prefix. They can be user-managed or Google-managed, with different authentication and access control mechanisms.
- 🔄 The concept of service account impersonation is explained, allowing one service account to act on behalf of another, which can be legitimate for tasks like delegated access or potentially malicious for privilege escalation.
- 🔗 Delegation chains are highlighted as a series of impersonations where a service account may need to impersonate multiple other service accounts to access resources, which is crucial for tracing back the origin of operations in an incident.
- 📝 The structure of cloud audit logs is detailed, including authentication info and service account delegation info, which are vital for understanding the 'who, what, and where' of actions taken within GCP.
- 📡 Workload identities offer a solution for external entities to access GCP resources securely without the need for managing keys, using an IDP trust and token exchange process.
- 👥 Workforce identity federation is mentioned as a similar solution for human users, allowing them direct access to GCP resources after authentication with an IDP, without the need for impersonation.
- 🕵️♂️ The speaker suggests using the knowledge of identities and logs to investigate incidents effectively, tracing actions back to their origin and understanding the context of operations within GCP.
- ⚠️ The script warns about cases where Google redacts caller IPs or identities for privacy reasons, and advises checking Google's documentation for understanding such cases in logs.
- 📈 The talk concludes with a call to action for attendees to review their GCP logs to understand the activities, and in time, to be able to write detections based on their environment's impersonation patterns.
Q & A
What is the main focus of the talk 'Who Touched My GCP Project'?
-The talk focuses on understanding the principal part in Cloud audit logs to identify who performed actions within a GCP project, which is crucial for security incident investigation and resolution.
Why is it important to understand who touched a GCP project?
-Understanding who touched a GCP project is important to reduce time loss and potential financial loss due to false investigations, and to quickly resolve security incidents by identifying the actions and impacts of the involved entities.
What is the role of Periso in cloud security?
-Periso is a threat detection company that provides a purpose-built platform for detecting threats in cloud-based environments. It uses human and non-human identity attribution to analyze data across authentication boundaries for actionable detections.
What are the three main identity types in GCP?
-The three main identity types in GCP are users, workload identities, and service accounts. Users are human entities, workload identities are for external entities, and service accounts are for service-to-service operations.
What are the two main sections of cloud audit logs in GCP?
-The two main sections of cloud audit logs in GCP are data access and admin activity. Data access logs record access to certain data, while admin activity logs record administrative operations.
What is a service account in GCP and how is it used?
-A service account in GCP provides an identity for a resource to access another resource. It is used for service-to-service operations, such as when an application on a VM needs to access a GCP bucket.
What are the two methods of accessing a service account in GCP?
-The two methods of accessing a service account in GCP are activation and impersonation. Activation involves logging in as the service account, while impersonation allows a service account to act on behalf of a user to access a resource.
What is the purpose of service account impersonation in GCP?
-Service account impersonation allows a service account to act on behalf of a user to access a resource. It can be used for legitimate cases like delegating access or for malicious purposes such as data exfiltration or privilege escalation.
What is a service agent in GCP and how does it differ from a service account?
-A service agent in GCP acts on behalf of certain services and is usually attached to them for logging operations. Unlike service accounts, service agents are completely managed by Google and cannot be impersonated.
What is workload identity and how does it provide a solution for external entities accessing GCP resources?
-Workload identity provides external cloud providers or SaaS applications with an identity to access GCP resources without the need for keys. It uses keyless authentication through an IDP trust to GCP, allowing external entities to authenticate and access GCP resources securely.
What is the significance of the Google Kubernetes Engine default service account in the context of the talk?
-The Google Kubernetes Engine default service account is significant because it is often used for operations within the Kubernetes environment. Any operation from a VM, even with an attached service account, happens through a delegation from the default service account, which is an important behavior to recognize during investigations.
Outlines
🔍 Introduction to GCP Audit Logs and Sponsors
The speaker, Gabrielle, begins by acknowledging the sponsors, specifically Periso, a threat detection company specializing in cloud-based environments. Periso's platform is highlighted for its ability to analyze human and non-human behaviors in cloud and SaaS environments, providing high fidelity and actionable detections. Gabrielle introduces the talk's focus on understanding the principal part in Google Cloud Platform (GCP) audit logs, emphasizing the importance of identifying who accessed or modified GCP resources to prevent time and money loss due to false investigations.
🛠 Understanding Service Accounts and Service Agents in GCP
Gabrielle explains the concept of service accounts in GCP, which provide an identity for a resource to access another resource, using the example of an application needing to access a GCP bucket. Three types of service accounts are discussed: user-managed, Google-managed, and service agents. The speaker outlines the methods of accessing service accounts, including activation and impersonation, and the importance of understanding these in the context of audit logs. The talk also touches on the legitimate and malicious use cases for impersonation and the concept of delegation chains in service account operations.
🔗 Delegation Chains and Service Agent Operations
The speaker delves into the intricacies of service account delegation chains, where multiple impersonations may be necessary due to insufficient permissions. Gabrielle clarifies how to identify delegation chains and single invocations in audit logs by examining the permission codes and metadata. The discussion includes the default service account behavior in Google Compute Engine and the role of service agents, which are managed by Google and cannot be impersonated. Examples of service agents and their representation in logs are provided, along with a cautionary note on their proper identification to prevent security risks.
🌐 Workload Identities and Workforce Identity Federation
Gabrielle introduces workload identities, which allow external cloud providers or SaaS applications to access GCP resources securely without the need for managing keys. The process of authenticating through an external Identity Provider (IDP) and obtaining a GCP access token is explained. The speaker also covers how these identities appear in logs, with examples of token exchanges and impersonations. Workforce identity federation is briefly mentioned as a similar solution for users, providing direct access to GCP resources after IDP authentication.
🕵️♂️ Analyzing GCP Logs for Incident Investigation
The speaker provides insights into using GCP logs for incident investigation, highlighting how to trace operations back to their origins using session IDs and other identifiers. Gabrielle discusses the importance of recognizing default service accounts associated with services like Google Kubernetes Engine and the potential for Google to redact caller IPs or identities for privacy reasons. The talk concludes with a challenge for the audience to analyze their own GCP logs, understand the activities, and write detections based on their environment's impersonation patterns.
🤝 Closing Remarks and Audience Interaction
Gabrielle concludes the presentation by summarizing the key points about different identities in GCP and how they can be investigated through logs. The speaker encourages the audience to engage with their GCP logs to enhance their understanding and detection capabilities. A Q&A session follows, where Gabrielle addresses questions about identifying service accounts that mimic service agents and the use of Google's access transparency logs, showing a readiness to share knowledge and learn from the audience's experiences.
Mindmap
Keywords
💡GCP
💡Cloud Audit Logs
💡Principal
💡Service Accounts
💡Impersonation
💡Delegation Chains
💡Workload Identities
💡Workforce Identity Federation
💡Service Agents
💡Redaction
💡Forensics
Highlights
Introduction to the importance of understanding principal parts in Cloud audit logs for GCP security.
Sponsorship acknowledgment for Pera, a threat detection company specializing in cloud-based environments.
Explanation of how Pera's platform uses human and non-human identity attribution for high-fidelity detections.
The necessity of knowing 'who touched our GCP project' to mitigate time loss and financial loss from false investigations.
Overview of the three main identity types in GCP: users, workload identity, and service accounts.
Description of the four log categories in GCP: service logs, agent logs, network logs, and cloud audit logs.
Emphasis on the significance of cloud audit logs in determining 'who did what and where' in GCP.
Discussion on the authentication information provided in cloud audit logs, including principal email and service account key details.
Explanation of service account delegation and how it appears in audit logs.
Introduction to service accounts, their purpose, and how they are identified.
Differentiation between user, custom-built, and Google-managed service accounts.
Explanation of service account activation and impersonation, and their respective commands.
Identification of legitimate and malicious use cases for service account impersonation.
How service account operations appear in audit logs, including activation and impersonation.
Concept of service account delegation chains and their necessity for accessing resources.
Detailed look at how Google Compute Engine default service account operates and its appearance in logs.
Introduction to Service agents, their function, and how they differ from service accounts.
Explanation of workload identities and their role in providing external entities access to GCP resources.
Description of the process for external entities to authenticate and access GCP resources using workload identities.
How workload identity operations appear in logs, including token exchange and impersonation.
Introduction to Workforce identity Federation and its benefits over traditional user management.
Summary of how to use the understanding of identities and logs to investigate incidents in GCP.
Recommendation for the audience to check their GCP logs to understand and investigate incidents.
Q&A session discussing the challenges of identifying malicious service accounts mimicking service agents.
Mention of Google's Access Transparency logs and their potential use in investigations.
Transcripts
all right we're going to uh we're going
to get right back into it uh our next
talk is uh who touched my gcp project
understanding the principal part in
Cloud audit logs uh before we get
started I'd like to thank our sponsors
specifically per
miso uh permiso is a threat detection
company that finds evil in cloud-based
environments detection in identity cloud
and SAS environments have suffered too
long from a one siiz fits-all sim
approach modern environments require a
purpose-built platform periso uses human
non-human identity attribution to pair
runtime and static data in cloud and SAS
into complete sessions where detections
no longer have to rely on single events
pera's platform provides High Fidelity
High resiliency and maximum content for
actionable detections that cut through
the Noise by analyzing human and
non-human entitlements and behavior
across authentication boundaries periso
gives companies a complete picture of
every entity operating in and across
their Cloud environments and does not
allow threat actors to hide in silos of
overwhelming Cloud activity data please
welcome Gabrielle
[Applause]
hello can you hear me awesome hey
everyone and welcome to who touch my gcp
project where today we're going to learn
about the principal part in gcp audit
logs every day in our environment we
could have gcp buckets being accessed
jobs being created and unfortunately
cyber security incidents now sadly we
could lose a lot of time and
unfortunately money on false
investigations when we can understand
who who did what and how so in order to
resolve this and have lower time loss
and possibly not any loss of money it's
critical for us to understand who
touched our gcp project so in order to
do so I'd like to go today over some gcp
principles and logs we have service
accounts and Service agents where do
they come into play workload identities
and Workforce identities what solution
they provide and we'll mention some
cases that Google redacts caller
identities and IP
identities I'm Gabriel freed I'm a
senior security researcher at mtiga I
have 10 years experience in cyber
security this is my first conference I'm
talking in and I'm a whiskey dude so if
anyone knows where I could get a drink
please let me
know so let's talk about gcp principle
and logs we
have in gcp we could categorize three
main identity types we have users
workload identity and service accounts
where Workforce identities which I
mentioned are somehow a hybrid between
the first two users are like human
entities like me and you logging in with
our emails to access our gcp resources
workload identities provide external
entities coming from things like
external Cloud providers in SAS access
to our gcp resources and service
accounts are for service to service
operations meaning if we have some
resource needing tax as another it will
use this as as its
entity in gcp we have four kind of log
categories we could which we could
categorize we have service logs which
are for standard error and outputs we
have agent logs which are for agents
which we could install on certain gcp
resources and get information from that
agent we have Network logs which are for
VPC flow logs Nat gateways and firewall
rules and what will focus on
today the cloud audit logs the cloud
audit logs tell us who did what and
where and they could be categorized into
two sections data access and admin
activity where data access is stuff
we're using to access certain data and
admin activity is about administrative
operations an example for this could be
a creation of a bucket would be under
admin activity but accessing the data in
the bucket would be under data access so
these are the four types of logs you
could categorize in
gcp in our Cloud audit logs we have a
section called authentication info this
tells us who did what and
how and it tells us the authentication
method a principal email a service
account key information which we'll talk
about in service account section it will
tell us about third party information
and some delegation info for delegated
impersonations which we'll talk about
later later on this structure looks
something like
this we could see in this structure we
have another structure called service
account delegation
info this is for impersonation chain of
service account and Service agents which
we'll talk about later down the line but
this section just to keep in mind looks
something like this so something small
we can remember we have four SE types of
logs three main identities and two
interesting structures so once we talked
about that let's shift to the service
account and Service
agents what are service accounts service
accounts provide an identity for a
resource to access another resource and
an example usage for this to get this
into our uh head is kind of like having
an application on a VM which needs to
access our gcp bucket so this will be
done using a service account we could
identify these by the GS service
account.com prefix
wait okay there are three types of
service accounts service accounts just
to a head start when they authenticate
they needs credentials which are usually
ajon or p12 key so we have three main
categories we have user which are
custommade built-in service accounts and
Google manage service accounts the
custom ones that a user could create are
we could act we could give them certain
types of roles specific Scopes and we
manage their keys
for access the Google built-in service
accounts are kind of Google's way to
give to give us a head start on certain
services for example if we have the
Google compute engine and we enable this
API Google will create the service
account and its keys to get us started
and the Google managed are technically
what we will later refer to as the
service agents but these are for
internal operations happening in our
Google environment and their keys are
completely managed by
Google so
how can we access these service accounts
well we have two types of meth methods
of accessing them through through
activation and through impersonation
when we activate a service account we're
essentially logging in as the service
account we need to provide a key and
then we're essentially if our g-cloud uh
command is activating it we're kind of
logged in as disservice account this
could be done using this command
now this is what I'd call a bad
impersonation but now let's talk about
service account
impersonations a service account
personation allows a service account to
act on our behalf to access a certain
Resource as we could see here a user
might need to impersonate a service
account to access a resource this
service account has this will allow us
to have temporary credentials to the
service account to access the
resource now we could have legitimate
cases for impersonation some malicious
cases for
impersonation and two types of
impersonations a legitimate case could
be when we want to delegate access
because we don't want our users to
always have access to everything we
might need this to automate tasks on
certain software we have to impersonate
service accounts for resources and we
might want to De like give external
entities access to our resources which
we'll talk about later on to get um
resources from outside into our gcp
environment a malicious actor might use
this to either exfiltrate data cuz a
service account might have the right
permissions escalate their privileges
because it might have higher Privileges
and certain types of attacks using their
behavior and to possibly cover their
tracks why because you might say Okay a
service account is operating in these
logs it's all
good now there are two forms of
impersonation one is the activation we
mentioned previously and one is a
specific
invocation what happens when we have a
specific invocation we're essentially
getting temporary credentials to act on
one certain command to to allow this
service account to act on our behalf
what we need for this is a role called
service account token Creator why
because what happens when we impersonate
we're generating an access token which
allows us to have temporary credentials
for the service account to allow it to
operate on our behalf so we understood
so far that in service accounts we have
activation and impersonation let's see
how this looks in our audit
logs when we're activating a service
account as we mentioned we're
essentially logging in as the service
account so the authentication info
section will tell us our principal email
was the actual service account operating
because now our session is as the
service account but when we impersonate
a service account we will see our user
which is instigating this specific
invocation in
impersonation doing a generate access
token operation to get the token for the
service account and in our resource
section of the logs we will see our
service account as what we're trying to
impersonate so this is how these look in
the logs but we mentioned before
something called delegation chains so
what is that a service account
delegation chain means that sometimes we
might need to have several
impersonations going out because service
account needs doesn't have the
sufficient permissions to do something
it needs another service account or you
might want to move around some type of
permissions so a user might impersonate
service account a which needs to
impersonate service account B to access
our resources for this we'll need the
get access token and again the service
account token
Creator now let's see how this looks and
belongs when we're starting a delegation
chain we will see the invoker or in this
case our username
doing a generate access token for the
full list of service account we need to
impersonate because each entity in this
chain will now also need to generate an
access token for the next one in the
chain but when we see sorry yes but when
we see the operations happening with the
delegation chain what we'll actually see
is oh sorry is the last service account
in our delegation chain which we needed
to actually access our resources but in
the um identity delegation info we'll
see the reverse order of who actually
initiated this operation which is
critical because if we see an operation
an operation happen in an incident you
could trace this back to the actual
original
invoker but sometimes because we
mentioned each one needs to generate an
access token we're not sure if this
started as a specific invocation or a
delegation chain well for this I have a
tip you could look at the permission Cod
when we see an event for generating a
token if we have a single invocation our
permission will be the get access token
and our identity will only involve one
service account but when we're starting
a delegation our permission scope will
involve implicit delegation and in our
resources we will see the information uh
sorry in our metad data we will see the
information of all the identities we
want to
impersonate so so far we understood that
service accounts have these operations
impersonations and delegation chains now
I'd like to talk about the curist case
the Google uh compute engine default
service account we mentioned that
default service accounts are kind of
like a hit start to our operations right
so one of these is the Google compute
engine service now Google allows us to
attach our own custom service account to
AVM if we want certain operations to
happen and this will be the entity
acting for this virtual machine kind of
like managed identities if anyone knows
how they are in Azure so what I'd expect
the behavior to be is that we see an
operation from this VM Happening by this
attached uh service account what we
noticed
is that each operation even with an
attached service account actually
happens by a delegation from the default
computer engine service account so
seeing this is not like something
unusual but it's just unusual as a
behavior and something cool to
notice so now let's talk about Service
agents Service agents act on behalf of
like certain type of services in Google
they're usually attached to them like
the Google Sync uh for logging operation
these things are logged for transparency
and their keys that we mentioned before
like we use for Activation are
completely managed by Google and
therefore you cannot impersonate Service
agents uh here are some example of
Service agents but I'd highly recommend
looking at Google's documentation for a
full list let's see how Service agents
look in our logs um here again we see
our Google compute engine default
service account which we remember and
this was actually an operation invoked
by our service agent as we see over here
so this kind of sums up our service
agents and service account section we
learn about impersonation delegation
chains word cases with the GCE and
Service agents now let's talk about
workload identities and Workforce
identities workload identities provide
external cloud cloud providers or SAS
applications and identity to access our
gcp resources why would we need the
solution because we mentioned that
sometimes we might want to uh get a
service account through keys or log into
our gcp environment and having keys
around is a security risk and we also
don't want the complexity of managing
external entities so what solution do we
have the solution here is that we have
keyless authentication because we have
an IDP trust to our gcp environment now
let's see how this
works our workload entity will then will
first start by authenticating with its
external IDP it will then get its
credentials back from the IDP and send
these credentials to the Google SDS
service which is the security token
service in Google which verifies the
authentication with the IDP once this
entity is is verified the SDS will
return a gcp access token now we
remember what we could do with these
tokens we can
impersonate a service account which
gives us access to the gcp resource so
this is how this works now let's see how
we could identify these things in our
logs let's talk about the SDS to token
Exchange in our
logs something really cool here is we
have an external workload identity
coming from AWS where we could see this
the account ID the role it assumed and
the session ID which means like for
forensic investigations this is gold
because we could trace something all the
way back to a specific session ID in
AWS um we will see here um Google giving
us a mapped identity for this external
entity this is how this identity will
now be represented in our logs and the
operation will be the token exchange for
Google here's another example but why am
i showing you first the mapped identity
because what we see here is that when it
comes from Azure we get the object ID
which we could trace back to a to Azure
if we had an incident starting from
there so once we have our token we our
STDs token we want to impersonate a
service account so what we see here is
that we're starting with our maap entity
which we saw in the token Exchange
change doing a generate access token for
a resource which is the service account
which we need to impersonate to access
our
logs this before what we saw sorry was
an AWS which we know what we could trace
back but now let's see what happened
once we
impersonated once we're impersonated or
AWS we will see a service account
operating cuz we already impersonated
and our our principal subject in the in
the
in the service account delegation info
will be our external entity now GitHub
has something called GitHub actions
which are for cicd operations which on
uh which kind of like on P requests
could allow us to like spawn VMS to try
check our deployment so what we see here
is a service account operating but our
principal subject is actually a GitHub
and we get it straight from the
repository the path all the way up to
the pr of what spawn or did actions in
our G GP environment which is really
cool cuz if someone had access to our GC
to our GitHub we can actually trace this
all the way back to
gcp let's talk a little bit about work
force identity Federation this is
essentially kind of like the same
solution but for users but without
impersonation a user authenticates the
IDP and then has direct access to our
gcp environment the solution here is
that also we don't have to sync users if
anyone knows entra ID and on Prem you
have to sync your users and you have to
manage them we could manage them using
Workforce identity
pools this is how Google describes the
operation fairly similar but no
impersonation where we authenticate to
the IDP get a token verified and access
our gcp resources the way this will look
in our logs is fairly similar the way
our user authenticated to its IDP in
this case a user a user add an
email Short Pop Quiz let's see if anyone
got anything so far few seconds let's
see if anyone can understand what
happened
here so what happened here something I
need I I I stumbled upon myself was that
I saw a service account operating but I
saw this uh principal subject that SVC
ID goo which I didn't understand what it
was this is actually
the Google kubernetes engine default
service account attached with an IM
policy to a service account so this is a
hint if we're investigating anything
from Google kubernetes engine and we see
the SBC goog we could even Trace things
back to the name space and the
kubernetes service account that we had
inside our environment which is really
cool for
investigations now let's have a short
brief about cases where Google redacts
caller IPS or caller
identities uh in big query sometimes we
have cross project access with different
domains so Google might redact this for
privacy gdpr and all those fun reasons
uh call or email in Virtual networks
they might call it Google internal and
if we have the MS without external IPS
they might just call it like Google
internal or remove it I highly recommend
looking at Google's documentation for
all the use cases we
have so in order to summarize this up We
Now understand the different types of
identities how they look in our logs and
how we could investig investigate these
incidents I would like you tomorrow to
check your gcp logs see if you could
understand what happened there in a week
from now if you have an incident can you
tie it back and in a month write
detections based on your uh environment
impersonation patterns and what you know
if there's one thing I could have you
all take away from this is that once we
understand the principles we could
easily investigate and understand who
touched our gcp project thank you very
much that was gel
freed were all the were all the logs
that you showed in the default audit
logs that are enabled by default or did
you have to turn on data access dat yeah
so some of them were from data access
and someone for the from the audit LS
that that that link oh this link is just
my LinkedIn if anyone feel free yeah
yeah uh one other question great talk by
the way um one of the things we've
noticed when we see ransomware in gcp is
that um their mechanism for persistence
is to create a service account but try
to make it look like a service agent
with nomenclature um do you have any
tips on like identifying that inside the
delegation chain or anything like that
so A few things you could have is first
of all Google has an immense list of all
the service agents there are specific
like uh resources which have designated
Service agents another cool thing which
I also noticed myself is that each
listed service agent they might try to
Def fake has a list of of specific uh
roles it should have so if you see it
acting something that it shouldn't be
doing that's a really cool point because
for some reason Google allows you to add
additional roles but they're like don't
do it you might break stuff but yeah so
if you'd see it
operating hi uh so I actually work at
Google then that's it's a good talk I
actually didn't know as much about uh Ro
delegation and service account we
because I re use mostly Borg but uh we I
I have a service called access
transparency which which I don't know if
youve used which tells you when googlers
access your data have you had experience
with that and I thought I was getting
your thoughts so I'm aware of the access
transparency log which like Google kind
of like allows you to see what they're
doing but I haven't seen the logs myself
to ingate cuz like what I know from Main
detection and investigation people go to
the cloud audit but that's something I
definitely should look
into thank Gabrielle cheers
[Applause]
Посмотреть больше похожих видео
Google Cloud Platform Tutorial - Part #1 | Introduction to GCP | Cloud Computing Basics | @SCALER
Chapter #8 - Cloud IAM Basics | identity & access management on google cloud platform (gcp)
GCP Data Engineer Live Q&A for job readiness
Connect to services on another VPC via Private Service Connect (PSC)
DE Zoomcamp 1.3.1 - Introduction to Terraform Concepts & GCP Pre-Requisites
Your Complete Cloud Cheatsheet for Cracking Accenture!
5.0 / 5 (0 votes)