The only Cloud services you actually need to know
Summary
TLDRThis script offers an insightful overview of cloud services, emphasizing the ubiquity of virtual machines as the foundational layer for all cloud offerings. It distinguishes between managed and unmanaged services, elucidating the convenience of services like AWS's S3 and RDS, which abstract infrastructure management. The talk also touches on the concepts of serverless computing with AWS Lambda and the importance of observability in cloud applications. Additionally, it discusses the trade-offs between regional and global services, highlighting the scalability and latency benefits of global databases.
Takeaways
- đ€ The script discusses the redundancy in cloud services, emphasizing that most services are built on the same underlying technology, such as virtual machines (VMs).
- đ It highlights the importance of understanding the top five cloud services and recognizing that many other services are just repackagings of these core offerings.
- đ The speaker introduces the concept of managed versus unmanaged services, explaining that managed services abstract away the infrastructure management, while unmanaged services require more hands-on control.
- đ The difference between regional and global services is explored, with regional services being confined to a specific location and global services distributed worldwide.
- đŸ The script explains that object stores like Amazon S3 are managed services that simplify file storage without worrying about the underlying infrastructure.
- đ It also covers the complexity of databases and how cloud providers offer managed database services like Amazon RDS and DynamoDB to alleviate the burden of database management.
- đ The concept of vendor lock-in is introduced, where using proprietary services like DynamoDB can make it difficult to migrate to other platforms.
- đ ïž The script touches on AWS Lambda, a serverless compute service that abstracts away the infrastructure, allowing developers to focus solely on their code.
- đ Networking services like load balancers and CDNs are mentioned as important but not the focus of the script; their role in cloud architecture is acknowledged.
- đ Observability is discussed as crucial for monitoring and logging application performance, with external tools like Datadog offering enhanced capabilities over native cloud provider tools.
- đ Data warehouses and specialized platforms like Snowflake are presented as alternatives to cloud provider offerings, suggesting that focused platforms may provide superior solutions.
Q & A
Why do many people prefer using cloud services instead of hosting their own databases?
-People prefer cloud services because they offer managed solutions that abstract away the complexities of hosting databases, such as replication, scaling, and backups, allowing users to focus on their applications rather than infrastructure management.
What is a Virtual Machine (VM) in the context of cloud services?
-A Virtual Machine (VM) is a service that allows users to rent a computer in the cloud, complete with allocated resources like CPU, RAM, and disk space, which can be remotely accessed and managed, often via SSH.
What is the main difference between managed and unmanaged services in cloud computing?
-Managed services are those where the cloud provider handles the infrastructure, scaling, and maintenance, abstracting away the underlying complexity from the user. Unmanaged services, on the other hand, only provide the basic resources, leaving the user to manage and configure the environment as needed.
Can you explain the concept of 'serverless' in cloud computing?
-Serverless refers to the abstraction of server management, where the cloud provider dynamically manages the allocation of machine resources. The user submits their code to the cloud, which then runs it in an environment without the user having to manage servers or runtime environments.
What is Amazon S3 and how does it differ from storing files on a VM?
-Amazon S3, or Simple Storage Service, is an object store that allows users to store and retrieve any amount of data from anywhere on the web. It differs from storing files on a VM in that it abstracts away the complexities of file management, such as scaling storage and handling file replication, providing a simple interface for file operations.
What is the significance of understanding the difference between regional and global cloud services?
-Understanding the difference is crucial for optimizing latency, cost, and data residency requirements. Regional services are hosted in specific locations, which can affect performance for users outside that region. Global services, on the other hand, distribute data and processing across multiple regions, reducing latency and improving performance for a worldwide user base.
What are some examples of managed database services offered by cloud providers?
-Examples of managed database services include Amazon RDS for relational databases and Amazon DynamoDB for NoSQL databases. Other providers like Google Cloud and Microsoft Azure offer similar services such as Google Cloud SQL, Firestore, and Azure Cosmos DB.
What is AWS Lambda and how does it fit into the serverless computing paradigm?
-AWS Lambda is a Functions-as-a-Service (FaaS) offering that allows users to run code without provisioning or managing servers. It is a highly serverless service as it abstracts away almost all infrastructure management, executing code in response to events and automatically scaling the compute resources.
What is meant by 'vendor lock-in' in the context of cloud services?
-Vendor lock-in refers to the situation where an application is built using proprietary services or tools offered by a specific cloud provider, making it difficult to migrate to another provider without significant changes or data migration due to the lack of equivalent services.
How do observability services like Datadog differ from those provided by major cloud providers?
-Observability services like Datadog specialize in providing comprehensive monitoring, logging, and alerting solutions that are often considered easier to use and more feature-rich than those offered by cloud providers. However, they may require additional setup and integration work compared to the native tools provided by cloud services.
What is the role of a Content Delivery Network (CDN) in cloud architecture?
-A CDN is a globally distributed network of proxy servers and data centers that deliver content to users based on their geographic location, ensuring faster delivery of web content and reducing latency. It is inherently a global service due to its nature of distributing content worldwide.
Outlines
đ Understanding Cloud Services and Virtual Machines
The speaker begins by addressing the common sentiment that hosting one's own database is undesirable and that AWS, while not necessarily innovative, offers a plethora of cloud services that can be overwhelming to understand. The focus is on explaining the underlying similarities among cloud services and highlighting the top five services to understand deeply. The concept of managed versus unmanaged services is introduced, with virtual machines (VMs) being the foundation of most cloud services. VMs provide a sense of renting a computer with allocated resources, and they form the basis for other services. The speaker emphasizes the extensibility of VMs and their role as an unmanaged service, where the cloud provider only allocates resources and the user has full control.
đ Managed Services: Storage and Databases
The speaker discusses the benefits of managed services, starting with file storage solutions like AWS's S3, which abstracts the complexities of file management and provides scalability and simplicity. S3 is an example of a managed service where AWS handles the infrastructure, allowing developers to focus solely on their application logic. The speaker also touches on the challenges of managing databases, which are complex and often best handled by managed database services like AWS's RDS for relational databases or DynamoDB for non-relational databases. The use of open-source software by cloud providers to create managed services is highlighted, as well as the proprietary nature of some services like DynamoDB, which can lead to vendor lock-in.
đ The Spectrum of Cloud Services: From VMs to Serverless
The speaker explores the spectrum of cloud services, from fully managed services like AWS Lambda, which is a serverless compute service, to less managed services like VMs. AWS Lambda represents a high level of abstraction where the developer only needs to focus on the code, with the cloud provider handling the execution environment and scaling. The speaker also discusses the trade-offs of serverless architectures, such as the lack of persistent state and the limitations on disk access. The importance of understanding the spectrum of services is emphasized, as it allows developers to choose the right tool for their specific needs.
đ Networking, Observability, and Specialized Services
The speaker briefly mentions networking services like load balancers and CDNs, which are essential for managing traffic and delivering content globally. The focus then shifts to observability, which involves logging, monitoring, and alerting to ensure applications run smoothly. External tools like Datadog are highlighted for their ability to simplify observability tasks. The discussion also includes data warehouses and specialized platforms like Snowflake and Databricks that excel in specific areas, offering potentially better solutions than general cloud provider services. The speaker emphasizes the importance of trying out these services to gain hands-on experience.
đ Regional vs. Global Services and the Power of Global Databases
The speaker concludes by differentiating between regional and global services, using a CDN as an example of a global service and VMs as inherently regional. The benefits of global databases, which manage data distribution and replication to reduce latency and handle scale, are underscored. The speaker notes that while global services and managed services tend to be more expensive, they solve complex problems and can be worth the cost. The importance of practical experience with these services is reiterated, encouraging users to build applications using them.
Mindmap
Keywords
đĄCloud Services
đĄVirtual Machine (VM)
đĄManaged Services
đĄUnmanaged Services
đĄServerless
đĄDatabase Services
đĄVendor Lock-in
đĄAWS Lambda
đĄObservability
đĄData Warehouse
đĄRegional and Global Services
Highlights
The speaker emphasizes the ubiquity of cloud services and the difficulty in keeping up with the vast array of offerings, particularly from AWS.
An explanation is provided on why most cloud services are fundamentally similar, suggesting that understanding a few core services can generalize to others.
The importance of understanding the concept of managed versus unmanaged services and what 'serverless' truly means in the context of cloud computing.
A detailed exploration of the differences between regional and global services in cloud computing, and their implications for latency and data storage.
An introduction to the concept of a Virtual Machine (VM) as a fundamental building block for cloud services.
The extensibility of VMs and how they serve as the base for almost every cloud service, including compute and storage.
A discussion on the unmanaged nature of VMs, where the cloud provider only provisions resources and the user has full control.
The introduction of Amazon S3 as an example of a managed service for file storage, abstracting away the complexities of file management.
The benefits of using managed services like S3, including scalability, ease of use, and automatic data replication.
An explanation of the complexities involved in self-hosting databases versus using cloud-managed database services like AWS RDS.
The role of open-source software in cloud services, with AWS RDS being an example of a managed service built on open-source databases.
The concept of vendor lock-in when using proprietary cloud services like AWS DynamoDB, and the implications for scalability and cost.
An introduction to AWS Lambda as a serverless computing service, abstracting away infrastructure management from the developer.
The trade-offs of using serverless services like Lambda, including the lack of persistent state and the limitations on disk access.
A brief overview of networking services in cloud computing, such as load balancers and CDNs, and their importance for application performance.
The importance of observability in cloud services, including logging, monitoring, and alerts for maintaining application health.
The role of third-party services like Datadog in providing enhanced observability compared to native cloud provider tools.
A discussion on data warehouses and the offerings from cloud providers like AWS Redshift and Google BigQuery, and the emergence of specialized platforms like Snowflake.
The emergence of companies like Versel that aim to simplify the use of AWS resources by providing a more user-friendly interface.
The speaker's recommendation for hands-on experience with cloud services, emphasizing the importance of practical application over theoretical knowledge.
Transcripts
nobody wants to host their own database
AWS isn't doing anything creative
they're actually just taking open-
Source soft logic wise we can do pretty
much everything with these Services
nowadays there's so many different cloud
services that even just knowing the
names of all the services that AWS
offers is difficult let alone
understanding what they actually do well
the good thing is that I'm going to
explain to you why most of the cloud
services are all really the same under
the hood and I'm also going to be giving
you roughly my top five services that
you should actually understand and know
and everything else is just a
repackaging of those top five we're
going to go even deeper than that and
learn about the differences between
managed and unmanaged services because I
think that's kind of a buzzword nowadays
it kind of gets into the whole meaning
of what does serverless actually mean
we'll also touch on the difference
between a regional and a global Service
and we'll also be talking about all
kinds of different cloud services and
providers from the big boys like AWS gcp
as well as the startups versel kind of
the midsize companies like snowflake
data dog and kind of talk about how they
fit into the overall picture now the
single most important service to know
and to know really well is something
that you're probably already familiar
with you can in a sense rent a computer
as a service it's called a VM a virtual
machine now technically you might not be
renting a physical computer but you can
think of it as that cuz you'll have
resources allocated to your VM for
example some CPU some amount of ram disk
space which you can usually add on to so
from your perspective you're getting a
computer well it turns out that pretty
much every single service is built on
top of this any cloud service you're
going to use is in a sense built on top
of VMS if you want to run some type of
compute or server it's going to be
running on top of some type of machine
using some CPU and other resources if
you want to store a bunch of files it's
going to be stored on top of a computer
because even though the files are going
to be stored probably on disk well to
read and write to those files you're
going to need some type of computer to
run a database I probably don't have to
tell you it's going to require all of
these everything is just built on top of
computers and pretty much every cloud
provider will offer some like this
Amazon has ec2 Google has compute engine
I'm sure Azure has their own the good
thing about these is that they're so
extensible if you really wanted to you
don't need any of those other services
you can do everything you need to using
just VMS and so this is an example of an
unmanaged service because the cloud
provider isn't really doing anything
other than provisioning the resources
and then letting you do whatever you
want to and of course they're not going
to like physically mail the VM to you're
going to be connecting to it remotely
probably via SSH now what if you want to
store some files such as images videos
and other types of assets that you
probably need for your app yeah you can
store them on the VM and then serve them
from the VM that would work but
computers aren't specialized for
handling files so a better solution a
more scalable solution and just more
efficient in general is to use some type
of file storage mechanism on AWS it's
called S3 simple storage service it's
basically an object store that's what
they're called and so an object store is
kind of like a black box even though
I've drawn it as like a white box here
but all you really do is read and write
files to it you can delete files of
course but it's very simple the
interface from a programming perspective
is really really simple compared to
manually handling files via like a VM
you don't worry about where they're
being stored you don't worry about
allocating additional space with a VM
you'll have let's say like 100 gbt or
something you know you try to upload a
file and you run out of space you don't
have space on cloud services it's easy
to get more space but it's something you
still have have to kind of manage
yourself as a developer well object
stores like S3 abstract all of that away
for you you'll never run out of space
you'll probably never need to do
anything to scale your read and writs
now as you can imagine these files are
still being stored somewhere they're
being stored on a disk somewhere
persistently and to read and write the
files of course you're using CPU and
probably Ram but again you don't have to
worry about that this is an example of a
managed service because AWS or your
cloud provider is managing all the
infrastructure for you it's also
sometimes called serverless but it's not
to say that there isn't a server
involved there definitely is you can't
do any of this stuff without computers
but it's abstracted away from you and
that's really powerful cuz all you have
to do is worry about your application
logic reading and writing the correct
files you don't worry about the
infrastructure at all it's obviously
less powerful in the sense that you
don't have precise control over what
this service is doing but you usually
have enough control like if you wanted
your files to be stored in a particular
region usually services like this will
let you choose the region that you want
to they'll usually also automatically
replicate files for you so that's
another benefit of using a managed
service because if we stored a file on
our own VM what if the VM dies or you
know some type of Hardware failure well
if you only stored it on one computer
you lose that data that was on that dis
well object stores solve this problem by
replicating the files for you now object
stores are good for storing data but
they're not good at storing everything
what if you had actual application data
not files and images but like you want
to store your database data well
databases are really really complicated
so you usually want to run some type of
specific database software like MySQL or
some other type of relational or
non-relational storage like databases
are basically just a really really
complex wrapper over these resources
particularly dis space well of course if
you want to you can just run your own
MySQL on top of a VM and you can
replicate it among other VMS if you want
to like create replicas of it if you
wanted to do sharding and scale your
database like that you can have some of
your data split among the machines
that's doable but it's complicated isn't
it and so to solve this Cloud providers
will usually give you a managed solution
for storing your data your database data
in the world of AWS it's RDS relational
database service that's if you want a
relational database there's also Dynamo
if you want a non-relational database
once again the whole point of using a
cloud service for this rather than
hosting your own database is because
nobody wants to host their own database
nobody wants to deal with those issues
unless they have to now is a good chance
actually to talk about proprietary
software in many cases AWS isn't doing
anything creative they're actually just
taking open-source software kind of like
we discussed right now my sequel or
postgress is open source to solve this
problem of making the user experience
better is just to take open source
software and then host it and make a
manage service out of it that's a lot of
what relational database service does
but there is something else Dynamo DB if
you want to use this type of database
like this database service you can only
do so on AWS so this is an example of
proprietary software if you built your
app around Dynamo DB and now you decide
that it's costing you too much money you
want to move to a different service well
you're going to have to migrate all of
your data because AWS is the only one
who offers Dynamo DB if you want to go
to Google Cloud you can't and so this is
commonly talked about as vendor lockin
generally you don't want to be locked in
but recognize that it's probably not
going to be a problem unless your app
actually starts scaling because if
you're spending like a 100 bucks a month
on Dynamo DB hopefully your company is
making more than that much money but if
you start spending thousands and
thousands then it becomes an issue and
you'll also find that different
companies have their own versions of
databases for example Google's version
of a Dynamo DB would be either fir store
or Cloud spanner Microsoft has their own
Cosmos DB I think it's called I've never
used it the idea is that once you know
about one service you you know about one
database that AWS offers then going to a
different cloud provider is pretty
trivial because they have their own
version so once you know one you've
learned the others like every cloud
provider has an object store they all
have a VM service they all have their
own proprietary or open source database
Services I talked about how a lot of
services just take open- Source software
and then just make a managed service out
of it well one of the services that
didn't do that was AWS Lambda commonly
known as functions as a service this was
in a sense taking this CPU component the
compute component like if you wanted to
actually create an API or run some type
of compute not necessarily storing
persistent data you would use something
called AWS Lambda and pretty much every
big cloud provider has copied it Google
has Cloud functions I'm sure Azure has
their own but the idea is all you have
to do you don't worry about the
infrastructure at all you don't worry
about the VM the disc you hardly worry
about the CPU and the ram you set like
memory limits and things like that but
basically you take your code as a
developer and that code is pretty much
deployed for you you don't have to worry
too much about the environment you just
take your code and it runs now the
problem with functions as a service this
is one of the most serverless services
that you can do and the reason I say
that is because serverless is really a
spectrum there are services that are
completely managed by the cloud Lambda
would be a pretty good example of that
and then there are other services like
VMS that are not managed at all and then
there are database Services which are
half managed and half not sometimes
Dynamo DB is very managed but the idea
is that there's a spectrum and Lambda or
functions as a service abstract away
pretty much everything for you but that
does come with some downsides for
example if you want to manage any type
of State in your API which is becoming
less and less common with rest apis and
microservices you probably won't need to
do that but if you do you probably won't
be able to use Lambda unless you use
some type of external storage because
when you run a Lambda function you don't
really have access to your disc that's
abstracted away from you it can be good
sometimes and sometimes it can be bad so
it depends on what type of problem
you're trying to solve but with Lambda
most apis can be solved with this way
and if they can why would you ever want
to manage your own VM it's a pain so now
if you just look at these four
categories of services you can build
almost anything with these the one big
category I've probably been lacking on
is networking and I probably won't go
super in depth into that for example
like load balancers CDN and things like
that but logic wise we can do pretty
much everything with these services and
you probably don't even need ec2 you
don't need VMS cuz VMS you can handle
everything with anyway but just looking
at this we have persistent file storage
we have persistent like application data
storage and we have compute we can serve
our front end or our back end where we
go from here is kind of opinionated so
there's two categories I kind of wanted
to talk about one is just observability
in general and I'm not an expert when it
comes to AWS with observability but I
think the services are going to be
pretty similar to Google Cloud but it's
about like your logging your monitoring
data and your alerts being able to
observe how your application is running
via like servers side logs or metrics
for example if you want to see like how
much CPU your VMS are using or how long
your queries are taking that would be
monitoring data and if you wanted to
know if one of these Services went down
for some reason or the latency got too
high you'd probably want to be alerted
of that via some push notification in
real time so that's what observability
is about and the big cloud providers
have done a decent job of giving you
tools for observability but there
actually are other services such as data
dog it's a relatively big company
actually and their whole proposition is
that they can handle observability
better than the big cloud providers even
though there's actually a bit more
friction in using one of these external
tools because there's more work involved
in like setting up the monitoring agents
and things like that but generally
speaking people are big fans of services
like data do because they make this
stuff actually easier for you it's not
just that when you think about something
called Data warehouses the big cloud
providers have I think aw us is is
called red shift Google has big query
and they do a decent job some of them do
better than others but even given that
they're actually are other platforms
like snowflake like data bricks that are
basically companies dedicated to solving
these problems and if they're spending
all their energy on one specific area
then they think and in many cases they
do do better than the big providers
tools like AWS have made development so
much easier than it was years ago where
you needed your own servers still people
complain that AWS is really really
complicated to use and so now there are
companies like versel which pretty much
at a high level are rappers around AWS
and I don't mean that as an insult
because in a sense AWS is just wrappers
around like other open source software
the whole goal of these Cloud providers
is to give you an easier way to use AWS
resources just like how these resources
RDS functions as a service they're just
wrappers around VMS everything is just
an abstraction on top of other things
and that's okay because the whole point
is to make development easier now lastly
I wanted to talk about the differences
between Regional and Global Services cuz
it's kind of a buzzword you see it
thrown around a lot the definition is
kind of fuzzy for example if we talk
about a Content delivery Network a CDN
well that's inherently a global Service
because you have a world m app and a CDN
is basically just a distribution of
servers around the world that store some
static data usually and so you can't
really have a regional CDN now take the
other side take a VM for example VMS are
inherently Regional you can't have one
VM split in five different regions right
you just have one VM in a location you
could have a second VM somewhere else
and a third one somewhere else and you
know you could have these VMS under a
load balancer serving traffic in
different parts of the world that's
perfectly fine the service itself is
regional though a VM is regional
sometimes it gets kind of fuzzy if we're
talking about databases like Dynamo DB
or Google Cloud spanner and before I
even get into those let's just talk
about a regular relational database
service like RDS and I think Google's is
cloud SQL I really prefer the names in
Google Cloud they're just a bit more
intuitive but with a regional database
service you have your world map and you
have your database service hosted in a
certain region even though it can be
partially managed for example like
replicas database backups and a lot of
things can be managed for you by the
cloud provider it's still in a single
region and it'll be serving traffic
probably around the world so some places
in the world are going to receive data a
little bit slower usually it doesn't
matter though but when you get into
Global databases these are super super
super powerful because they actually
manage sharding and replication for you
and they distribute your data all around
the world even though it's distributed
around the world they will work together
generally your request will be routed to
the nearest location the power of a
global database is huge even though most
people probably won't need it it is very
powerful in just handling the scale for
you and making the latency as small
small as possible because data is stored
as close to the user as possible kind of
like with a Content delivery Network and
generally you'll find that Global
Services usually cost more and managed
cloud services also cost more because
the more work that the cloud provider
does for you the more they're going to
charge you of course sometimes it's
worth it though like this is a very hard
problem to solve you probably don't want
to implement your own Global database by
yourself you probably just want to pay
somebody to do it for you this is a lot
of open-ended discussion but I think
that's what's most important I could
have gone into an example of one of
these Cloud providers but looking at it
from a high level I think is most
important and getting hands on with it
which is something we didn't do is
something I'd also recommend to you like
don't just watch me talk about this
stuff try using some of these Services
try building an application out of them
Voir Plus de Vidéos Connexes
Intro to AWS - The Most Important Services To Learn
Google Cloud Platform Tutorial - Part #1 | Introduction to GCP | Cloud Computing Basics | @SCALER
24 MOST Popular AWS Services - Explained in 13 mins (2024)
AWS In 10 Minutes | AWS Tutorial For Beginners | AWS Cloud Computing For Beginners | Simplilearn
Top 50+ AWS Services Explained in 10 Minutes
What is the cloud? | CNBC Explains
5.0 / 5 (0 votes)