How To Make AWS Not Suck

Summary

This video offers a guide for software developers to optimize their use of cloud platforms like AWS, Google Cloud, and Azure. It explores the spectrum of dependency on cloud services, from self-hosting purists to fully outsourced SaaS users. The video introduces tools like Pulumi, SST, Docker Build Cloud, LocalStack, and Encore to help developers leverage cloud services efficiently, avoid vendor lock-in, and maintain control over their infrastructure. It emphasizes the importance of infrastructure as code and provides practical solutions for managing cloud resources effectively.

Takeaways

• 🌟 The video discusses how to optimize cloud computing usage for software developers, avoiding dependency extremes on cloud platforms.
• 💼 It introduces a spectrum with 'self-hosters' at one end,反感专有闭源软件，and 'SaaS sommelier' at the other, unaware of the underlying AWS services they use.
• 🛠️ Emphasizes the importance of Infrastructure as Code (IaC) for efficient and error-free cloud deployment, contrasting it with the risks of manual deployment.
• 🌐 Recommends using open-source tools like Pulumi, which allows writing infrastructure code in multiple languages and provides a strong type system and IDE support.
• 📝 Introduces SST (Serverless Stack) as an optimized IaC tool for fullstack JavaScript developers, allowing infrastructure and databases to be defined in a single TypeScript file.
• 🚀 Highlights Docker and Docker Build Cloud for efficient containerization and image building, with cache sharing across different machines.
• 🌐 Suggests LocalStack for local emulation of AWS services, enabling offline development and testing.
• 📈 Mentions Cloudcraft for visual architecture planning and LocalStack dashboard for local management of cloud services.
• 🎉 Presents Encore as a tool that combines infrastructure provisioning with application development, streamlining the deployment process.
• ⚖️ Discusses the trade-offs of using Encore, such as reduced flexibility due to its opinionated nature and assumptions about technology stack and frameworks.

Q & A

What is the main topic of the video?

The main topic of the video is how to make cloud computing more efficient and cost-effective by leveraging big cloud platforms optimally.

What are the two extremes of the cloud dependency spectrum mentioned in the video?

The two extremes are the 'base self-hoster' who does everything manually on a low-cost VPS and the 'SaaS Some Leer' who relies heavily on cloud services and spends a lot without realizing it.

Why is it not ideal to be at either extreme of the cloud dependency spectrum?

Being too self-reliant is time-consuming and error-prone, leading to a superiority complex, while being too reliant on cloud services can lead to vendor lock-in, financial strain, and reduced technical control.

What is the role of infrastructure as code (IaC) in cloud computing according to the video?

Infrastructure as code is crucial for deploying cloud resources efficiently and safely without manual errors. It helps manage infrastructure in a repeatable and automated way.

What is the issue with using cloud platforms directly without IaC tools?

Directly using cloud platforms without IaC tools can lead to over-provisioning resources like Kubernetes clusters, which may be unnecessary and can result in homelessness due to overspending.

What is Palumi and how does it differ from Terraform?

Palumi is an IaC tool that allows writing code in multiple languages and takes a declarative approach. Unlike Terraform, it is open-source and does not require learning a special language.

What is SST and how does it optimize IaC for fullstack JavaScript developers?

SST (Serverless Shen pans and Tom Foolery) is an IaC tool that optimizes the process for fullstack JavaScript developers by allowing infrastructure and databases to be defined in a single TypeScript file.

How does Docker Build Cloud improve the Docker image building process?

Docker Build Cloud provides a shared cache that speeds up the Docker image building process both locally and in CI by only rebuilding layers that have changed.

What is LocalStack and how does it help with cloud service emulation?

LocalStack emulates major AWS services from a single container, allowing developers to test and work with services like S3, Lambda, and DynamoDB locally without an internet connection or AWS account.

What is the purpose of the tool Cloudcraft mentioned in the video?

Cloudcraft is used to create visual diagrams of cloud architecture, showing how different services connect and interact with each other in a 3D representation.

What does the tool Encore offer that is different from other IaC tools?

Encore not only provisions infrastructure but also serves as a backend application framework that integrates infrastructure semantics into the application code, automating devops tasks and reducing the need for separate IaC code.

What are the trade-offs mentioned for using the Encore tool?

The trade-offs for using Encore include reduced flexibility, as it assumes the use of TypeScript or Go, adherence to certain framework conventions, and the use of SQL databases like RDS.

Outlines

00:00

🌟 Optimizing Cloud Computing Efficiency

This paragraph introduces the concept of finding a balance in cloud computing dependency, highlighting the extremes of self-hosting and over-reliance on cloud services. It emphasizes the importance of leveraging cloud platforms efficiently without vendor lock-in. The speaker introduces infrastructure as code (IaC) as a key strategy, mentioning tools like Terraform and Pulumi, which allow for code-based infrastructure management across multiple languages. Pulumi is highlighted for its ability to define, deploy, and manage cloud resources with strong typing and Intellisense support in the IDE. The paragraph also touches on the benefits of using open-source tools to avoid being 'enslaved' by cloud service providers.

05:01

🛠️ Leveraging Tools for Cloud Development

The second paragraph delves into specific tools that can enhance cloud development workflows. It starts with SST (Serverless Shenpans and Tom Foolery), a framework for fullstack JavaScript developers that allows defining infrastructure and databases in a single TypeScript file. The paragraph also discusses the advantages of containerization with Docker and the efficiency gains from Docker Build Cloud, which enables shared caching across builds. LocalStack is introduced as a tool for local emulation of AWS services, facilitating offline development and testing. The paragraph concludes with a mention of Encore, a tool that integrates infrastructure provisioning with application code, aiming to streamline the development process by automating DevOps tasks. The trade-offs of using Encore, such as reduced flexibility due to its opinionated approach, are also briefly discussed.

Keywords

💡Cloud Computing

Cloud computing refers to the delivery of computing services, including servers, storage, databases, networking, software, analytics, and intelligence, over the Internet (the cloud) to offer faster innovation, flexible resources, and economies of scale. In the video, the speaker discusses how to optimize the use of cloud computing platforms like AWS, Google Cloud, and Azure to achieve greater efficiency and avoid vendor lock-in.

💡Vendor Lock-in

Vendor lock-in occurs when a customer is dependent on a single vendor for a product or service, making it difficult to switch to another vendor without significant costs or inconvenience. The video discusses the risks of vendor lock-in with cloud services and how to use open-source tools and infrastructure as code to mitigate this risk.

💡Infrastructure as Code (IaC)

Infrastructure as Code is a method of managing and provisioning computer data centers through machine-readable definition files, rather than physical hardware configuration or interactive configuration tools. The video emphasizes the importance of IaC for deploying cloud resources efficiently and consistently, mentioning tools like Terraform and Pulumi.

💡Pulumi

Pulumi is an infrastructure as code tool that allows developers to use familiar programming languages to define cloud infrastructure. The video highlights Pulumi as an alternative to Terraform, noting its support for multiple languages and its ability to provide a better developer experience with features like type safety and IntelliSense.

💡Serverless

Serverless computing is a cloud computing execution model where the cloud provider dynamically manages the allocation and provisioning of servers. The video mentions serverless frameworks and how they can simplify the deployment of applications without managing servers, using AWS Lambda as an example.

💡Docker

Docker is an open-source containerization platform that allows developers to package an application with all of its dependencies into a standardized unit called a container that can run consistently on any platform. The video discusses Docker Build Cloud, a service that enhances Docker's capabilities by providing a shared cache for building Docker images more efficiently.

💡LocalStack

LocalStack is an open-source cloud stack simulation framework that emulates AWS services on your local machine. The video explains how LocalStack can be used to test cloud applications locally, which is beneficial for developers who want to iterate quickly without relying on an internet connection or cloud credits.

💡Containerization

Containerization is the process of packaging software and all its dependencies into a container that can run on any system. The video mentions containerization as a powerful technique to free applications from vendor lock-in, with Docker being a widely used tool for this purpose.

💡Encore

Encore, as mentioned in the video, is a tool that combines infrastructure as code with backend application frameworks, allowing developers to write application code in Go or TypeScript while integrating infrastructure semantics. It aims to streamline the process of provisioning infrastructure and managing application deployment.

💡TypeScript

TypeScript is a statically typed superset of JavaScript that adds optional types to the language. The video discusses how tools like Pulumi and SST (Serverless Components and Toolkit) allow developers to use TypeScript for defining cloud infrastructure, which can lead to better developer experience and code reliability.

Highlights

The video teaches how to optimize cloud computing usage for software developers.

Developers are categorized based on their dependency on cloud platforms like AWS, Google Cloud, and Azure.

At one end of the spectrum are self-hosters who handle high traffic with minimal resources.

At the other end are developers heavily reliant on SaaS tools, often unknowingly using AWS services.

Being too self-reliant or too dependent on cloud services has its drawbacks.

A balanced approach is suggested, leveraging cloud services efficiently without vendor lock-in.

Open source tools are recommended for optimal cloud usage.

Infrastructure as Code (IaC) is essential for deploying on cloud platforms.

Terraform is a popular IaC tool, but it requires learning a special language.

Pulumi is an alternative to Terraform, supporting multiple programming languages.

Pulumi allows for declarative infrastructure management with strong typing and IDE support.

Serverless Stack (SST) is an optimized IaC tool for fullstack JavaScript developers.

SST allows defining infrastructure and databases in a single TypeScript file.

Docker and Docker Build Cloud are highlighted for efficient containerization and image building.

LocalStack emulates AWS services locally for offline development and testing.

Cloudcraft is a tool for visually diagramming cloud architecture.

Encore is introduced as a tool that combines IaC with application code for streamlined deployment.

Encore automates devops tasks and integrates infrastructure with application code.

The video concludes with advice on using the right tools to make cloud computing work efficiently.

Transcripts

in this video you're going to learn how

to make cloud computing great again if

you're a software developer that means

you're on the Spectrum a spectrum that

defines how dependent you are on big

cloud platforms like AWS Google cloud

and Azure on one end we have the based

self Hoster this Dev does a million plus

an annual recurring revenue and handles

5,000 requests per second with PHP and

jQuery on a $5 FreeBSD VPS the thought

of proprietary close Source software

triggers their gag reflex and they pity

the fools paying for Jeff beus yacht

when Lambda HTTP request at a time but

at the other end of the spectrum we have

the SAS Som leer this Dev bought a $200

template and now pays $20 a month for

website hosting at $50 a month for

postgress at $10 a month for a

notetaking app and $20,000 for data dog

all of which is funded by rewards credit

cards instead of customer revenue and

they had no idea all the things they

were paying for are just rappers of AWS

but being on either end of the spectrum

is not ideal being two based is timec

consuming and error prone and leads to

superiority complex but not being based

enough allows the AWS rapper industry to

enslave you which will make you poor and

reduce your te count luckily there is

another path a path where you leverage

big cloud at optimal efficiency with

amazing DX and zero vendor lockin and

there are some amazing open source tools

that can help us get there but first why

do I even need a tool well if you go on

big Cloud's website you'll look around

for the right tool for the job end up

deploying a massive kubernetes cluster

you don't need then forget about it and

end up homeless if you're deploying on

big cloud you should never do this and

instead use infrastructure as code there

are many tools that can get the job done

with one of the most popular options

being terraform but it's not open-

source and requires its own special

language a good alternative is palumi

which itself is written in go but allows

you to write your code in multiple

languages like typescript python go the

c or just damel taking a declarative

approach like this is not only safer but

just far more efficient like in this

example here I'm using palumi to create

a Google Cloud virtual machine I can

Define all the machine settings here

like the size and operating system as

well as all the network and firewall

settings and anything else you can

imagine all of which is strongly typed

and provides intellisense right here in

the IDE now if I want to deploy it I can

just do pumi up and it will

automatically deploy this stuff to my

Google Cloud account and likewise when I

want to take it down I can do that with

a single command in addition you can use

their dashboard to keep track of all

your projects and there's a bunch of

templates for the most common use cases

it's incredibly powerful but if you like

palumi and also like typescript a tool

you're really going to like is SST which

stands for serverless Shen pans and Tom

Foolery when I found out it was a rapper

of palumi I thought it was just a scam

but actually it's the most optimized IAC

tool if you're a fullstack JavaScript

developer you define all of your

infrastructure and databases in a single

typescript file and what's cool about it

is that they've designed specific

components or recipes for all the

popular Frameworks like nextjs spel kit

angular and so on like if I'm deploying

a spel kit app I can define a server and

then link other resources to it like an

S3 storage bucket and now that all

that's in place I can easily reference

it in my application code because it's

also written in typescript we can then

deploy the app with a single command or

set up the SST console to deploy on get

push and it even works with Frameworks

like nextjs which is notorious for

trapping developers in a triangle that

they can never leave it feels good to

get pure uncut product by going directly

to the supplier but one of the most

powerful techniques to free your app

from vendor lockin is containerization

with Docker the sponsor of today's video

we all use Docker but what you likely

don't know about is Docker build Cloud a

way to build Docker images way faster

both locally and in continuous

integration Docker is smart enough to

know that when you run the build command

it should only rebuild layers that have

changed and use the cach for everything

else the problem is that that cash only

exists on your machine with build Cloud

though this cash becomes available to

every machine remotely that means if Bob

builds an image in Tokyo Alice can take

advantage of that cash in Austin

instantly but most importantly it fixes

the massively annoying bottleneck of

waiting for your CI server to hurry up

and fail the shared cach can make

complex builds 39 times faster which

means more developer happiness for

individuals and potentially millions of

dollars saved for a medium-sized company

try out Docker build Cloud for free

right now with the link below one thing

that sucks about the cloud though is

that it's utterly useless without an

internet connection or credit card the

good news is that there's a tool called

local stack which emulates all the major

services on AWS from a single container

that you can run locally that means you

can cat up and test all of your buckets

lambdas and Dynamo DBS on a flight and

then push them to production the moment

you land that's impressive because AWS

is a massive collection of stuff you

don't need and while local stack doesn't

cover everything it does cover a lot of

things most importantly though it's a

playground where you can over engineer

your architecture without any real world

consequences what I like to do is first

create a diagram of my architecture

using a tool like cloudcraft which can

visually represent different services in

3D and how they connect to one another

like I might have an S3 static website

which has a form that triggers a Lambda

function that uploads to another bucket

which triggers a resizer Lambda which

triggers the simple notification service

to send an email with simple email

service but to test out my prototype I

can now use the AWS local CLI to create

all my lambas and buckets locally but

once deployed locally I can then go to

the local stack dashboard and manage

everything just like it was AWS I can

even see my uploaded S3 files right here

pretty awesome but I've saved the best

for last The Encore of this video is a

tool named Encore every developer should

know how to use infrastructure as code

tools but one drawback is that they're

totally decoupled from the code that

matters your application code in a

perfect world we just build an app click

deploy and never worry about it and

Encore gets us pretty close to that

world it's a tool that can help you

provision infrastructure on AWS or

Google Cloud but it's also a backend

application framework that allows you to

write your application code in go or

typescript however the infrastructure

semantics are built into the application

code it can do this because at compile

time it creates an application model

where all the resources required to run

your app are identified where you can

then configure and customize them from

Encore development dashboard board you

get the benefits of IEC but without

having to actually write the code and it

automates all the other devops work like

continuous integration and deployment

observability tracing IM management and

more it sounds too good to be true so

what are the trade-offs the big

trade-off is flexibility it assumes your

application code is written in

typescript or go that you're using a SQL

database like RDS and that you adhere to

OnCourse framework conventions in some

cases it might be better to keep your

infrastructure code separate it's

opinionated but if you agree to those

terms you could save yourself thousands

of hours of development time and that

concludes my Ted Talk on how to make AWS

not suck thanks for watching and I will

see you in the next one