Implementing Infrastructure as Code with Terraform | AWS Cloud Resume Challenge - Part 6

Summary

In part six of the AWS Cloud resume challenge, host Richard dives into the concept of Infrastructure as Code (IaC) with Terraform, chosen by the audience. He guides through setting up a provider configuration, creating a Lambda function, and deploying it with Terraform. Richard also covers deploying a Lambda function URL and addressing IAM permissions for accessing DynamoDB. The video concludes with a functional Lambda setup, a walkthrough of Terraform files, and suggestions for further automation with GitHub actions, inviting viewers to explore creating IaC for DynamoDB, S3, and CloudFront.

Takeaways

• 😀 The video is part of a series on the AWS Cloud resume challenge, hosted by Richard.
• 🛠️ The series has covered various topics including CI/CD setup for an S3 bucket, creation of an S3 bucket with CloudFront, a DynamoDB table, and a Lambda function in Python.
• 📈 The main focus of this part is on IaC (Infrastructure as Code), with an emphasis on using Terraform over CloudFormation based on audience poll results.
• 📝 The tutorial guides through setting up the `provider.tf` file for Terraform, detailing the use of AWS CLI profiles or direct access key and secret key setup.
• 🔧 It demonstrates creating a Lambda function and an IAM rule using Terraform, including the use of a zip file for the Lambda function code deployment.
• 📁 The script mentions creating a `func.py` file containing the Python code for the Lambda function, which is then referenced in the Terraform files.
• 🚀 The process of deploying infrastructure with Terraform is explained, including the use of `terraform init`, `terraform plan`, and `terraform apply` commands.
• 🔗 The importance of the state file in Terraform for tracking infrastructure changes is highlighted.
• 🔒 A section is dedicated to adding a Lambda function URL with proper authorization settings to allow public access or restrict it to specific domains.
• 🛡️ The video addresses a common error related to IAM permissions, showing how to attach a policy to an IAM role to grant the Lambda function access to DynamoDB.
• 🔄 The script suggests automating further deployments with GitHub Actions and setting up a CI/CD pipeline for backend changes and testing.

Q & A

What is the main topic covered in part six of the AWS Cloud resume challenge?

The main topic covered is Infrastructure as Code (IC), specifically using Terraform for deploying AWS resources.

What AWS services were set up in the previous parts of the challenge?

In the previous parts, they set up CI/CD for an S3 bucket, created an S3 bucket with CloudFront to host a website, and created a DynamoDB table and a Lambda function in Python.

Why did the audience choose Terraform over CloudFormation?

The audience preferred Terraform based on a poll conducted by the host, although the exact reasons for their preference are not specified in the script.

What is the purpose of the Lambda function created in the script?

The Lambda function is used to get a viewer count from a DynamoDB table and display it on the website.

How is the Lambda function code transferred to the AWS environment using Terraform?

The Lambda function code is transferred using a zip file that contains the Python code written for the Lambda function.

What is the significance of the 'terraform plan' command?

The 'terraform plan' command shows what changes are planned to be made to the AWS infrastructure, such as adding or modifying resources.

Why was an 'Internal Server Error' encountered when trying to access the Lambda function URL?

The error occurred because the Lambda function's IAM role did not have the necessary permissions to access DynamoDB, which it requires to get and update the viewer count.

What is the role of the 'terraform apply' command in the deployment process?

The 'terraform apply' command is used to create or update resources in the AWS infrastructure according to the Terraform configuration files.

What is the purpose of the 'terraform state' file?

The 'terraform state' file keeps track of the current state of the infrastructure, allowing Terraform to understand what has changed and what needs to be updated or created.

How can the deployment process be automated using GitHub Actions?

GitHub Actions can be set up to automatically deploy changes made to the Terraform files, ensuring that updates to the infrastructure are applied quickly and consistently.

What additional resources could be managed by Terraform as suggested in the script?

The script suggests creating Terraform files for managing a DynamoDB table, S3 bucket, and CloudFront distribution, in addition to the Lambda function.

Outlines

00:00

🚀 Introduction to Terraform in AWS Cloud Resume Challenge

In the sixth part of the AWS Cloud Resume Challenge, host Richard reviews the series' progress, which includes setting up CI/CD for an S3 bucket, creating a website-hosting S3 bucket with CloudFront, and establishing a DynamoDB table and a Python Lambda function to display viewer counts. Richard introduces the main topic of this segment, Infrastructure as Code (IaC), and decides to use Terraform based on audience feedback. He explains the process of setting up the 'provider.tf' file for Terraform, detailing the use of AWS CLI profiles or manual access key and secret key entry. The main infrastructure code, including the Lambda function and its related resources, is to be written in the 'main.tf' file.

05:00

🛠 Deploying Lambda Function with Terraform

Richard demonstrates deploying a Lambda function using Terraform by creating a 'my_func' Lambda function and an IAM rule. He explains the process of uploading the Lambda function code through a zip file and setting up the necessary directory and 'func.py' file with the Python code for the Lambda function. Richard then shows the deployment process using 'terraform init', 'terraform plan', and 'terraform apply', highlighting the importance of the state file in tracking infrastructure changes. After deploying the Lambda function, he discusses the need to configure a function URL for triggering the Lambda function and updates the Terraform files accordingly.

10:03

🔒 Fixing Permissions and Finalizing Lambda Function Deployment

After deploying the Lambda function, Richard discovers an 'Internal Server Error' due to missing permissions for the Lambda function to access DynamoDB. He rectifies this by creating a new IAM policy that grants the necessary 'update-item' and 'get-item' actions for the DynamoDB table. Richard then attaches this policy to the IAM role associated with the Lambda function. After making these changes and running 'terraform plan' and 'terraform apply' again, the Lambda function is successfully updated with the correct permissions, and the function URL starts returning the correct viewer count from the DynamoDB table.

🔄 Automating Deployments with GitHub Actions and Future Plans

Richard concludes the video by discussing the automation of deployments using GitHub Actions for any changes made to the Terraform files. He suggests setting up a '.gitignore' file to exclude sensitive credentials from commits. Richard also invites viewers to create additional Terraform files for other resources like DynamoDB tables and S3 buckets, and to consider using Pulumi or CloudFormation for IaC. He hints at future content involving CI/CD setup for the backend, including GitHub Actions for deployment and testing of the Python code. Richard encourages viewers to like, subscribe, and comment on their experience with the series.

Keywords

💡AWS Cloud

AWS Cloud, short for Amazon Web Services Cloud, is a comprehensive and widely adopted cloud platform that offers a variety of services, including computing, storage, databases, and more. In the video, AWS Cloud is the main platform where the host, Richard, is demonstrating the setup and deployment of various cloud resources as part of the 'resume challenge' series.

💡CI/CD

CI/CD stands for Continuous Integration/Continuous Deployment, a set of practices in software development designed to improve the process of integrating changes from multiple contributors into a single software project. In the script, CI/CD is mentioned in the context of setting up for an S3 bucket, which is part of the front-end infrastructure being automated.

💡S3

S3, or Amazon Simple Storage Service, is an object storage service offered by AWS that provides developers and IT teams with secure, scalable, and highly available storage. The script discusses the creation of an S3 bucket to host a website, which is a common use case for S3 in web development.

💡CloudFront

CloudFront is a content delivery network (CDN) service provided by AWS. It accelerates the distribution of static and dynamic web content, such as .html, .css, .js, and image files, to users around the world. In the video, CloudFront is used in conjunction with the S3 bucket to improve the website's performance.

💡DynamoDB

DynamoDB is a fully managed NoSQL database service provided by AWS that offers fast and predictable performance with seamless scalability. In the script, a DynamoDB table is created to store and retrieve data, such as a visitor count for the website, which is a typical application of DynamoDB in web applications.

💡Lambda

AWS Lambda is a serverless compute service that lets you run code without provisioning or managing servers. In the video, a Lambda function written in Python is created to get a visitor count from the DynamoDB table and display it on the website, showcasing the serverless architecture capability of AWS.

💡Infrastructure as Code (IaC)

Infrastructure as Code (IaC) is a DevOps practice where infrastructure is described and managed using code, allowing for version control and automation. The script introduces IaC as a main concept of DevOps and discusses using Terraform, a popular IaC tool, for managing AWS resources.

💡Terraform

Terraform is an IaC tool developed by HashiCorp that enables users to build, change, and version infrastructure safely and efficiently. In the video, Terraform is chosen over CloudFormation based on a poll and is used to script the deployment of AWS resources such as the Lambda function.

💡IAM

IAM stands for AWS Identity and Access Management, a service that helps you securely control access to AWS resources. The script mentions setting up an IAM user with the necessary permissions to deploy resources, which is a critical step in managing access to AWS services.

💡Lambda Function URL

A Lambda Function URL is a feature that allows direct invocation of a Lambda function via an HTTP endpoint. In the script, the host configures a Lambda Function URL to trigger the Lambda function that retrieves the viewer count, demonstrating a practical use of this feature for serverless applications.

💡GitHub Action

GitHub Action is a feature of GitHub that allows you to automate workflows, such as software development processes including build, test, and deployment. The script suggests setting up a GitHub Action for further deployments of the Terraform configurations, which would automate the process of updating the infrastructure when changes are made.

Highlights

Introduction to part six of the AWS Cloud resume challenge series by host Richard.

Overview of previous accomplishments in setting up CI/CD for S3, creating an S3 bucket with CloudFront, and developing a DynamoDB table and Lambda function.

Introduction of the main concept of DevOps: Infrastructure as Code (IC).

Decision to use Terraform over CloudFormation based on audience poll results.

Explanation of setting up the AWS provider with either a profile or access keys in the provider.tf file.

Diving into writing Terraform code, starting with the Lambda function configuration.

Mention of AWS Lambda function URLs as a new resource in the AWS Terraform provider.

Instructions on creating a Lambda function and IAM rule using a zip file for code deployment.

Details on structuring the Lambda directory and func.py file for the Python code.

Demonstration of deploying infrastructure with Terraform and the importance of the state file.

Use of Terraform commands 'plan' and 'apply' to deploy AWS resources.

Verification of Lambda function deployment and the addition of a function URL for triggering.

Encountering an internal server error due to missing permissions for the Lambda function to access DynamoDB.

Correction of the IAM policy to include necessary permissions for DynamoDB access.

Attachment of the corrected IAM policy to the IAM role for the Lambda function.

Final deployment using Terraform to fix the internal server error and successfully retrieve viewer count.

Summary of building Terraform files for deploying a Lambda function directly from a code editor.

Suggestion to set up GitHub actions for automated deployments and include testing for the Python code.

Invitation for viewers to provide feedback and suggestions for future parts of the series.

Transcripts

welcome to part six of the AWS Cloud

resume challenge this is your host

Richard and you know I've been doing

this challenge for a while now but we

have covered a lot of topics in the

previous five parts so we have set up

cicd for our S3 which is the front end

we created an S3 bucket with cloudfront

which hosts our website we created a

dynamodb table and also a Lambda

function in Python that gets a severe

count from our dynamodb table and

displays it on our website so I hope

you're enjoying this series today I'm

gonna cover some really interesting

topic which is one of the main concepts

of devops known as IC which stands for

infrastructure as code and as I did a

poll few weeks ago on what should I

choose as the tool you know terraform or

cloud formation a lot of you said

terraform so we'll be trying out some

terraform today

coming up to the cloud resume challenge

website so this is the part that we'll

be covering which means we are almost

done with this challenge you should have

a working project by now with that being

said let's Dive Right into writing some

terraform

so in your provider.tf

you can either have the profile so if

you have set up AWS CLI with profiles

you can list that profile here if you

don't know I'll leave it linked down

below and you can read more about how to

set up adobe CLI profile if not you will

have to set up the access key and the

secret key for your IM user that has

access to deploy the resources that

we'll be deploying today so it's either

this or your profile

so now the main.tf comes into play so

this is where all of the main

infrastructure code will go so in our

case I'm going to start with the Lambda

function and I also would you like to

remind that we used Lambda function URLs

which is now an available resource in

the AWS terraform provider but before we

can get started with the URL we first

need a Lambda function so let's create a

Lambda function called my function or

I'll make it short for my func

and now we'll also create a im rule for

our Lambda function

and the way we upload our Lambda

function through terraform is by using a

zip file so I'll add the code to do that

here now

so now what we have to do as you can see

as per the instructions that I mentioned

here I need a Lambda directory and

inside that Lambda directory we'll be

using a file called func Dot py

and this func.py will basically contain

our python code that we wrote in the

console for a Lambda function so just to

remind you this is how our Lambda

function looked like in the AWS console

what I'm going to do is just copy this

python code and paste it in our func.py

file here and don't get confused by the

names you can name whatever you want

your Lambda function to be which in my

case is my func or even the func.py file

just make sure to reference them in our

terraform files accordingly so I'm just

gonna do Ctrl s and coming back to the

main.tf everything looks good so now let

me show you how it would look like if it

did deploy this infrastructure using

terraform so what I'm going to do is

going to provider.tf since I don't want

to set up access key or secret key I do

have a profile set up on my local

machine here which is called default I

believe so if I click controller s and

open up my terminal here we have to go

into the infra directory right and now

here we will do a terraform in it which

will initialize the terraform directory

that we have so it launches so it

creates a bunch of files like terraform

lock and also the state files which

remembers what the state of our

infrastructure is and you will

understand more as I go through this

demo here so let's do terraform plan

which will show us what do we plan on

deploying to our AWS infrastructure and

as you can see plan says two to add one

of them being the Lambda function and

then the I am rule that we have so let's

do

terraform apply enter yes

it'll start creating the Lambda function

for us so we'll give it a minute to do

that

okay so the apply has been completed and

you can see two resources were added as

per our terminal the region was a

central so let's go look at our AWS

console to see if the Lambda function

was deployed so you can see my func was

deployed 34 seconds ago and if you click

on that you will see that the code has

been pushed right as we wanted it to be

the only thing is it won't have a

function URL because we didn't configure

it in our terraform file so let's go

back and do that because we needed the

URL to trigger our Lambda function right

because that's how we get the viewer

count from dynamodb okay so coming back

to our main.tf we need to add a chunk of

code here to deploy function URL and the

way you do that is by using a resource

called AWS underscore Lambda underscore

URL

so this is the resource remember I have

set the authorization Type To None So

that the URL is publicly accessible but

you can set up calls so that the

function can only be called from your

own resume website so in allow regions

make sure you have your own website's

domain so I'll do Ctrl s which would

save domain.tf and open the terminal

again here and let's do terraform plan

again this is where these State Files

come into play so as you can see it

plans on adding one other resource now

which is our Lambda function URL so this

is how it identifies that oh if I'm

changing the infrastructure because

remember our main.tf has all three

resources right the Lambda function the

IM Rule and the Lambda function URL but

instead it only said want to add because

it checked in the state file that the

other two resources are already deployed

so I hope you understand the concept of

the state file here

so we'll do terraform apply and like

last time we'll enter yes

and there we go the apply was complete

let's go back to our Lambda function

here and let's do a quick refresh we'll

see that it was the last modified 15

seconds ago and if you look at

configuration you can see we have a

function URL

so if we click on that I want you to

guess in the comments what would happen

I think that we should get a viewer

count but we forgot one crucial step

internal server error so the reason we

get an internal server error is because

the Lambda function that we set up it

has an IEM rule remember in our main.tf

and in that policy statement we didn't

include dynamodbe and our Lambda

function does need access to dynamodb in

order to get and update the account so

let's change this policy okay so let's

create another IM role policy that we

will attach to our IAM rule so coming

down here and I'm just also going to

link the AWS documentation that you will

find handy when you're creating this

policy so I'm just going to quickly type

it down

okay so I have my I am policy here as

you can see AWS I'm policy and I've just

named it I am policy for a resume

project and you can change this right

and if you look at the policy it is

allowing the action here called update

item and get item for dynamodb

and let's make sure the intent is nice

the resource make sure you update this

to your table name so in my case the

table name is resume Dash challenge as

you can see in the python function here

so whatever your dynamodb table name is

make sure it's you know pointing to that

if I click Ctrl s that will save our

main.tf file

and now the last thing to do is attach

this policy to our IM rule which we can

do here at the bottom so if I do

resource AWS IM role policy attachment

so you can find more information on this

resource on the AWS provider for

terraform and I am attaching the rule

that we created called IAM for Lambda to

the policy that we created called IM

policy for a resume project so let's do

that

and then the policy is

there we go so we have our that resource

ready so if we do Ctrl s and open my

terminal here

let's do terraform plan

so we have some syntax error on line 61

which is right here so I forgot

to put dot name at the end so we'll do a

control s again and do terraform plan

again to see if everything looks good so

yes there are two changes or two things

that need to be added so we'll do

terraform apply and I'll ask for a yes

or no we'll go with Yes again and there

we go two resources were added so now if

I go to my functions here click on my

func and we know that it already had the

role name as I am for Lambda as we

described in our main.tf so if we look

at the permission you can see that there

is a policy attached now right and if I

open that up and you can see it has

dynamodb read and write the table name

is resume Dash challenge

so now if we click on this URL and I

hope we get the viewer account and you

can see we are getting 29388 which is my

Tano DB tables value right now so we

finally have our function working again

in this video we build terraform files

to deploy Lambda function so this is the

python function that got deployed right

from our code editor and we can set up

GitHub action for you know further

deployments of this terraform so let's

say we change you know something in our

policy or if we go to Lambda and we

changed our function dot pi to do

something different we can have a GitHub

action workflow that will deploy the

changes automatically

so just to wrap it up we have main.tf we

have provider.tf and we have function.pi

which is our Lambda function in Python

and all of this is available under the

infra directory on my GitHub repo so you

can check it out I would also recommend

adding a git dot ignore to your info

folder to make sure you don't commit you

know your credentials and stuff so in

order to take this challenge folder what

you can do is create write terraform

code for creating dynamodb table

terraform code for your S3 bucket so

like all of the infrastructure bits that

we created you know dynamodb Lambda and

S3 are created by terraform and you can

use palumi or cloud formation to do your

ioc so for the next part comment down

below but what I was thinking is have

the CI CD pod set up for the backend

which means the GitHub action that I was

talking about which would deploy any of

the changes we do to our main main.tf or

any of the terraform files and also

include some testing so I remember we

left out testing so maybe we set up some

tests for our python code so you know

the Lambda function and we do some mock

tests for how the dynamodb gets called

and like some mock responses that we can

do so let me know if you're interested

in that but I feel pretty good about

this Challenge and as I said you can

take it a step further by creating

terraform files for your dynamodb and S3

and cloudfront I hope you're liking this

series if you are please make sure you

like and are subscribed to the channel

to stay up to date I have some more

projects coming down the line and make

sure you comment down if you found any

issues because there are always some

weird I am issues that I leave out and

then I try to cover them in the next

video so yeah I'll see you in the next

one peace