Day-16 | Infrastructure as Code | #terraform #IaC
Summary
TLDRIn this 16th installment of the complete DevOps course, Abhishek introduces Infrastructure as Code (IaC), focusing on the challenges of managing cloud resources across different providers. He discusses the limitations of platform-specific IaC tools like AWS CloudFormation and Azure Resource Manager, and presents Terraform by HashiCorp as a universal solution that simplifies the process through API as code. The video promises a live demonstration of Terraform in the next session, aiming to clarify its advantages in a multi-cloud environment.
Takeaways
- π Abhishek introduces a dedicated video on Dynamic Inventory to address viewer doubts.
- π The video focuses on 'Infrastructure as Code' (IaC), explaining its importance and use cases.
- π€ It discusses the problem of managing infrastructure across different cloud providers and the limitations of platform-specific tools like AWS CloudFormation Templates (CFT) and Azure Resource Manager.
- π The script highlights the need for a DevOps engineer to automate infrastructure creation, initially using AWS CLI or CFT, and then adapting to changes in cloud providers.
- π The concept of 'API as Code' is introduced, where Terraform is presented as a solution to the problem of learning multiple cloud provider-specific tools.
- π Terraform's ability to interact with various cloud providers' APIs through a single scripting language is emphasized.
- π§ The script explains how Terraform simplifies the process of migrating infrastructure from one cloud provider to another by making minimal changes to the scripts.
- π It also touches on the hybrid cloud model, where organizations use multiple cloud services for different needs.
- π The importance of understanding APIs for automation is underscored, with a simple explanation of what an API is and how it works.
- π The video promises a follow-up session with live examples of Terraform, including installation and creating resources on AWS.
- π The key takeaways from the script are the concepts of IaC and API as Code, and how Terraform leverages these to provide a unified approach to cloud infrastructure automation.
Q & A
What is the main topic discussed in the video script?
-The main topic discussed in the video script is Infrastructure as Code (IaC) and the introduction of Terraform as a solution to manage IaC across different cloud providers.
What is the problem with managing infrastructure manually across different cloud providers?
-Managing infrastructure manually across different cloud providers is problematic because it requires learning and using multiple tools specific to each provider, such as AWS CloudFormation Templates (CFT) and Azure Resource Manager, which can be time-consuming and inefficient.
What is a Compute Resource as mentioned in the script?
-A Compute Resource, as mentioned in the script, refers to hardware resources like CPU and RAM, which can be created on any cloud platform or on-premises.
What does the script suggest as an alternative to learning multiple IaC tools for different cloud providers?
-The script suggests Terraform as an alternative to learning multiple IaC tools, as it allows DevOps engineers to write scripts that can automate infrastructure across any cloud provider using a single tool.
What is the significance of the term 'API as code' in the context of Terraform?
-The term 'API as code' in the context of Terraform refers to the concept where Terraform scripts are used to interact with the APIs of cloud providers, abstracting the need to write direct API calls.
What is the role of Terraform in automating infrastructure across multiple cloud providers?
-Terraform's role in automating infrastructure across multiple cloud providers is to act as an intermediary that translates Terraform scripts into API calls that the respective cloud provider can understand and execute.
What is the concept of a Hybrid Cloud Model as discussed in the script?
-The concept of a Hybrid Cloud Model, as discussed in the script, is where an organization hosts part of its infrastructure on one cloud provider and another part on a different cloud provider, leveraging the strengths of each.
What is the advantage of using Terraform over specific IaC tools for individual cloud providers?
-The advantage of using Terraform over specific IaC tools for individual cloud providers is that it simplifies the learning curve and migration process, as it can handle automation across different providers with minimal changes to the scripts.
What is the difference between Infrastructure as Code (IaC) and Terraform?
-Infrastructure as Code (IaC) is a broader concept where infrastructure is defined and managed using code, which can be done with various tools like AWS CFT or Azure Resource Manager. Terraform, on the other hand, is a specific tool that implements IaC and adds the capability to interact with multiple cloud providers' APIs through a single scripting language.
What is the script's stance on the complexity of managing IaC without a tool like Terraform?
-The script's stance is that managing IaC without a tool like Terraform can be complex and inefficient due to the need to learn and use multiple specific tools for different cloud providers, which can lead to wasted effort and time.
What is the purpose of the video script's explanation of API in the context of Terraform?
-The purpose of the video script's explanation of API in the context of Terraform is to clarify how Terraform uses APIs to communicate with cloud providers, allowing for automation without the need for direct API call scripting.
Outlines
π Introduction to DevOps and Infrastructure as Code
In this introductory paragraph, Abhishek, the host of the channel, welcomes viewers back and outlines the agenda for the day. He briefly recaps the previous lessons on configuration management and Ansible, addressing the audience's request for a dedicated video on Dynamic Inventory. The main focus for the current session is Infrastructure as Code (IaC). Abhishek explains the importance of understanding the problem before exploring the solution, which includes the use cases and the concept of IaC. He uses the example of a DevOps engineer at Flipkart to illustrate the challenges and processes involved in managing compute resources across various platforms like AWS, Azure, and on-premises.
π The Challenge of Multi-Cloud and Hybrid Infrastructures
This paragraph delves into the complexities faced by DevOps engineers when managing infrastructure across multiple cloud providers or hybrid environments. Abhishek discusses the limitations of provider-specific tools like AWS CloudFormation Templates (CFT) and Azure Resource Manager, which necessitate rewriting scripts when migrating infrastructure from one provider to another. He highlights the impracticality of learning and using multiple tools for different cloud providers and introduces the concept of a hybrid cloud model, where organizations may host parts of their infrastructure on different platforms based on specific needs or cost benefits.
π Introducing Terraform and API as Code
Abhishek introduces Terraform, a tool developed by HashiCorp, as a solution to the problem of managing infrastructure across multiple cloud providers without learning multiple tools. Terraform operates on the concept of API as Code, allowing DevOps engineers to write scripts that Terraform then translates into API calls for the respective cloud provider's API. This approach simplifies the process of automating infrastructure, as it abstracts the need to interact directly with each provider's API. The paragraph also discusses the benefits of Terraform in terms of reducing the learning curve and streamlining the migration process between cloud providers.
π Understanding APIs and Terraform's Role in Automation
In this segment, Abhishek clarifies what an API (Application Programming Interface) is and how it facilitates programmatic communication with applications. He uses Google as an example to explain how APIs can be used to retrieve information without the need for manual interaction through a user interface. The paragraph further explains how Terraform leverages APIs to automate infrastructure provisioning across different cloud providers, thereby simplifying the process for DevOps engineers and reducing the need for multiple specialized tools.
π Wrapping Up and Previewing Future Lessons
To conclude the video script, Abhishek summarizes the key concepts discussed, namely Infrastructure as Code (IaC) and API as Code. He emphasizes the utility of Terraform as a unified language for infrastructure automation across various cloud platforms. Abhishek invites viewers to provide feedback and questions, especially if they need further clarification on certain topics. He also encourages viewers to like, comment, and subscribe to the channel for more educational content, promising a future video with live examples and a project using Terraform.
Mindmap
Keywords
π‘DevOps
π‘Infrastructure as Code (IaC)
π‘Dynamic Inventory
π‘AWS CLI
π‘AWS CloudFormation Templates (CFT)
π‘Hybrid Cloud
π‘API as Code
π‘Terraform
π‘Azure Resource Manager
π‘OpenStack
π‘Heat Templates
Highlights
Introduction to a dedicated video on Dynamic Inventory to clear doubts.
Exploration of Infrastructure as Code (IaC) and its importance in DevOps.
Understanding the problem with manual creation of compute resources across different platforms.
The role of a DevOps engineer in managing infrastructure for a company like Flipkart.
Different options for compute resource deployment, including cloud platforms and on-premises.
Automation of AWS infrastructure using AWS CLI and CloudFormation Templates (CFT).
Challenges faced when migrating infrastructure scripts from AWS to Azure due to vendor lock-in.
The concept of hybrid cloud and its adoption in organizations for flexibility.
Explanation of the shift from AWS to Azure and the need to rewrite automation scripts.
Introduction to Terraform as a solution to the problem of learning multiple IaC tools.
Terraform's ability to abstract away the complexities of different cloud provider APIs.
The smooth transition between cloud providers facilitated by Terraform's API as code approach.
Terraform's role in reducing the effort required to migrate infrastructure across cloud providers.
Explanation of API as code, allowing automation of any provider's infrastructure through their APIs.
Terraform's use of API as code to communicate with cloud provider APIs, simplifying the process for DevOps engineers.
Upcoming live examples and a project using Terraform to demonstrate its practical application.
Learning outcomes from the video, including Infrastructure as Code (IaC) and API as code concepts.
Invitation for feedback and questions from viewers to improve the content and understanding.
Transcripts
hello everyone my name is Abhishek and
welcome back to my channel so today we
are at day 16 of our complete devops
course in the previous classes we talked
about configuration management we talked
about ansible and yeah I know few of you
have uh doubts with respect to Dynamic
inventory that is something that we did
not talk in the previous class so with
your request what I've decided is yeah
I'll make a completely dedicated video
on Dynamic inventory and I'll make sure
that all your doubts are cleared but for
today we'll be talking about
infrastructure as code okay so today's
topic is infrastructure as code and in
today's class we'll completely talk
about the theory part uh by Theory I
mean to say we'll try to understand what
is the problem right because first of
all we need to understand what is the
problem and then we will see what are
the use cases
and finally we'll try to understand what
is the solution okay
so without wasting any time let's
quickly start the video and try to
understand infrastructure as code
now let's say that you are a devops
engineer okay so you are a devops
engineer who is working for an
organization called uh let's say
Flipkart okay
so you're working for a Flipkart
so Flipkart as a company they can create
their compute resources anywhere right
so what is a compute resource compute
resource is nothing but uh CPU Ram or
you know any other Hardware resources
they can create on any of the cloud
platform or they can also create on
their on-premises that is Flipkart has
let's say 300 applications okay so to
deploy these 300 applications they would
need servers because servers is a player
where you deploy your applications and
expose to the customers now Flipkart can
deploy these applications on different
options like they can deploy these
applications on AWS
or they can deploy these applications on
azure
Google Cloud platform or they can also
deploy it on their physical servers that
is on-premises right
and there are a bunch of other options
these are not only the four options okay
this can exceed like you know you have
Oracle Cloud you have digital ocean
hundreds of options are there so as a
devops engineer let's say you have
decided that okay you evaluated all of
these things and you said that okay
so Flipkart will host all of their
infrastructure on AWS okay so let's say
as a devops engineer you have taken this
decision and this decision went very
well and what you have done is you know
you started to create servers on like it
can be easy to instances or it can be S3
buckets okay or it can be RDS database
all of these resources are required for
your organization and as a good devops
engineer what you have done is you have
automated the entire process on AWS like
instead of creating manually through the
console what you have decided is you
will use the AWS CLI okay or you can
also use something like AWS cloud
formation templates okay CFT let's say
cloudformation templates as CFT so in
the previous class when we talked about
AWS automation I already mentioned you
like there are multiple options you can
use AWS CLI you can use AWS CFT you can
use uh you know AWS cdk many options
right
so as a devops engineer you decided that
your organization will use AWS cloud
formation templates okay so what we have
done is you have used AWS cloud
formation templates which is called as
CFT and you have automated the
infrastructure on AWS Cloud platform
okay so let's say this is AWS cloud and
you have what you have you done you have
automated this stuff
so any developer who comes to you and
tells you okay Abhishek I want
10
ec2 instances so what you'll do you'll
say that yeah using cfts I already wrote
the script now I just need to execute
the CFT script for ec2 instances and you
have 10 ec2 instances will be generated
in no time similarly somebody comes to
it with S3 buckets everything okay so
you wrote some hundreds of scripts okay
so you wrote some hundreds of scripts
and all these hundreds of scripts are
stabilized and your work is going pretty
well okay now you don't have any
problems but what happened all of a
sudden is your AWS okay the decision
that you take in okay AWS your
management said that okay we were using
AWS till date but due to some reasons
like you know you are not happy with the
AWS support or you know the costing that
AWS is uh you know the cost that AWS is
uh giving to you you are not happy with
it so what you have decided is you will
shift from AWS to Microsoft azure okay
so what have you decided from AWS you
want to move to Microsoft Azure now all
these hundreds of scripts that you have
written will be of no use because again
if some developer comes to you and says
that Abhishek I would need 10 virtual
machines on your Azure platform okay so
you cannot say that I have written CFT
you can use the CFT because CFT is very
specific to your AWS cloud provider
right so CFT only belongs to your AWS
cloud provider
so again you have to put all the effort
and you have to migrate this hundreds of
scripts into Azure resource manager like
similar to AWS CFT as your support
something called as Azure resource
manager okay so you put a lot of effort
and again you converted the hundreds of
scripts into Azure resource manager and
you have again automated the entire
process okay let's say you have again
automated the entire process now what
your organization said is okay whether
it is Flipkart or any organization said
okay we moved to Azure as well and we
were not even happy with Azure okay so
again the support with Azure is not good
or you know you are not happy with the
costing so you wanted to move from Azure
this time you want to move to on-premise
okay so that means you want to maintain
your own infrastructure okay so on
premise so what you have done is you
purchase your own servers and on top of
the servers you used openstack okay
openstack is used to create your own
cloud okay so you created your entire
Data Center and on the data center you
installed openstack okay and you have
decided that okay anybody who comes with
a request for 10 VMS or 20 VMS or you
know for any kind of request we will say
that okay we have on-premise and we have
installed openstack so you will get it
but for that reason what you need to do
is you have to again migrate these
scripts okay that worked firstly on AWS
CFT now you migrated them to Azure
resource manager and now you have to
migrate them to heat templates
okay so heat templates is a way how
openstacks automates the process okay so
if you want to automate any
infrastructure on uh openstack you will
use something called as key templates so
again you waste a lot of time and you
migrated the entire thing to heat
templates so this was the process that
was going on and you know this is just
an example these days what usually
companies are doing is they are usually
operating in hybrid Cloud Model okay so
what is hybrid Cloud Model so any
organizations okay depending upon the
requirements what they can do is they
can host part of their infrastructure on
AWS and part of their infrastructure on
Azure now why is this happening this is
because okay this was scenario number
one okay so what I explained you here
this is scenario number one and in
scenario number two there is another
option that people are moving to hybrid
Cloud okay so what is hybrid Cloud
hybrid cloud is a way using which you
can create part of your infrastructure
on AWS and part of your infrastructure
on Azure that is because let's say AWS
is giving you good offers or you know
AWS is giving you good support with
respect to the uh let's say for example
it is giving good support with respect
to the storage Services okay so for
storage Services you have decided to use
AWS and for devops related services for
the build related services or you know
for the
um
the project management related Services
you want to use Azure devops okay so for
devops you want to use Azure and for the
build or sorry for the storage related
Services you want to use AWS this is
because AWS is good with the storage
Services just an example and Azure is
good with the
uh devops related services so your
organization that is Flipkart has
decided to go with this hybrid Cloud
architecture okay so now as a devops
engineer what you need to do is you need
to learn both
okay so you need to learn both AWS CFT
for writing the automation on AWS and
you need to learn Azure resource manager
for automating stuff on Azure and this
problem can vary from organization to
organization and also team to team uh
you know there might be some secure
Services even for a Flipkart where you
know you don't want to use AWS you don't
want to use Azure and you want to use
on-premise okay so again for that reason
you have to write heat templates or
something so to avoid all these problems
okay to solve the problem of learning
too many things okay to instead of uh I
just gave you an example with two
infrastructure resources that is two
Cloud providers that is AWS and Azure
this problem can go can grow leaps and
bonds right there can be hundreds of
cloud providers as a devops engineer
which tool will you learn so to solve
this problem what people have come with
is terraform okay so who is addressing
this problem terraform and the concept
here is
API
as code okay
so what all of the previous tools were
called so here for example AWS CFT or
Azure resource manager they were all
called as infrastructure as code because
you are writing your infrastructure as
code but the problem there was even you
are writing your infrastructure as code
you know you are not manually creating
the resources you are creating the
infrastructure using Code so that was
good but the problem was in the world of
hybrid cloud and in the world of
multiple crowd providers so you cannot
solve the problem with different
different options like you know AWS CFT
Azure resource manager you have to learn
hundreds of tools so terraform came who
is terraform terraform is a tool that is
developed by hashicor
okay so hashikov came ahead and said
that okay devops engine is don't learn
hundreds of tools okay so they said
devops Engineers that instead of
learning hundreds of tools just learn
one tool and that tool is called as
terraform okay now will telephone solve
all your problems or not that is uh you
know I'll explain you there are some
problems with terraform as well in the
future videos or I already created a
video on my Channel with respect to
terraform but the main purpose of
terraform is to solve the problem of
learning too many tools or you know
execution of too many tools now
terraform K header came ahead and said
that
okay if you are a devops engineer what
you can do is basically you can create
terraform scripts okay so write some
terraform Scripts
and terraform what it will basically do
is terraform will take care of
automating the resources whether it is
AWS whether it is azure whether it is uh
like you know
let's say this is AWS this is azure
and this is gcp okay so as a devops
engineer what you need to do is you need
to write the terraform scripts and
terraform will take care of automating
the resources on the cloud provider so
you just need to tell the terraform that
okay my provider is AWS
okay so once you say that my provider is
AWS so terraform will say that okay uh
give me the scripts for AWS and I can
automate the stuff related to AWS now
you might ask me okay so the problem is
the same even now if you want to migrate
from One Cloud provided to other cloud
provider you just have to modify I mean
you you still need to modify the
terraform scripts right so in the
previous slides I explained you that
if you want to move from one cloud
provider to the another cloud provider
the problem is that you have to migrate
your scripts but within terraform what
terraform tells you is that the
migration is very smooth like your
organization if it decides to migrate
from AWS to Azure you need to make some
minimum changes like you know you need
to update some modules or uni you know
you need to update the provider details
and once you do that you know you can
easily migrate from AWS to Azure okay so
this is how terraform solves the problem
and the other thing is instead of
learning too many tools okay so you
don't have to learn about AWS CFT you
don't have to learn about Azure resource
manager and again uh you know heat
templates or XYZ tools you can learn
just one tool and this one tool can take
care of creating new modules for you and
you know just looking at the terraform
documentation you can write automation
for any infrastructure so this concept
is called as API as code
okay so what is apis code so API as code
is a concept using which you can
automate any provider whether it is AWS
Azure or openstack anything using their
apis okay so what terraform does
internally is terraform will talk to the
API of azure terraform will talk to the
API of gcp or terraform will talk to the
API of openstack so using which like you
know you don't have to write any code so
once you write the terraform templating
or you write the terraform script files
terraform will convert them depending
upon the provider details that you have
provided so in one of the files called
provider.tf you say that the provider is
AWS so once you write the terraform
module terraform will convert that
script into AWS readable API okay or the
API request that AWS can understand and
it executes the action and gives you the
result back okay so this is how
terraform is one tool that can automate
your resources on any cloud provider
Okay so this is a concept of API as code
now when I explained a similar Concept
in the last class many people were
asking me what is API
okay so many people did not understand
what is API so let me take a moment and
explain you what is API so API is
nothing but application
interface okay so basically using API
you can talk to any application and you
can get the response back for example
okay let's take a basic example you want
to talk to Google
okay so what is the usual process you
open your laptop okay you open your
laptop and on your laptop you will open
your favorite browser and you will type
www dot google.com Okay so once you type
this what happens is you send your
request to Google and you get a browser
page and on the browser page you can
search the information related to Google
so this is basically a user interface
right so this is a way how you manually
do things but whenever you want to
automate things or you know whenever you
want to do things programmatically so
this way is not suitable right because
let's say you want to get some
information from Google in a
programmatical manner okay so instead of
logging into your laptop instead of you
know opening the browser and getting the
information from Google what you want to
do is you just want to run a script okay
and this script will talk to the Google
okay this script will talk to Google and
it executes the action and sends you the
result back
to perform this kind of things what
developers have come okay whether it is
Google or whether it is uh Argo CD or
whether it is any application what
developers have done is they have come
up with a concept called API okay so
using API you can programmatically talk
to any applications so whether it is
GitHub whether it is Google whether it
is uh Flip Flip I don't know if Flipkart
has an API sorry or any other
applications what they will do is Google
GitHub will expose their API okay so
GitHub will tell you that okay instead
of manually logging into GitHub
authenticating with me and you know
opening your browser and going through
the flows what you can do is you can
open your terminal okay once you open
the terminal you can use a curl request
or you know you can use a HTTP get
request and using this HTTP get request
or curl request you can talk to my API
and get whatever information that you
want and I'll send you the result
right so this way you can talk to any
applications programmatically using the
apis okay so that is the only difference
either you manually talk to the
applications using the user interface or
you talk to the applications using a
programmatic interface so the
programmatic interface in this case is
called as API that's the concept of API
now terraform uses the same concept okay
what terraform does is terraform AWS has
their own API okay and Azure has their
own API what terraform has done is
instead of okay instead of developers
directly talking to the apis because if
you want to talk to the apis you need to
know some programming language okay
sometimes get requests are easy but you
know you might want to create resources
on it and you might want to make some
post calls so instead of you writing
this calls in python or shell scripting
what terraform said is that okay we'll
actively look at this apis
okay
terraform will actively look at this
apis and terraform has written their own
modules or terraform has come across
let's say you want to create an ec2
instance on AWS so terraform said that
instead of you directly making API call
with AWS you can write some terraform
file okay and in this terraform file you
can go through the terraform
documentation which will definitely do
tomorrow so terraform will tell you that
okay there is a module called ec2 and
using this ec2 module you just write
bunch of lines just like you write in
English and what we will do is once user
submits this request okay in terraform
file once you do this uh terraform will
receive this request and it will take
your input and it will convert your
input into API call
okay this is happening at the back end
terraform will convert your request into
API call and it gets the request back
and again terraform will send you the
request back to you so as a user you are
not directly talking to the apis of AWS
and Azure but you are writing terraform
scripts and terraform is converting your
request into API calls okay so this is
concept of terraform and this is how
terraform utilizes the concept called
API
as code
so in tomorrow's class we will see
terraform live examples and you know we
will see a live project using terraform
we'll start with the installation we'll
try to create ec2 instances or any
specific stack on AWS and I'll show you
how exactly terraform works so what did
we learn today we learned about two two
concepts one is infrastructure as code
right IAC
and we talked about API as code
so infrastructure as code is a concept
using which
okay so infrastructure as a code is a
concept using which you can automate
your infrastructure
so it can be anything like you know you
can write your cloud formation templates
or you can write your Azure resource
manager or anything so all of these
things fall into the category of
infrastructure as code whereas terraform
is a concept which is used to solve all
of these problems like you know
terraform also works on the same concept
of infrastructure as code but terraform
has advanced capabilities called API as
code right instead of you writing all of
these things terraform says that okay
just learn one language instead of
learning uh AWS uh cloud formation
templates Azure resource manager just
learn one language that is called as
terraform and terraform will take care
of talking to all of those Cloud
providers using the concept of API as
code I hope you understood the concept
well and if you have any questions
please provide me the timestamp and tell
me okay Abhishek I did not understand
the concept here so that I can make a
Youtube short video or you know if it is
possible I can also explain you in the
comment section okay so this is a
concept which you did not understand and
uh yeah your feedback will definitely
help me to improve as well so if you
like the video click on the like button
and if you have any questions not not
just about today's class even in the
previous classes do post them in the
comment section and finally don't forget
to subscribe to our Channel if you
haven't I'll see in the next videos
thank you so much take care everyone
Browse More Related Video
DE Zoomcamp 1.3.1 - Introduction to Terraform Concepts & GCP Pre-Requisites
Terraform Interview Questions | Terraform Scenario Questions | DevOps Interview Series | Terraform
Infrastructure as code
Implementing Infrastructure as Code with Terraform | AWS Cloud Resume Challenge - Part 6
What is AWS Cloudformation? Pros and Cons?
Mastering Terraform Interview Questions: 15 Essential Questions & Answers | Demystifying Terraform
5.0 / 5 (0 votes)