Introduction to HashiCorp Terraform with Armon Dadgar

HashiCorp
29 May 201818:38

Summary

TLDRThe transcript introduces Terraform, a tool for infrastructure provisioning, emphasizing its ability to handle both initial setup (day one) and ongoing evolution (day two plus). Terraform operates on an infrastructure as code approach, allowing users to define their infrastructure in config files. The core process involves three main commands: refresh, plan, and apply, which help reconcile the desired state with the current state of infrastructure. As usage scales from individuals to organizations, Terraform supports collaboration through version control and Terraform Enterprise, and modularity for broad consumption, promoting a unified workflow for infrastructure management.

Takeaways

  • 🚀 Terraform is a tool for infrastructure provisioning that addresses both the initial setup (day one) and the ongoing evolution of infrastructure (day two plus).
  • 📜 Infrastructure as code is a core concept of Terraform, where infrastructure is described declaratively in human-readable configuration files.
  • 🌐 Terraform manages infrastructure across various providers, including cloud providers like AWS, Azure, and GCP, as well as on-premise solutions like OpenStack and VMware.
  • 🛠️ The main Terraform workflow involves three commands: refresh, plan, and apply, which help in reconciling the real-world state with the desired configuration.
  • 🔄 The refresh command updates Terraform's understanding of the current infrastructure state by querying the respective infrastructure providers.
  • 📝 The plan command helps in determining the actions needed to achieve the desired state from the current state, providing a roadmap for infrastructure changes.
  • 🚀 The apply command executes the changes outlined in the plan, managing dependencies and sequencing to ensure a smooth transition to the desired infrastructure state.
  • 🔄 Terraform's architecture consists of a monolithic core that manages the lifecycle of resources and supports numerous providers for connecting with different services.
  • 🔐 Terraform Enterprise is a solution for team collaboration, providing centralized state management, sequential execution of changes, and secure handling of sensitive credentials.
  • 📦 Modules in Terraform allow for encapsulation of infrastructure patterns, making complex setups accessible to a broader audience by defining input and output variables.
  • 🌐 A central registry for Terraform modules exists, enabling sharing of common infrastructure patterns and facilitating self-service infrastructure management within organizations.

Q & A

  • What is Terraform and what problem does it solve?

    -Terraform is a tool for infrastructure provisioning. It addresses the challenges of not only setting up initial infrastructure (day one) but also evolving it over time (day two and beyond). Terraform allows users to define their infrastructure in a set of configuration files, adopting an infrastructure as code approach.

  • What is the significance of the 'infrastructure as code' approach that Terraform uses?

    -The 'infrastructure as code' approach allows users to declaratively define their infrastructure in human-readable configuration files. This makes the infrastructure reproducible, version-controlled, and easier to manage. It simplifies the process of provisioning and evolving infrastructure by treating it as a software development task.

  • What are the three main commands used in Terraform's workflow?

    -The three main commands in Terraform's workflow are 'terraform refresh', 'terraform plan', and 'terraform apply'. 'Refresh' reconciles Terraform's view of the infrastructure with the real world, 'plan' determines the changes needed to reach the desired state, and 'apply' executes the plan against the real world to update the infrastructure accordingly.

  • How does Terraform handle dependencies between different infrastructure components?

    -Terraform intelligently determines the correct order for provisioning resources based on their dependencies. It can identify which resources need to be created or updated sequentially and which can be done in parallel, ensuring that the infrastructure is built and managed efficiently.

  • What is the role of Terraform's core in its architecture?

    -Terraform's core is responsible for processing the configuration files provided by the user and managing Terraform's state. It figures out the relationships between different resources, determines what needs to be created, updated, or destroyed, and manages the entire lifecycle of the infrastructure.

  • How does Terraform support connectivity with various infrastructure providers?

    -Terraform supports many different providers that allow it to connect with a wide range of infrastructure services, including cloud providers like AWS, Azure, and GCP, on-premise infrastructure like OpenStack or VMware, and higher-level platforms and services like Heroku, Kubernetes, and Lambda functions.

  • What is Terraform Enterprise and how does it enhance the Terraform workflow?

    -Terraform Enterprise is a version control system for Terraform configurations. It allows teams to collaborate on managing the same infrastructure by centralizing the state file and ensuring sequential application of changes. It also provides features for managing sensitive credentials securely and avoiding conflicts when multiple team members work on the same infrastructure.

  • What challenges arise when multiple team members collaborate on Terraform configurations?

    -When multiple team members collaborate, challenges include maintaining a consistent view of the configuration, avoiding conflicts when running changes in parallel, and managing sensitive credentials securely. Terraform Enterprise addresses these issues by centralizing state management, ensuring sequential application of changes, and encrypting sensitive variables.

  • How do Terraform modules help in managing and sharing infrastructure configurations?

    -Terraform modules encapsulate infrastructure configurations as black boxes, with input variables to customize their behavior and output variables to provide results. They simplify the management and sharing of common infrastructure patterns by hiding the complexity and making them easily consumable for a broader audience.

  • What is the role of the Terraform registry in the ecosystem?

    -The Terraform registry is a central repository where users can find and share modules for common infrastructure patterns. It includes official modules from cloud providers and contributions from the community, making it easier for users to adopt best practices and manage infrastructure in various environments.

  • How can organizations use a private Terraform registry to standardize infrastructure management?

    -Organizations can run a private Terraform registry within their infrastructure to standardize and manage their internal infrastructure patterns. Publishers can contribute approved patterns to the registry, allowing a larger base of consumers to self-service their infrastructure needs while adhering to organizational best practices.

Outlines

00:00

🛠️ Introduction to Terraform and Provisioning

This paragraph introduces Armon Dadgar speaking about Terraform, a tool for infrastructure provisioning. It explains the two main challenges in provisioning: the day one challenge of going from nothing to something, and the day two plus challenge of evolving existing infrastructure over time. Terraform addresses these by using an infrastructure as code approach, allowing users to declaratively define their infrastructure in configuration files. This approach enables the description of infrastructure topology in a simple, human-readable way, covering aspects like VPCs, security groups, virtual machines, and load balancers. The paragraph outlines the Terraform workflow, which includes commands like refresh, plan, and apply, to manage the creation, modification, and destruction of infrastructure resources.

05:01

🔄 Terraform Workflow and Evolution

The second paragraph delves into the Terraform workflow, emphasizing the consistency between day one and day two experiences. It describes how Terraform's refresh command updates its view of the infrastructure, the plan command figures out the necessary changes to reach the desired configuration, and the apply command executes these changes. The paragraph also touches on the concept of decommissioning infrastructure using Terraform's destroy command. The advantages of this workflow include the ability to manage infrastructure evolution and the efficient cleanup of resources at the end of their lifecycle. The paragraph then discusses Terraform's architecture, highlighting its core component responsible for lifecycle management and the providers that connect Terraform to various services and resources.

10:01

📈 Scaling Terraform in Teams and Organizations

This paragraph explores how Terraform is used in team and organizational settings, starting with individual practitioners and scaling up to larger groups. It addresses the challenges of collaboration, such as maintaining a consistent view of configurations and avoiding conflicts when multiple team members make changes simultaneously. The paragraph introduces Terraform Enterprise as a solution for centralizing state management and ensuring sequential application of changes. It also discusses the management of sensitive credentials and the transition from early operators to a broader development audience. The concept of Terraform modules is introduced as a way to encapsulate complexity and make infrastructure management more accessible to a wider audience.

15:04

🌐 Sharing and Managing Terraform Modules

The final paragraph focuses on the management and sharing of Terraform modules, both through a global public registry and private registries within organizations. It explains how modules published by cloud providers and internal teams can simplify infrastructure management by abstracting away complexity and presenting a standardized approach to provisioning. The paragraph discusses the benefits of using a private registry within an organization's Terraform Enterprise setup, allowing for the publication of approved infrastructure patterns and self-service access for internal developers. It outlines different ways of consuming modules, from direct Terraform code usage to a more graphical, WYSIWYG approach for those less familiar with Terraform. The paragraph concludes by emphasizing Terraform's role in enabling a wide range of users to self-serve infrastructure while adhering to organizational standards and not overburdening the providers with custom infrastructure generation.

Mindmap

Keywords

💡Terraform

Terraform is an open-source tool for infrastructure provisioning and management. It allows users to define their infrastructure in a declarative language, which Terraform then uses to create, update, and destroy resources as needed. In the video, Terraform is presented as a solution to both the 'day one' challenge of initially setting up infrastructure and the 'day two plus' challenge of managing and evolving that infrastructure over time.

💡Provisioning

Provisioning refers to the process of preparing and configuring infrastructure or resources for use. In the context of the video, Terraform is used for provisioning by defining the desired state of infrastructure and then taking the necessary actions to achieve that state. This includes everything from setting up virtual machines to configuring network security.

💡Infrastructure as Code (IaC)

Infrastructure as Code is a practice that involves managing infrastructure through code-based tools and processes, similar to how software is managed. In the video, Terraform embodies this concept by allowing users to write configuration files that describe the desired state of their infrastructure, thus treating infrastructure as a 'code' that can be version-controlled and replicated.

💡Configuration Files

Configuration files in the context of Terraform are the documents where users define their infrastructure's desired state using a declarative language. These files are human-readable and specify the parameters and settings required for each infrastructure component.

💡Workflow

In the context of Terraform, a workflow refers to the sequence of steps or actions taken to manage infrastructure. This includes the core commands of 'refresh', 'plan', and 'apply', which are used to synchronize the real-world infrastructure with the Terraform's configuration and desired state.

💡State Management

State management in Terraform involves keeping track of the current state of all provisioned resources. This state is used by Terraform to understand what actions need to be taken to maintain the desired state of the infrastructure. Proper state management is crucial for ensuring consistency and avoiding conflicts when multiple users are working with the same infrastructure.

💡Providers

Providers in Terraform are the components that enable interaction with various infrastructure services, such as cloud platforms or on-premise resources. Each provider offers a set of resources that Terraform can manage, allowing users to leverage a wide range of technologies and services within their infrastructure code.

💡Terraform Enterprise

Terraform Enterprise is a commercial version of Terraform that offers additional features for team collaboration, workflow management, and scale. It includes a central repository for storing Terraform configurations and state files, as well as tools for managing access control and variable encryption.

💡Modules

Modules in Terraform are reusable units of infrastructure that encapsulate a set of resources and their configurations. They accept input variables to customize their behavior and can output variables that provide information about the resources they create. Modules simplify complex infrastructure patterns and make them accessible to a broader audience.

💡Registry

A Terraform registry is a centralized repository where modules can be published, shared, and discovered. It provides a catalog of pre-built infrastructure patterns that can be used by Terraform users to quickly and easily deploy common configurations without having to write the code from scratch.

Highlights

Terraform is a tool for infrastructure provisioning.

Provisioning involves two main challenges: day one and day two plus.

Day one challenge is going from running nothing to running something.

Day two plus challenge is evolving existing infrastructure over time.

Terraform uses an infrastructure as code approach.

Infrastructure is defined in Terraform config files.

Terraform allows for the declarative definition of infrastructure topology.

The Terraform workflow involves three main commands: refresh, plan, and apply.

Terraform refresh reconciles Terraform's view with the real world.

Terraform plan reconciles the real world with the desired configuration.

Terraform apply executes the plan against the real world.

Terraform's architecture consists of a monolithic core and multiple providers.

Providers are how Terraform connects to different infrastructure and services.

Terraform can manage a wide range of resources, from IaaS to SaaS.

Terraform is an open source project with over 100 providers and thousands of contributors.

Terraform Enterprise is a tool for team collaboration and infrastructure management.

Terraform modules encapsulate infrastructure complexity and can be shared through a registry.

A private registry can be run within an organization for internal module sharing.

Terraform enables self-service infrastructure for a broad audience while maintaining compliance.

Transcripts

play00:09

- Hi, my name is Armon Dadgar,

play00:10

and I wanna talk a little bit about Terraform today.

play00:13

So Terraform, if you've never heard of it,

play00:15

is our tool for doing provisioning.

play00:17

So when we talk about provisioning,

play00:19

there's really two different problems

play00:21

we're talking about, right?

play00:22

So as we provision, there is our day one challenge

play00:25

where we haven't actually started running anything yet.

play00:28

And so how do we go

play00:29

from running nothing to running something?

play00:31

And then there's actually our more challenging day two plus,

play00:34

which is: great, day one, we had nothing and we provisioned

play00:37

our initial set of infrastructure.

play00:39

And then day two, we have an existing set of infrastructure,

play00:42

but we're trying to evolve it.

play00:43

We're changing it over time.

play00:44

We're adding new services, removing services,

play00:47

and generally evolving the way our infrastructure looks.

play00:51

So how does Terraform really solve this problem?

play00:54

We fundamentally believe in taking

play00:55

an infrastructure as code approach.

play00:57

And so what I mean by that is we allow you

play01:00

to declaratively define in a set

play01:02

of Terraform config files

play01:04

what you want your infrastructure to look like.

play01:07

So in something that's very simple and human readable,

play01:10

you describe your overall topology.

play01:13

And so what I mean by that is when we talk about

play01:15

infrastructure as many moving pieces,

play01:17

I might have a VPC or my core network.

play01:20

On top of that, I might provision my security group rules,

play01:23

the way my network security is set up.

play01:25

And then within that environment,

play01:27

I define my set of virtual machines.

play01:28

I might have some VMs that I wanna provision.

play01:31

And then overlaid on top of that, I have a load balancer.

play01:34

And so you can imagine how this graph extends over time.

play01:37

I add different pieces,

play01:39

different amounts of a complexity to it,

play01:41

and I incrementally evolve my infrastructure

play01:43

day over day, week over week.

play01:46

And so while this is sort of graphical in nature,

play01:49

the way we capture it in a Terraform configuration

play01:52

is very lightweight in describing

play01:54

sort of high-level resources.

play01:55

We describe, you know, the VPC resource

play01:58

and the fields that are required

play01:59

to create this element of our infrastructure.

play02:02

And then as we define these other things,

play02:04

we can reference the other components of our infrastructure.

play02:08

And so what Terraform does is gives us a workflow

play02:12

for creating and managing this.

play02:14

And ultimately, this is sort of three different

play02:16

main commands that we run.

play02:18

The first command is Terraform refresh.

play02:21

And what Terraform refresh does is basically reconcile

play02:24

what Terraform thinks the world looks like,

play02:27

Terraform's view of our infrastructure,

play02:30

with the real world.

play02:32

And so how this works is Terraform goes out

play02:34

and talks to the real world.

play02:35

It'll query our different infrastructure providers,

play02:38

whether that's VMware, or Amazon, or Azure,

play02:40

and ask them, what's actually running?

play02:42

And so in this way, Terraform gets an up-to-date view

play02:45

of what our infrastructure actually is.

play02:47

Then we run a plan.

play02:49

A plan is where Terraform

play02:50

figures out what it needs to do.

play02:52

And so you can think about a plan as reconciling

play02:55

sort of the real world,

play02:56

what things are actually running,

play02:59

with what we want to be running,

play03:00

our sort of desired configuration.

play03:04

So when we talk about Terraform's configuration,

play03:07

what we're putting into this graph,

play03:08

this is our desired state of the world.

play03:10

This might not actually be the world as it is.

play03:13

So as an example, on day one, when we execute this,

play03:16

nothing is yet running.

play03:17

So when Terraform runs the plan,

play03:19

it realizes that there is nothing

play03:20

yet running in the real world.

play03:22

So to get to this desired configuration,

play03:25

Terraform needs to boot all four of these resources.

play03:27

And so this becomes the plan.

play03:29

Terraform will give us an output that says:

play03:31

to bring the world to look like what you want it to be,

play03:34

I'm gonna go boot these four things.

play03:36

So then the last part is actually doing this.

play03:38

This is the apply phase of Terraform's execution.

play03:41

So when we apply Terraform's execution plan,

play03:43

it starts with our plan of what we need to do,

play03:45

and it goes out and executes it against the real world.

play03:49

And so in doing this, Terraform will go out and figure out,

play03:52

what is the right order in which these need to be done?

play03:55

So there's a natural dependency here.

play03:57

The network must be defined first

play03:59

before we can define the security groups around it.

play04:01

Once those are defined, we can boot all of our VMs,

play04:03

whether we're booting one or booting,

play04:05

you know, hundreds of these in parallel.

play04:07

And Terraform will figure out where it has

play04:09

that opportunity to create things in parallel

play04:11

and where it needs to be sequential.

play04:13

And once these VMs are available,

play04:15

then Terraform can lastly create

play04:16

our load balancer and say, all done.

play04:20

So this gives us a way to do our day one infrastructure.

play04:23

So day one, we described

play04:24

what we want our universe to look like.

play04:26

We ran Terraform refresh,

play04:28

it said there was nothing existing yet.

play04:29

Plan said all of these things are gonna be created,

play04:31

and then apply goes out and builds it.

play04:33

What now happens over day two

play04:35

is we start evolving our infrastructure.

play04:37

So where this was our infrastructure day one.

play04:40

Day two we might come in and say,

play04:41

I really want a DNS record that points to my load balancer.

play04:45

I want to have, you know,

play04:47

a content distribution network

play04:48

like Fastly sitting in front of that.

play04:51

And I want my VMs to be connected to a monitoring system,

play04:52

so I might monitor these different VMs.

play04:57

And so now we change our Terraform configuration,

play05:01

we enhance it by adding, you know,

play05:02

the definition of these other elements to our TF config,

play05:07

and we rerun the exact same workflow.

play05:09

So when we run refresh,

play05:10

Terraform will realize these first four resources exist,

play05:13

but these new ones do not.

play05:14

And so when we run our plan, it'll tell us

play05:16

that nothing needs to be done with these four.

play05:18

They already exist and they're in the desired state,

play05:21

but that we must add three new elements

play05:23

to get to where we need to be.

play05:24

And so now when we run Terraform apply,

play05:26

it goes ahead and talks to the real world

play05:28

and adds these three new things

play05:30

and brings the world into looking like our desired state.

play05:34

The advantage of this workflow is a few fold.

play05:36

One, our day one experience is identical to our day two.

play05:39

And this really is important

play05:40

because day two is sort of forever:

play05:42

this is something we're gonna keep doing.

play05:44

And then the other advantage of this

play05:47

is something we actually skipped,

play05:49

which is day N when we decide we might need

play05:51

to decommission this.

play05:52

We don't need this service anymore.

play05:54

We don't want this infrastructure.

play05:55

Maybe this was just a staging environment

play05:57

or a test environment.

play05:58

We then have the option to come in and destroy.

play06:01

And when we do a destroy, it's basically an unwinding

play06:04

of everything that's been done, right?

play06:06

In some sense, it's a specialized version of apply,

play06:09

which really looks at, creates a special destroy plan,

play06:13

and then goes out and talks to the real world

play06:15

to destroy all the elements that exist.

play06:18

And so now we can efficiently, at the end of our lifecycle,

play06:21

go from nothing to something, evolving it day over day.

play06:24

And the final day, when we need to decommission it,

play06:26

Terraform knows how to clean up all the resources

play06:29

associated with that application.

play06:32

So, so far, everything we've talked about

play06:34

is really about the workflow.

play06:35

This is not specific to any technology.

play06:38

So how does Terraform actually make any of this stuff work?

play06:41

This is where Terraform's architecture becomes important.

play06:45

So when we talk about the way Terraform actually works,

play06:48

there's sort of two major components.

play06:51

One is Terraform's core.

play06:53

So the monolithic core is responsible for a few things.

play06:57

It takes the Terraform configuration,

play07:01

which is being provided by the user,

play07:04

and then it takes Terraform's view of the world

play07:06

or Terraform's state.

play07:08

This is managed by Terraform itself.

play07:10

And so these get fed into the core,

play07:12

and the core is responsible for figuring out,

play07:14

what is that graph of our different resources?

play07:16

How do these different pieces relate to each other?

play07:19

What needs to be created, what needs to be updated,

play07:21

what needs to be destroyed?

play07:23

And so it does all of the essential lifecycle management.

play07:26

On the backside, Terraform supports

play07:28

many different providers.

play07:31

And providers are how Terraform

play07:32

connects out to the rest of the world.

play07:35

So these can be things like cloud providers,

play07:37

things like AWS, or Azure, or GCP.

play07:42

They could be on-premise infrastructure.

play07:44

So things like OpenStack or VMware,

play07:49

but it's not restricted to just infrastructure as a service.

play07:53

Terraform can also manage higher-level things,

play07:55

platforms as a service such as Heroku, or Kubernetes,

play08:01

or things like Lambda functions.

play08:04

So these are higher level

play08:06

than what we would traditionally consider

play08:08

infrastructure as a service,

play08:09

more in the realm of a platform as a service.

play08:13

The other types of things Terraform can manage

play08:14

are pure software as a service.

play08:16

Very high-level things, maybe monitoring like Datadog,

play08:19

or CDNs like Fastly,

play08:22

or even things like managing GitHub teams.

play08:27

So these are what we'd consider pure software as a service.

play08:29

These are externally managed services,

play08:32

but the view that we take as Terraform authors

play08:35

is that all of these pieces are connected.

play08:37

When I talk about a modern infrastructure,

play08:40

I might be composing all of these different resources.

play08:42

So I might have some set of my infrastructure

play08:45

that is infrastructure as a service,

play08:47

and then I might compose that and say, you know what?

play08:49

I'm gonna provision a Kubernetes cluster,

play08:52

and on top of my Kubernetes cluster,

play08:54

I'm going to define a namespace,

play08:56

and I'm going to define services.

play08:58

And so these now exist

play09:00

on top of infrastructure as a service,

play09:02

and I might compose even these

play09:03

with higher-level things like DNS and CDNs.

play09:08

And so all of these

play09:09

are important pieces of our infrastructure.

play09:12

We can't deliver our application without the IaaS,

play09:15

without the platform as a service,

play09:16

and without the higher-level software as a service.

play09:18

They're all part of our logical end-to-end delivery.

play09:21

And what we wanna get to with Terraform

play09:23

is a single, unified workflow.

play09:25

We don't want it to be: you use one tool to manage this,

play09:28

and now you have to figure out, you know,

play09:30

how you get past this disjointed experience

play09:32

to manage the next section,

play09:33

and then another disjointed experience.

play09:35

And you have to piece together all these things.

play09:37

Instead, anything that really has an API

play09:40

and has a lifecycle associated with it,

play09:42

how can we enable that to be a provider

play09:44

such that Terraform can help us

play09:45

solve this problem end to end?

play09:48

And this is really where providers come in.

play09:50

Today, we have over 100 providers,

play09:53

and those individually can manage

play09:54

over 1,000 different resources,

play09:56

where a resource can be, you know, an AWS EC2 VM,

play10:01

or an Azure Block store, right?

play10:03

These different types of resources or anything that a cloud

play10:06

or even these higher-level things expose.

play10:10

And that list is constantly growing.

play10:11

Terraform is an absolutely massive open source project

play10:14

with thousands of contributors.

play10:16

And so every day, there's new providers and new resources.

play10:19

So how does this really start getting used

play10:22

at different levels within teams?

play10:25

So this becomes an interesting workflow question

play10:27

around how you actually manage Terraform.

play10:31

So initially, this goes back

play10:32

to that core workflow we talked about.

play10:35

So we start with a single individual practitioner

play10:38

who's using Terraform locally,

play10:40

and their workflow is to start

play10:43

by writing some Terraform configuration.

play10:45

Then they run Terraform plan.

play10:48

Plan tells them, what is this configuration gonna do?

play10:50

What does Terraform think needs to be done

play10:53

to apply these changes?

play10:54

And if this looks good,

play10:56

the practitioner will apply the changes.

play10:58

And then this cycle continues.

play11:00

They continue to evolve the infrastructure.

play11:02

This might look very similar to software development,

play11:05

where we're writing some code, we're applying unit tests,

play11:07

we're committing those changes,

play11:09

and then we're continuing our cycle.

play11:11

So what happens is we go from a single individual,

play11:14

to now we add multiple team members.

play11:16

We have other people we wanna collaborate

play11:18

on managing the same infrastructure.

play11:20

Well, there's a few challenges,

play11:22

much like with software writing, which is,

play11:24

how do we make sure we have a consistent view

play11:26

of what the configuration actually is?

play11:28

And how do we make sure we don't step on each other's toes

play11:31

and run multiple changes in parallel?

play11:33

So if we both try and add five new VMs,

play11:36

how do we avoid booting ten new VMs instead?

play11:39

So this problem, in some sense, ends up being very similar

play11:43

to the problem of using Git locally

play11:45

versus using Git as a team.

play11:46

So as we go to a team, we use a system like GitHub

play11:50

to provide that central collaboration.

play11:52

And so our equivalent of that

play11:54

is what we call Terraform Enterprise.

play11:56

So the Terraform Enterprise workflow

play11:58

augments this a little bit.

play12:00

So what we would do instead of now running it locally

play12:03

and running these coordination risks, as an individual,

play12:06

we're still writing our Terraform and planning locally,

play12:09

but now we're pushing that into a version control system.

play12:13

So this could be GitHub, could be Bitbucket,

play12:16

could really be anything.

play12:17

But the idea is, how do we move

play12:18

to having a central repository where we're managing this

play12:21

and coordinating, much like we do for source control?

play12:24

And then we drive off of this

play12:27

our Terraform Enterprise application.

play12:29

And the goal here is a few fold.

play12:31

One is how do we make sure

play12:32

the state management is done locally?

play12:35

So like I said, Terraform keeps track of all the resources

play12:38

it's provisioned so that it knows at destroy time

play12:41

what it needs to tear down,

play12:42

or as we incrementally make changes, what already exists.

play12:45

So this state file is an important thing

play12:47

to have centrally managed so it doesn't diverge

play12:49

or end up with sort of different lineages where, you know,

play12:52

we fork off and then independently run Terraform,

play12:54

and now the state files conflict with one another.

play12:57

So by managing that centrally,

play12:58

we avoid those sort of conflicts.

play13:01

The other important advantage here

play13:02

is that we make sure we're only doing

play13:04

one run at a time, right?

play13:05

So we ensure there's the sequential application.

play13:08

So if Alice and Bob are both working

play13:10

on the same infrastructure

play13:12

and they both try and make a change,

play13:13

they don't step on each other's toes.

play13:15

Lastly, one of the challenges is,

play13:17

oftentimes, in this environment,

play13:19

developers are using local variables

play13:21

to populate things like AWS IAM keys

play13:24

or other sensitive credentials.

play13:26

And so we don't want these sensitive credentials

play13:28

to be strung about on all of our developers' machines.

play13:31

Instead, we want those variables

play13:33

to be kept centrally and encrypted

play13:36

so that we don't have to worry about these things

play13:38

getting exposed or leaking out in plain text.

play13:41

We can put them in a central place

play13:42

and version control and access controls

play13:44

who can actually modify and view these things.

play13:47

So this sort of changes our workflow

play13:48

and now enables a handful of people

play13:51

to collaborate safely together.

play13:54

Well, what starts happening as we start wanting

play13:56

to add even more people, right?

play13:58

So for example, we might be going from

play14:00

sort of our early operators

play14:03

who are very familiar with the cloud environment

play14:05

to exposing it more to our development groups.

play14:09

This is where Terraform's notion of a module comes in handy.

play14:12

So a Terraform module

play14:14

is sort of like a black box of infrastructure

play14:17

where what we can define is a set of input variables

play14:21

that let us toggle the behavior of this module.

play14:23

And the module might emit a set of output variables.

play14:28

So an example of this might be,

play14:30

I might define a module for doing Java,

play14:33

for deploying my Java application in AWS.

play14:36

And my inputs might be, what's the path to my JAR?

play14:38

And how many instances of my Java application do I want?

play14:41

And now, inside this module, we can operate as a black box

play14:45

and deploy either VMs using EC2,

play14:48

we can deploy Lambda functions,

play14:49

we could deploy containers with ECS.

play14:51

All of these become a detail hidden within this module

play14:54

that a consumer doesn't need to know about.

play14:56

And their output can just be, what's the address

play14:58

of the load balancer that hits my application?

play15:01

So in this way, we encapsulate that complexity

play15:03

and make it more accessible to a broader audience.

play15:07

So how do we actually manage and share these modules?

play15:09

In general, we publish these things to a central registry.

play15:15

So there's a central registry

play15:16

that exists today for Terraform.

play15:18

If you go to registry.terraform.io,

play15:21

that's our global public registry.

play15:23

And there you'll find hundreds of modules

play15:25

that cover common forms of infrastructure,

play15:28

common architectural patterns.

play15:30

And many of these are actually published

play15:31

by the cloud providers themselves.

play15:32

So you can log in and see Google's official recommendation

play15:36

for how to provision their network,

play15:38

or Azure's official recommendation

play15:39

for how to help manage a virtual network inside of Azure.

play15:42

And so these modules make it much more consumable for us

play15:46

to operate in these environments,

play15:47

because we don't have to know every detail of it.

play15:49

We can just go and say,

play15:50

I wanna network in Amazon, or Google, or Azure,

play15:53

and fill in a few of the inputs that matter to us,

play15:55

and let the module take care of the rest

play15:57

of the complexity, right?

play15:58

And so the same idea can be extended

play16:00

to different applications,

play16:01

different types of infrastructure.

play16:04

One way we can apply this same pattern internally

play16:07

to our organization is by running a private registry.

play16:10

So instead of using the public registry

play16:11

that's available sort of globally,

play16:13

we can run a private registry within our organization.

play16:16

And this is a capability

play16:17

integrated into Terraform Enterprise.

play16:19

And so what this really lets us start to do

play16:22

is make this split between the set of people

play16:24

internally who are publishers

play16:28

and are familiar with our internal infrastructure

play16:30

and how it should be managed and cloud environments.

play16:33

And they can push in to basically

play16:35

a catalog or a module registry,

play16:38

all of the different key patterns that are needed.

play16:42

So we might publish, here's how we do a Java app,

play16:44

here's how we do a C# app, here's how we do a database,

play16:48

and expose this out to our internal developer community.

play16:51

And then our much larger base of consumers

play16:56

can simply go into the catalog

play16:58

and pull out the things they need.

play17:01

And so these consumers can know much less

play17:03

about how cloud works, but still get these approved modules

play17:06

that are sort of the best practice internally, right?

play17:08

And there's sort of a few ways of consuming this.

play17:11

For people that are more familiar with Terraform,

play17:13

they can consume it just like a normal Terraform module

play17:16

and do an infrastructure as code approach.

play17:18

For folks that are less familiar,

play17:20

they might wanna take more of a WYSIWYG approach,

play17:22

and they can do that with Terraform Enterprise

play17:24

by selecting their modules,

play17:25

filling in the different variables,

play17:27

and basically letting the system automatically generate

play17:30

the equivalent Terraform for them.

play17:32

So they can either use that equivalent Terraform

play17:34

as a starting point and customize it further from there.

play17:36

Or if they have really no interest in learning

play17:38

and using Terraform directly,

play17:39

they can just use the auto-generated Terraform

play17:41

and self-service their infrastructure.

play17:43

And so our goal really becomes,

play17:45

how do we enable this large audience of consumers

play17:48

to get to self-service on infrastructure in a way

play17:51

that is still going to be compliant

play17:53

with how the business wants to do it

play17:55

and not create this immense workload on publishers

play17:58

to constantly generate custom infrastructure

play18:00

and custom definitions for the vast body

play18:02

of consumers within an organization.

play18:04

So at a snapshot, this is Terraform,

play18:06

this is how it applies an infrastructure

play18:08

as code approach to provisioning,

play18:10

really looking at both the day one challenge,

play18:13

and more importantly and more challengingly,

play18:15

the day two-plus challenge.

play18:16

And then the natural journey

play18:18

as we go from a single individual using Terraform

play18:21

to a small team, to kind of an entire organization

play18:24

who's trying to leverage Terraform

play18:25

for infrastructure management.

play18:27

If you're interested in learning more about Terraform

play18:29

or Terraform Enterprise,

play18:30

I recommend checking out our online resources.

play18:32

Thank you.

Rate This

5.0 / 5 (0 votes)

Related Tags
InfrastructureAsCodeProvisioningSolutionsTerraformWorkflowCodeToInfrastructureDayOneProvisioningDayTwoEvolutionTerraformEnterpriseCollaborativeIaCDevOpsEnablementCloudResourceManagement