Receitas: A Terraform Provider

What is Terraform?

In short we can say that Terraform:

Terraform is an infrastructure-as-code software tool created by HashiCorp. Users define and provide data center infrastructure using a declarative configuration language known as HashiCorp Configuration Language (HCL), or optionally JSON.

A little story

I'm not sure about you but I may need a bit more of context here. Maybe we start this with a little story!?

In the software context, infrastructure is all that boring stuff that is needed for our applications to run. When people talk about software development most of the time the talk is all about our specific program. However programs they do not run on ether, unfortunately. Our programs run on a machine, this machine needs an operative system installed, our program alone is not of much use. In real case scenarios our program often needs to communicate with external services, databases, message queues, etc. The machine needs to have an network address, this address needs to belong on a specific subnetwork. All this is infrastructure. All this needs to be managed.

Tradicionally the infrastructure part of software development was done by specialized engineers that were responsible to manage all this complexity. The process was usually ad hoc and manual. This means that it was not very much reproducible nor scalable.

It was clear we need a better way. This better way was to automate these manual processes. From an ad hoc manual process teams start to automate the infrastructure work. This relied heavily on bash scripts. Albeit automated this was yet ad-hoc. Big teams still needed a better way.

In 2014 Mitchell Hashimoto came with a revelation, he noted two distinct sides of infrastructure development.

The infrastructure business use case
The techinal implementation of the infrastructure business case

By infrastructure business case we mean:

I want a database
I want to define a subnetwork and assign ips to my machines
I want a message queue

By technical implementation we mean

I need all the implementation code needed to provide a postgres database
I need to write the network code to allocate, remove, extend private networks
I need the code responsible to install a message queue service and integrate with our auth solution

How does Terraform work

Mitchell Hashimoto noticed that these two are distinct and at the same time complementary. He noticed that the business case could be described in a declarative higher level language. A language simple that would enable us to describe the desired state of the infrastructure. The technical side, however, needed to be implemented with a programming language and should somehow react/interact to the desires made via the higher level language.

This is a very interesting concept. But how does this actually works? I believe this is a question may be popping in your head. The trick is the following.

When you think in declarative terms you can reduce the management of infrastructure to the following:

Creation of a resource
- This happens when you declare a resource for the first time
Change of a resource
- This happens when a resource changes the declaration definition
Delete of a resource
- This happens when the definition of a resource is deleted.

Terrafrom engine at the most basic level implements this concept. Terraform lets you define, in a declarative way, your desired infrastructure state. To be able to translate your desires into concrete actions Terraform needs a way to track these state changes. Unsurprisingly this is done by storing in a state file.

Receitas in Terraform

Receita is the portuguese word for recipe. To make these ideas a bit more clear we will use a demo example. In this small demo project our infrastructure is a restaurant and our resources will be recipes or receitas.

Here you can see an example of the declarative way of defining our infrastructure

terraform {  
  required_providers {
    receita = {
      source = "terraform.local/balhau/receita"
    }
  }
}

provider "receita" {  
  endpoint = "http://localhost:9999"
}

resource "receita_receita" "bola_carne" {  
  name = "Bola de carne"
  #name   = "Batata frita"
  author = "Maria Bacalhau"
}

resource "receita_receita" "bacalhau_todos" {  
  name   = "Bacalhau com todos"
  author = "Antonio Mariscada"
}

resource "receita_receita" "pato_bravo" {  
  name   = "Pato bravo"
  author = "Jose Pato"
}

The declarative nature is pretty clear now, I hope. If you notice there is nowhere in this terraform hcl code anything related to how do we create/update/delete recipes.

If I was not bluntly lying about all this we should have a non-declarative way of managing our recipes. Somehow terraform should be able to map our recipe desires into concrete actions.

We have. In the receita resource we can see how these desires are mapped into concrete actions. Here things are not so human friendly. We can, however reduce this to a set of simple steps.

type ReceitaResource struct {  
    providerData *ReceitaProviderData
}

func NewReceitaResource() resource.Resource {  
    return &ReceitaResource{}
}

// Resource data model
// This defines basically the contract between our resource and the user
type ReceitaResourceModel struct {  
    Name   types.String `tfsdk:"name"`
    Author types.String `tfsdk:"author"`
    Id     types.String `tfsdk:"id"`
}

// Resource terraform contract definitions

// Callback to set the resource metadata information
func (r *ReceitaResource) Metadata(ctx context.Context, req resource.MetadataRequest, resp *resource.MetadataResponse) {

}

// Terraform contract method to setup the schema associated with the terraform resource
func (r *ReceitaResource) Schema(ctx context.Context, req resource.SchemaRequest, resp *resource.SchemaResponse) {  
}

// Callback contract definition that serve as terraform engine configuration
func (r *ReceitaResource) Configure(ctx context.Context, req resource.ConfigureRequest, resp *resource.ConfigureResponse) {

}

//Callback used to create our terraform resource
func (r *ReceitaResource) Create(ctx context.Context, req resource.CreateRequest, resp *resource.CreateResponse) {

}

//Callback used when we update our resource
func (r *ReceitaResource) Update(ctx context.Context, req resource.UpdateRequest, resp *resource.UpdateResponse) {

}

func (r *ReceitaResource) Delete(ctx context.Context, req resource.DeleteRequest, resp *resource.DeleteResponse) {

}

//Callback used when terraform import our recipe 
func (r *ReceitaResource) ImportState(ctx context.Context, req resource.ImportStateRequest, resp *resource.ImportStateResponse) {  
    resource.ImportStatePassthroughID(ctx, path.Root("id"), req, resp)
}

Each of these methods will be invoked by terraform engine when we:

Create a recipe
Update a recipe
Delete a recipe
Import recipe information from terraform state

Now that the concepts are a bit more clear you can explore on your way the recipe demo. In this illustrative terraform provider we used terraform to implement these ideas. To be able to see this things happening we create a simple http server that will respond to create, update, delete requests. These will be triggered by terraform when we create, update, delete recipes.

Yes, this may be a bit overkill way of managing recipes. I confess I don't plan to use it. It is, however, a simple way of understanding the basic ideas behind terraform and how things glue together. These simple ideas are implemented in a more useful way in projects like aws terraform provider and google terraform provider. In these two examples instead of recipes you have all code needed for you to declare resources on both aws and google infraestructure. These are real world example providers with real world usage. They are, however, not so simple to understand, for that recipes taste better.

Menu

What is Terraform?

A little story

How does Terraform work

Receitas in Terraform