KEMBAR78
Infrastructure as Code with Terraform | PDF
Infrastructure as Code with Terraform
Tim Berry
About Me
Google Certified Professional Cloud
Architect + Data Engineer
Some RedHat certs
Run the Google Cloud Platform SRE
Team at Claranet
Have a familiar and boring origin
story based around a ZX Spectrum
linkedin.com/in/timhberry
Definition
Infrastructure as code (IaC) is the
process of managing and provisioning
computer data centers through
machine-readable definition files, rather
than physical hardware configuration or
interactive configuration tools.
(Wikipedia)
• Why Infrastructure as Code?
• The Terraform Lifecycle
• Best Practices
Overview
Why Infrastructure as Code?
A software engineering approach to
an operations function:
• Automated
• Quality Managed
• Modular/Re-usable
• Collaborative
• Auditable
Infrastructure as Code
• Developed by HashiCorp
• First released July 2014
• Open source (MPLv2.0)
• Written in Go
• Cloud Agnostic Neutral
• Rapid development
Terraform
• Major cloud vendors (AWS, GCP, Azure)
• PaaS (Heroku)
• SaaS (PagerDuty, DNSimple, CloudFlare)
• Databases (PostgreSQL, MySQL)
• Community providers
Terraform Providers
The Terraform Lifecycle
• Define your infrastructure
• Initialise your execution environment
• Plan your execution
• Apply your changes
• Destroy your infrastructure
Terraform Lifecycle
• A collection of configuration files in a single directory
• Expressed in HCL Terraform syntax or JSON
• Declarations of:
- Providers
- Resources
- Data Sources
Infrastructure Definition
• A collection of configuration files in a single directory
• Expressed in HCL Terraform syntax or JSON
• Declarations of:
- Providers
- Resources
- Data Sources
Infrastructure Definition
s u f n o b
• A collection of configuration files in a single directory
• Expressed in HCL Terraform syntax or JSON
• Declarations of:
- Providers
- Resources
- Data Sources
Infrastructure Definition
s u f n o b
s u f n u y u n o f
• Parameters can be set via TF or environment variables
• Local gcloud auth or service account (recommended)
Provider
• Resource definitions declare what should exist
• Unique to the provider (GCP resources != AWS resources)
• Cloud resources supported by the vendor API generally supported by
Terraform
• Each resource will require a number of arguments
• Each resource can provide attributes for further use
Resources
• Resource type is google_compute_network
• Resource name is bookface
• Everything else is optional
Resources: Define a Network
• Our subnetwork knows its parent network from this interpolation
• Accessing resource attributes: “${<type>.<name>.<attribute>}”
Resources: Define a Subnetwork
• Many resources require other resources to exist first
- subnetwork required network
• Start with the resource you need and work backwards
• A google_compute_instance will require
- subnetwork
- zone
- image
• Terraform will build a Directed Acyclic Graph of resources to ensure
dependencies are met
Resource Dependencies
• Retrieve or compute values to be used in configuration
• Avoids hard-coding
• Gives dynamic values for things like:
- Zones available in a region
- Images available for Compute Engine
Data Sources
Data Sources
• Find available zones in
europe-west2
• All arguments are optional for
google_compute_zones
• Retrieve a link to the latest
version of the image in the
ubuntu-1710 family
• From the ubuntu-os-cloud
project
Resources: Define a VM
• Prepares the environment
• Runs syntax checks
• Downloads providers
• Populates a .terraform directory
terraform init
terraform init
• Creates a plan for what actions to perform
• In other words, a DAG of changes
• Makes a three-way comparison between configuration, stored state
and actual state
• Outputs a plan confirming what will happen
• Use the saved plan file to apply changes
terraform plan
terraform plan
terraform plan
• Actually makes changes
• Will run another plan if you don’t provide one
• Takes anywhere from a few seconds to many minutes depending on
resources being created
• Changes are written to a JSON state file
- Defaults to a local terraform.tfstate
- Previous state preserved in terraform.tfstate.backup
terraform apply
terraform apply
terraform apply
Magic
• Destroys all the resources in state
• Provides a plan of what will be destroyed and prompts you to confirm
• There is no Undo!
terraform destroy
terraform destroy
Wait, seriously, why
Infrastructure as Code?
Living The Infrastructure Dream:
AUDITABLE - REPEATABLE - DEPENDABLE
• Interacting with a UI is none of those things
• Imperatively using a CLI is maybe 1 of those things
• Infrastructure gets BIG
• You don’t want to start from scratch with every project
Because
Best Practices
terraform fmt
• Rewrites configuration files to
a canonical style and format
terraform validate
• Validates syntax of
configuration files
Lint
• Store infrastructure state in the
project itself
• State locks itself during
operations
• Makes code portable and
enables collaboration
• Git all the things
Remote State
• Maintain separate variables per
environment
• Use Makefile to interact with
terraform
params/dev.tfvars
Environments
Makefile
• Reusable collections of resource
definitions and/or data sources
• Can accept arguments and
provide attributes
• Basically just a bunch of .tf files
in a directory
• Community modules available for
deploying common groups of
resources or applications
tf-gcp-standard-subnetwork
• Standard subnetwork
• HA NAT instances
• Public routes and default NAT routes
Modules
• Terrafile: https://github.com/claranet/python-terrafile
Version management for modules
• TfEnv: https://github.com/kamatama41/tfenv
Version management for TF itself
• Exekube: https://github.com/exekube/exekube
Combines Terraform with Helm for complete IaC on Kubernetes
• Pipelines!
(didn’t have time for this talk, sorry)
Worth a Look
• Terraform’s lifecycle is: init, plan, apply and eventually destroy
• Use data sources and variables and be as generic as possible
• Split variables into environment-specific parameter files
• Use git and remote backends for portability
• Use modules to save time
Summary
Questions?
linkedin.com/in/timhberry
www.claranet.co.uk/about-us/careers

Infrastructure as Code with Terraform

  • 1.
    Infrastructure as Codewith Terraform Tim Berry
  • 2.
    About Me Google CertifiedProfessional Cloud Architect + Data Engineer Some RedHat certs Run the Google Cloud Platform SRE Team at Claranet Have a familiar and boring origin story based around a ZX Spectrum linkedin.com/in/timhberry
  • 3.
    Definition Infrastructure as code(IaC) is the process of managing and provisioning computer data centers through machine-readable definition files, rather than physical hardware configuration or interactive configuration tools. (Wikipedia)
  • 4.
    • Why Infrastructureas Code? • The Terraform Lifecycle • Best Practices Overview
  • 5.
  • 9.
    A software engineeringapproach to an operations function: • Automated • Quality Managed • Modular/Re-usable • Collaborative • Auditable Infrastructure as Code
  • 10.
    • Developed byHashiCorp • First released July 2014 • Open source (MPLv2.0) • Written in Go • Cloud Agnostic Neutral • Rapid development Terraform
  • 11.
    • Major cloudvendors (AWS, GCP, Azure) • PaaS (Heroku) • SaaS (PagerDuty, DNSimple, CloudFlare) • Databases (PostgreSQL, MySQL) • Community providers Terraform Providers
  • 12.
  • 13.
    • Define yourinfrastructure • Initialise your execution environment • Plan your execution • Apply your changes • Destroy your infrastructure Terraform Lifecycle
  • 14.
    • A collectionof configuration files in a single directory • Expressed in HCL Terraform syntax or JSON • Declarations of: - Providers - Resources - Data Sources Infrastructure Definition
  • 15.
    • A collectionof configuration files in a single directory • Expressed in HCL Terraform syntax or JSON • Declarations of: - Providers - Resources - Data Sources Infrastructure Definition s u f n o b
  • 16.
    • A collectionof configuration files in a single directory • Expressed in HCL Terraform syntax or JSON • Declarations of: - Providers - Resources - Data Sources Infrastructure Definition s u f n o b s u f n u y u n o f
  • 17.
    • Parameters canbe set via TF or environment variables • Local gcloud auth or service account (recommended) Provider
  • 18.
    • Resource definitionsdeclare what should exist • Unique to the provider (GCP resources != AWS resources) • Cloud resources supported by the vendor API generally supported by Terraform • Each resource will require a number of arguments • Each resource can provide attributes for further use Resources
  • 19.
    • Resource typeis google_compute_network • Resource name is bookface • Everything else is optional Resources: Define a Network
  • 20.
    • Our subnetworkknows its parent network from this interpolation • Accessing resource attributes: “${<type>.<name>.<attribute>}” Resources: Define a Subnetwork
  • 21.
    • Many resourcesrequire other resources to exist first - subnetwork required network • Start with the resource you need and work backwards • A google_compute_instance will require - subnetwork - zone - image • Terraform will build a Directed Acyclic Graph of resources to ensure dependencies are met Resource Dependencies
  • 22.
    • Retrieve orcompute values to be used in configuration • Avoids hard-coding • Gives dynamic values for things like: - Zones available in a region - Images available for Compute Engine Data Sources
  • 23.
    Data Sources • Findavailable zones in europe-west2 • All arguments are optional for google_compute_zones • Retrieve a link to the latest version of the image in the ubuntu-1710 family • From the ubuntu-os-cloud project
  • 24.
  • 25.
    • Prepares theenvironment • Runs syntax checks • Downloads providers • Populates a .terraform directory terraform init
  • 26.
  • 27.
    • Creates aplan for what actions to perform • In other words, a DAG of changes • Makes a three-way comparison between configuration, stored state and actual state • Outputs a plan confirming what will happen • Use the saved plan file to apply changes terraform plan
  • 28.
  • 29.
  • 30.
    • Actually makeschanges • Will run another plan if you don’t provide one • Takes anywhere from a few seconds to many minutes depending on resources being created • Changes are written to a JSON state file - Defaults to a local terraform.tfstate - Previous state preserved in terraform.tfstate.backup terraform apply
  • 32.
  • 33.
  • 34.
  • 35.
    • Destroys allthe resources in state • Provides a plan of what will be destroyed and prompts you to confirm • There is no Undo! terraform destroy
  • 36.
  • 37.
  • 38.
    Living The InfrastructureDream: AUDITABLE - REPEATABLE - DEPENDABLE • Interacting with a UI is none of those things • Imperatively using a CLI is maybe 1 of those things • Infrastructure gets BIG • You don’t want to start from scratch with every project Because
  • 39.
  • 40.
    terraform fmt • Rewritesconfiguration files to a canonical style and format terraform validate • Validates syntax of configuration files Lint
  • 41.
    • Store infrastructurestate in the project itself • State locks itself during operations • Makes code portable and enables collaboration • Git all the things Remote State
  • 42.
    • Maintain separatevariables per environment • Use Makefile to interact with terraform params/dev.tfvars Environments Makefile
  • 43.
    • Reusable collectionsof resource definitions and/or data sources • Can accept arguments and provide attributes • Basically just a bunch of .tf files in a directory • Community modules available for deploying common groups of resources or applications tf-gcp-standard-subnetwork • Standard subnetwork • HA NAT instances • Public routes and default NAT routes Modules
  • 44.
    • Terrafile: https://github.com/claranet/python-terrafile Versionmanagement for modules • TfEnv: https://github.com/kamatama41/tfenv Version management for TF itself • Exekube: https://github.com/exekube/exekube Combines Terraform with Helm for complete IaC on Kubernetes • Pipelines! (didn’t have time for this talk, sorry) Worth a Look
  • 45.
    • Terraform’s lifecycleis: init, plan, apply and eventually destroy • Use data sources and variables and be as generic as possible • Split variables into environment-specific parameter files • Use git and remote backends for portability • Use modules to save time Summary
  • 46.