Terraform in deployment pipeline

Terraform in deployment pipeline

• 1.
  Terraform in deploymentpipeline by Anton Babenko
• 2.
  Hi! I am AntonBabenko, and I enjoy: ● AWS & DevOps ● AWS User Group Norway and DevOpsDays Oslo organizer ● Solve problems “Getting Started with Terraform”, terraform-community-modules, Terraform modules generator (Terrapin), and more… https://github.com/antonbabenko https://www.linkedin.com/in/antonbabenko
• 3.
  1. Become morefamiliar with managing infrastructure using CD pipeline 2. See scenarios of integrating Terraform and Packer 3. How to structure infrastructure code? 4. How to version your infrastructure between environments and make it DRY? Goals of this talk
• 4.
  Do you know? Whatis: ● Infrastructure as code? ● Deployment pipeline? ● Pipeline as code?
• 5.
  Featuring... Write, plan, andcreate infrastructure as code Build automated machine images
• 7.
  Typical CI/CD pipeline source:https://dzone.com/articles/what-is-continuous-delivery-pipeline
• 8.
  Where are infrastructurechanges here?
• 9.
  CI/CD pipeline (CircleCI2.0)
• 10.
  Structure - all-in-onevs split ~/all-in-one-repo/ ├── packer # Packer configs │ └── app.json ├── terraform # Terraform configs │ ├── main.tf │ └── terraform.tfvars └── web # Application code └── index.html ~/infra-repo/ ├── packer # Packer configs │ └── app.json └── terraform # Terraform configs ├── main.tf └── terraform.tfvars ~/app-repo/ └── web # Application code └── index.html
• 11.
  Structure - evolvinginfrastructure repository ~/infra-repo/ ├── packer # Packer configs │ └── app.json └── terraform # Terraform configs ├── modules # Terraform modules │ ├── network │ │ └── main.tf │ └── service1 │ └── main.tf ├── main.tf └── terraform.tfvars ~/infra-repo/ ├── packer # Packer configs │ └── app.json └── terraform # Terraform configs ├── modules # Terraform modules │ ├── network │ │ └── main.tf │ └── service1 │ └── main.tf └── environments ├── non-prod │ └── us-east-1 │ ├── main.tf │ └── terraform.tfvars └── prod ├── eu-west-1 │ ├── main.tf │ └── terraform.tfvars └── us-east-1 ├── main.tf └── terraform.tfvars
• 12.
  example1/main.tf resource "random_pet" "bucket"{} resource "aws_s3_bucket" "app" { bucket = "fullstackfest-${ random_pet .bucket. id}" acl = "public-read" website { index_document = "index.html" } } data "template_file" "index" { template = "${file("../../web/index.html")}" vars { BUILD_DETAILS = "${aws_s3_bucket .app.website_endpoint }" } } resource "aws_s3_bucket_object" "object" { bucket = "${aws_s3_bucket .app.id}" key = "index.html" content = "${data. template_file .index.rendered }" etag = "${md5(data. template_file .index.rendered )}" content_type = "text/html" acl = "public-read" } output "app_website_endpoint" { value = "${aws_s3_bucket .app.website_endpoint }" } FullStackFest! ${BUILD_DETAILS} $ terraform init ... $ terraform plan ... $ terraform apply ... Apply complete! Resources: 3 added, 0 changed, 0 destroyed. Outputs: app_website_endpoint = fullstackfest-feasible-basilisk.s3-website-eu-west-1.amazonaws.c om
• 13.
  example2/main.tf variable "subnet_id" { description= "ID of subnet where resources will be created" } variable "security_groups" { description = "ID of security group EC2 instance will use" } variable "instance_type" { description = "Type of EC2 instance to launch" } data "aws_ami" "app" { most_recent = true filter { name = "name" values = ["fullstackfest-demo-*" ] } } resource "aws_instance" "app" { ami = "${data. aws_ami.app.id}" instance_type = "${var.instance_type }" subnet_id = "${var.subnet_id }" vpc_security_group_ids = ["${var.security_groups }"] } output "app_public_ip" { description = "Public IP of EC2 instance running an application" value = "${aws_instance .app.public_ip }" } packer/app.json { "builders" : [ { "ami_name" : "fullstackfest-demo-{{uuid | clean_ami_name}}" , "ami_description" : "FullStackFest demo AMI based on Amazon Linux", "instance_type" : "t2.micro" , "region" : "eu-west-1" , "type": "amazon-ebs" , "ssh_username" : "ec2-user" , "source_ami_filter" : { "filters" : { "virtualization-type" : "hvm", "name": "amzn-ami-hvm-*-x86_64-gp2" , "root-device-type" : "ebs" }, "owners" : [ "137112412989" ], "most_recent" : true } } ], "provisioners" : [ { "type": "shell", "inline" : [ "# Install nginx, copy index.html into web-root" ] } ] }
• 14.
  # Avoid hard-codedvalues in *.tf files, use data sources or *.tfvars data "aws_ami" "app" { most_recent = true filter { name = "name" values = ["fullstackfest-demo-*" ] } } # Tag and name resources consistently resource "aws_instance" "app" { ami = "${data. aws_ami.app.id}" instance_type = "${var.instance_type }" subnet_id = "${var.subnet_id }" vpc_security_group_ids = ["${var.security_groups }"] tags { Name = "fullstackfest-demo-${var. environment }" } } variable "environment" { description = "Name of environment to create infrastructure (eg, staging, production)" } # terraform.tfvars environment = "non-prod" FTP (Frequent Terraform Problems) ● Avoid hard-coded values => use data sources ● Tag and name resources consistently Next: Terraform modules = reusability
• 15.
  module "sg_web" { source= "git@github.com:terraform-community-modules/tf_aws_sg.git//sg_web?ref=v0.2.3" security_group_name = "fullstackfest-demo-web" vpc_id = "vpc-12345678" source_cidr_block = ["0.0.0.0/0" ] } resource "aws_instance" "app" { # ... vpc_security_group_ids = ["${module. sg_web.security_group_id_web }"] # ... } Terraform modules ● Versioning ● Public/private access ● Local dir or hosted ● Allows: ○ code reuse ○ encapsulate groups of resources ○ testing
• 16.
  Demo - infrastructureas code and deployment pipeline https://github.com/antonbabenko/terraform-deployment-pipeline-talk
• 17.
  Further thoughts… ● Uselinters (tflint), coding styles (fmt), pre-commit hooks ● Automate (no excuses) ● Terraform workspaces, Terragrunt, Atlantis by Hootsuite (super!) ● Version & release infrastructure as the app code ● Using pipelines to manage environments with infrastructure as code by Kief Morris
• 18.
  Thank you!