Automating the development pipeline
DevOps Culture
Dev Ops
3
4
Dev
Wall of Confusion
Ops
Development + Operations = DevOps
Dev Ops
5
What is DevOps?
• DevOps is a combination of:
• Cultural philosophies
• Practices
• Tools
6
DevOps Culture
• Dev & Ops coming together
• No more “silos”
• Shared responsibility
• Ownership
• Visibility and communication
People over Process over Tools
7
DevOps Practices
• Microservices
• Moving away from “monolithic” application architecture to many individual services
8
DevOps Practices
• Continuous integration
• Continuous delivery & deployment
9
DevOps Practices
• Infrastructure as code
• Model your AWS resources using code
10
DevOps Practices
• Monitoring and logging
• Track and analyze metrics and logs
• Understand real-time performance of infrastructure and application
11
Benefits of DevOps
Improved Collaboration Rapid Delivery Reliability
Speed
Security Scale
12
Comprehensive set of enterprise-grade tools
Comprehensive set of enterprise-grade tools
CI/CD Tools Infrastructure as Code IDE Monitoring & Tracing Web Apps
AWS Cloud Dev.
Kit (CDK)
IDE and DevOps Toolkits CLI and Scripting Tools Languages Mobile
Visual Studio IntelliJ PyCharm Visual Studio Eclipse VSTS Amazon AWS
Code AWS CLI Tools for Corretto Amplify
PowerShell
SDKs
JavaScript Python PHP .NET Ruby Java Go Node.js C++
13
Cloud9
AWS Cloud9
• Cloud based integrated development environment (IDE)
• Allows write, execute, and debug code in your browser
• Share your environment with your team and collaborate in real
time
• Direct access to the terminal
• Great serverless development experience: Conduct local testing,
included SDK, libraries, plug-ins, etc.
16
Full Editor
17
Wide selection of Run Times
18
Full debug
19
Integrated Tools for Serverless Development
20
AWS CloudFormation
AWS CloudFormation
• Simplified way to create and manage a collection of AWS
resources
• Enables orderly and predictable provisioning and updating of
resources
• Enables version control of your AWS infrastructure
• Deploy and update stacks using the AWS Management Console,
the AWS Command Line Interface (CLI), or the AWS API
• Only pay for the resources you create
22
Overview
A running
environment
Template
AWS Stack
CloudFormation
API calls made on
your behalf
• JSON/YAML format template • FREE – you only pay for resources
• Presents template to AWS CloudFormation • All regions
• AWS CloudFormation translates it to an API • APIs are called in parallel
request • Manages dependencies/relationships
• Forms a stack of resources
23
Infrastructure
Infrastructure as as code
code
Template
• Single source of truth to deploy the whole stack
• Infrastructure that you can replicate, re-deploy, and re-
purpose
• Control versioning on your infrastructure and your
Stack Stack Stack
application together
• Service rolls back to the last good state on failures
• Build your infrastructure and run it through your CI/CD
pipeline
24
Stack creation
• You use a template to create and manage
a stack
• A stack is a collection of AWS resources
that you can manage as a single unit
• AWS CloudFormation ensures all stack
resources are created or deleted as
appropriate
25
Create and manage a stack using the AWS CLI
Install the AWS CLI using installation guide
aws cloudformation create-stack
--stack-name ec2InstanceCmdLineDemo
--template-url https://s3.amazonaws.com/cf-templates-deloitte-
workshop/Demo-1.json
--parameters ParameterKey=KeyPair,ParameterValue=KeyName
Returns the details of the created stack, in the output format of your choice
Returns the details of the created stack, in the output format of your choice
arn:aws:cloudformation:us-east-1:496891363831:stack/t1/625f07c0-1fef-
11e8-a501-50d5ca63261e
26
Template anatomy
{
"AWSTemplateFormatVersion" : "version date",
"Description" : ”Just another JSON String”,
"Metadata" : {
JSON objects that provide additional information about the template.
},
"Parameters" : {
Specifies values that you can pass in to your template at runtime
(when you create or update a stack )
},
1. Format version "Mappings" : {
Specifies values that you can pass in to your template at runtime
2. Transform (new) },
(when you create or update a stack)
3. Description "Conditions" : {
Defines conditions that control whether certain resources are created
4. Metadata or whether certain resource properties are assigned a value during stack creation
or update.
5. Parameters },
"Resources" : {
6. Mappings Specifies the stack resources and their properties, such as an
Amazon Elastic Compute Cloud instance or an Amazon Simple Storage Service
7. Conditions },
bucket.
8. Resources* (required) "Outputs" : {
Describes the values that are returned whenever you view your stack's properties.
9. Outputs }
}
Reference: http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/crpg-ref.html
27
Template anatomy
28
Continuous integration &
Continuous deployment
AWS Code Services
Software Release Steps
Source Build Test Production
30
AWS Code Services
Software Release Steps
Source Build Test Production
AWS CodeCommit
31
AWS Code Services
Software Release Steps
Source Build Test Production
AWS CodeBuild
32
AWS Code Services
Software Release Steps
Source Build Test Production
AWS Third Party
CodeBuild Tooling
33
AWS Code Services
Software Release Steps
Source Build Test Production
AWS CodeDeploy
34
AWS Code Services
Software Release Steps:
Source Build Test Production
AWS CodeCommit AWS Third Party
AWS CodeBuild AWS CodeDeploy
CodeBuild Tooling
AWS CodePipeline
35
AWS CodeCommit
• A secure, managed and scalabe Git repository
• Same standard Git tooling
• Scalability, availability and durability from Amazon S3
• Encryption at rest using customer keys
• Unlimited size repository
• Post-commit triggers can invoke Amazon SNS or Lambda
36
Control source code on the cloud
Safe Managed Highly Store
Available any object
37
AWS CodeCommit
Git objects on
Amazon S3
Git index on
Amazon DynamoDB
SSH or HTTPS
Git pull/push AWS CodeCommit Cryptography keys
on AWS KMS
38
Same Git Experience
$ git clone https://git-codecommit.us-east-1.amazonaws.com/v1/repos/aws-cli
Cloning into 'aws-cli'...
Receiving objects: 100% (16032/16032), 5.55 MiB | 1.25 MiB/s, done.
Resolving deltas: 100% (9900/9900), done.
Checking connectivity... done.
$ nano README.rst
$ git commit -am 'updated README'
[master 4fa0318] updated README
1 file changed, 1 insertion(+)
39
AWS CodeBuild
• Manage build server that compiles source-code, executes tests
and builds software packages
• Continuously scale and process several builds in parallel
• You can create custom build environments using Docker images
• Pay only for what you have used
• Native integration with AWS CodePipeline and Jenkins
40
How it works?
1. Downloads the source code
2. Runs the defined commands from buildspec file using temporary containers ( new
containers are created for each build)
3. Send build logs to the console and Amazon CloudWatch logs
4. Send the built artifacts to a Amazon S3 bucket
41
File Example buildspec.yml
version: 0.1
environment_variables:
plaintext:
JAVA_HOME: "/usr/lib/jvm/java-8-openjdk-amd64"
phases:
install:
commands:
- apt-get update -y
- apt-get install -y maven
pre_build:
commands:
- echo Nothing to do in the pre_build phase...
build:
commands:
- echo Build started on `date`
- mvn install
post_build:
commands:
- echo Build completed on `date`
artifacts:
type: zip
files:
- target/messageUtil-1.0.jar
discard-paths: yes
43
Building your code
• Typically, the Build process refers to the source code developed
using programming languages that needs to be compiled:
• .NET: C#, F#, VB.net, etc.
• Java e JVM: Java, Scala, JRuby
• Go
• iOS: Swift, Objective-C
• We also used the term Build to refer to Docker image creation
EC2
44
And if you don't need to build…
• Several programming languages doesn’t need to be compiled.
They are considered interpreted languages:
• PHP
• Ruby
• Python
• Node.js
• You just need to deploy your source code
EC2
45
AWS CodeDeploy
• Automates code deployments in any instance
• Handles the complexity of updating applications
• Prevents unavailability during deploy
• Performs automatic rollback in case of failure
• Deploy on Amazon EC2 or on-premises servers, using any
language and any operating system
• Integrates with third-party tools and AWS
46
File Example appspec.yml
version: 0.0
os: linux
files:
Send application files to one
- source: /
directory and configuration files to
destination: /var/www/html
another
permissions:
- object: /var/www/html
pattern: “*.html” Set specific permissions on
owner: root directories and files.
group: root
mode: 755
hooks:
ApplicationStop:
- location: scripts/deregister_from_elb.sh
Remove / add instances to Elastic
BeforeInstall:
Load Balancing
- location: scripts/install_dependencies.sh
Install dependency packages
ApplicationStart:
Start Apache
- location: scripts/start_httpd.sh
Confirm deployment success
ValidateService:
- location: scripts/test_site.sh
- location: scripts/register_with_elb.sh
47
Choose speed and deployment group
Dev deployment group
One at a time
v2 v1 v1 v1 v1 v1 Agent Agent
OR
Half at a time Prod deployment group
v2 v2 v2 v1 v1 v1
Agent Agent Agent
All at once
v2 v2 v2 v2 v2 v2 Agent Agent Agent
48
AWS CodePipeline
• Continuous delivery service for fast and reliable application
updates
• Model and visualize your software release process
• Builds, tests, and deploys the code whenever a change occurs
• Integrates with third-party tools and AWS
49
AWS CodePipeline
MyApplication
Source
Source
GitHub
Build
AWS CodeBuild Stage Pipeline
AWS CodeBuild Action
Transition
Deploy
JavaApp
AWS Elastic
Beanstalk
AWS CodePipeline
MyApplication
Source
Source
GitHub
Build
AWS CodeBuild Notify Developers
AWS CodeBuild AWS Lambda
Parallel actions
Deploy
JavaApp
Elastic Beanstalk
AWS CodePipeline
MyApplication
Source
Source
GitHub
Build
AWS CodeBuild NotifyDevelopers
AWS CodeBuild Lambda
Sequential actions
Test API
Runscope
Deploy
JavaApp
Elastic Beanstalk
AWS CodePipeline
MyApplication
Build
AWS CodeBuild
AWS CodeBuild
Staging-Deploy
JavaApp
Elastic Beanstalk
QATeamReview
Manual Approval
Manual approvals
Review
Prod-Deploy
JavaApp
Elastic Beanstalk
AWS Elastic Beanstalk
AWS Elastic Beanstalk
The easiest way to get your web applications up and running in minutes
Elastic Beanstalk is a managed service that automatically handles
• Infrastructure provisioning and configuration
• Deployment
• Load balancing
• Auto scaling Provision / Manage /
Deploy
Configure Monitor
• Health monitoring
• Analysis and debugging
• Logging
There is no additional charge for Elastic Beanstalk
55
AWS Elastic Beanstalk Supports
Platforms containing the most popular
runtimes as well as Docker images, allowing
you to bring your own
Elastic Beanstalk also supports custom
platforms and custom images
56
How Elastic Beanstalk works
Your code
HTTP Server
Elastic Beanstalk
Application Server
• Bundled your source code together
with the corresponding platform (OS, Language Interpreter
runtime, application server, and web
server) Operating System
• Safely deployed your web application
Host
57
AWS services are plug and play
You can adopt a systematic approach to CI/CD tools and methodologies while leveraging
existing investments
1 2 3 4
AWS
GitHub Jenkins CodeBuild
repository Pipeline
58
Challenge: Extending to CI/CD
Or you can create robust but simple CI/CD across release stages (dev, staging, production)
with increasingly safer deployment status, deployment failure reporting, and ongoing health
monitoring using AWS services.
1 2 3 4
AWS AWS AWS
CodeCommit CodePipeline CodeBuild
59
Questions?
Lab 2
• Go to https://aws.qwiklabs.com
• Login with your account.
• If you don’t have an account, register and then activate the account using the email that you
will receive.
• Start the Lab 2, and read the instructions.
• If you have doubts about the activity, please ask the trainer to receive help.
• You have 45 minutes to complete this lab.
61
