KEMBAR78
Chef - industrialize and automate your infrastructure | PDF
1
CHEF
INDUSTRIALIZE & AUTOMATE YOUR INFRASTRUCTURE
Created by Michael Lopez - © Maisons du Monde
2
TO DO WHAT ?
Con gure a machine in minutes without action
Deploy software or application
Ensure that all your machines are identical
Gain time !
3 . 1
HOW DOES IT WORKS ?
HOW DOES IT WORKS ?
3 . 24
CHEF VS PUPPET/ANSIBLE
PUPPET
Language : DSL Ruby/Json
Approach : Execution by dependency and
chained action
Stored data : YAML
Agent : Yes
ANSIBLE
Language : Python
Approach : Hierarchical Execution
Stored data : YAML
Agent : No
CHEF
Language : Full Ruby & DSL
Approach : Hierarchical Execution
Stored data : JSON
Agent : Yes
PUPPET
package { 'openssh-server':
ensure => present,
before => File['/etc/ssh/sshd_config'],
}
file { '/etc/ssh/sshd_config':
ensure => file,
mode => 600,
source => 'puppet:///modules/sshd/sshd_config',
require => Package['openssh-server'],
}
service { 'sshd':
ensure => running,
enable => true,
subscribe => File['/etc/ssh/sshd_config'],
}
5 . 15 . 2
PUPPET
Chained Actions
package { 'openssh-server':
ensure => present,
} -> # and then:
file { '/etc/ssh/sshd_config':
ensure => file,
mode => 600,
source => 'puppet:///modules/sshd/sshd_config',
} ~> # and then:
service { 'sshd':
ensure => running,
enable => true,
}
5 . 3
PUPPET
Virtual Resources
@user {'deploy':
uid => 2004,
comment => 'Deployment User',
group => www-data,
groups => ["enterprise"],
tag => [deploy, web],
}
realize User['deploy'], User['zleslie']
User <| tag == web |>
6 . 1
ANSIBLE
Playbook
---
# This role installs HAProxy and configures it.
- name: Download and install haproxy and socat
apt: name={{ item }} state=present
with_items:
- haproxy
- socat
- name: Configure the haproxy cnf file with hosts
template: src=haproxy.cfg.j2 dest=/etc/haproxy/haproxy.cfg
notify: restart haproxy
- name: Start the haproxy service
service: name=haproxy state=started enabled=yes
6 . 2
ANSIBLE
- apt: name=foo update_cache=yes
- apt: name=foo state=absent
- apt: name=foo=1.00 state=present
- apt: deb=/tmp/mypackage.deb
- file: path=/etc/foo.conf owner=foo group=foo mode=0644
- file: src=/tmp/{{ item.path }} dest={{ item.dest }} state=link
with_items:
- { path: 'x', dest: 'y' }
- { path: 'z', dest: 'k' }
- file: path=/etc/foo.conf state=touch mode="u=rw,g=r,o=r"
7
CHEF VS PUPPET/ANSIBLE
Advantages Disadvantages
Chef Fast & powerful
Dev Oriented => Flexible
Full Ruby
Search
Crypted data
Flexibility -> Complexity
Puppet Mature
Large Community
Lot of tools
Slow
Complex Language
Complex execution order
Ansible Agentless
« No code »
Multilingual plugins
Immaturity: Small community and few tools
Data stored in les
8 . 1
CHEF SERVER
Free/Basic Version
Free & no node limit
In your Infrastructure
› No access to "premium" features
8 . 2
CHEF SERVER
8 . 3
CHEF AUTOMATE
Deliver a continuous deployment pipeline for infrastructure and applications.
Gain insight into operational, compliance, and work ow events.
Identify compliance issues, security risks, and outdated software with customizable
reports
9
NOT ONLY CHEF SERVER...
Chef Solo
No Server
Deploy & Run recipes directly on the node
› No search & dedicated attributes on a node
› Need to push cookbooks on each nodes
› Not compatible with version constraint
Chef Zero
Chef Server instance in memory
Chef Local mode (-z)
embedded chef-zero (faster)
10
INSTALLATION MODES
Package
Gem (if ruby already exists on the system)
Bootstrap (client)
11
KNIFE – CHEF'S SWISS KNIFE
Management of chef environment
Search, SSH (Executing commands in parallel)
Plugins (VMware, solo, spork...)
12
CLIENTS AND NODES
A client is a registered machine with the server
A Node is a client that executes one or more recipe
→ A node is a client but a client is not necessarily a node
13
CHEF REPOSITORY
› git clone git://github.com/chef/chef-repo.git && cd chef-repo
14
THE COOKBOOKS – WHAT WE WILL COOK TODAY ?
Create a cookbook
Structure
Metadata
Recipes
Attributes
Files & Templates
Resources & Providers
LWRP & HWRP
De nitions
Library
15
THE COOKBOOKS - CREATE
Three options to create a cookbook:
Create your own
Get a cookbook from Chef Supermarket
Get a cookbook from Github (or other)
knife cookbook create nginx
chef generate cookbook nginx
knife cookbook site install nginx
16
THE COOKBOOKS - STRUCTURE
$ tree cookbooks/mdm/
cookbooks/mdm/
├── CHANGELOG.md
├── README.md
├── attributes
├── definitions
├── files
│   └── default
├── libraries
├── metadata.rb
├── providers
├── recipes
│   └── default.rb
├── resources
└── templates
└── default
17
THE COOKBOOKS - METADATA
18
THE COOKBOOKS - RECIPES
› cookbook/elasticsearch/recipes/default.rb
include_recipe 'java'
include_recipe 'elasticsearch::repo'
include_recipe 'elasticsearch::sysctl'
package 'elasticsearch'
template '/etc/default/elasticsearch' do
source 'elasticsearch.default.erb'
owner 'root'
group 'root'
mode 0644
notifies :restart, 'service[elasticsearch]', :delayed
end
template node[:elasticsearch][:conf_file] do
source 'elasticsearch.yml.erb'
owner 'root'
group 'root'
mode 0644
notifies :restart, 'service[elasticsearch]', :delayed
end
include_recipe 'elasticsearch::plugins'
service 'elasticsearch' do
supports :status => true, :restart => true
action [:enable, :start]
end
› cookbook/elasticsearch/recipes/repo.rb apt_repository "elasticsearch" do
uri "http://packages.elasticsearch.org/elasticsearch/1.4/debian"
distribution "stable"
components ['main']
arch "amd64"
key "https://packages.elasticsearch.org/GPG-KEY-elasticsearch"
action :add
end
19 . 1
THE COOKBOOKS - ATTRIBUTES
› memcache/attributes/default.rb
default['memcached']['memory'] = 64
default['memcached']['port'] = 11211
default['memcached']['udp_port'] = 11211
default['memcached']['listen'] = '0.0.0.0'
default['memcached']['maxconn'] = 1024
default['memcached']['max_object_size'] = '1m'
default['memcached']['logfilename'] = 'memcached.log'
case node['platform_family']
when 'ubuntu'
default['memcached']['user'] = 'memcache'
default['memcached']['group'] = 'memcache'
when 'debian'
default['memcached']['user'] = 'nobody'
default['memcached']['group'] = 'nogroup'
else
default['memcached']['user'] = 'nobody'
default['memcached']['user'] = 'nogroup'
end
19 . 2
THE COOKBOOKS - ATTRIBUTES
case node['platform']
when "debian"
case
when node['platform_version'].to_f < 6.0 # All 5.X
default['postgresql']['version'] = "8.3"
when node['platform_version'].to_f < 7.0 # All 6.X
default['postgresql']['version'] = "8.4"
else
default['postgresql']['version'] = "9.1"
end
default['postgresql']['client']['packages'] = ["postgresql-client-#{node['postgresql']['version']}","libpq-dev"]
default['postgresql']['server']['packages'] = ["postgresql-#{node['postgresql']['version']}"]
default['postgresql']['contrib']['packages'] = ["postgresql-contrib-#{node['postgresql']['version']}"]
when "fedora"
...
19 . 3
THE COOKBOOKS - AUTOMATIC ATTRIBUTES
19 . 4
THE COOKBOOKS - ATTRIBUTES
20 . 1
THE COOKBOOKS - TEMPLATES
template '/etc/memcached.conf' do
source 'memcached.conf.erb'
owner 'root'
group 'root'
mode '0644'
variables(
:listen => node['memcached']['listen'],
:logfilename => node['memcached']['logfilename'],
:user => node['memcached']['user'],
:port => node['memcached']['port'],
:udp_port => node['memcached']['udp_port'],
:maxconn => node['memcached']['maxconn'],
:memory => node['memcached']['memory'],
:max_object_size => node['memcached']['max_object_size']
)
notifies :restart, 'service[memcached]'
end
20 . 2
THE COOKBOOKS - TEMPLATES
› memcached/templates/default/memcached.conf.erb
# Run memcached as a daemon. This command is implied, and is not needed for the
# daemon to run. See the README.Debian that comes with this package for more
# information.
-d
# Log memcached's output to /var/log/memcached
logfile /var/log/<%= @logfilename %>
# Be verbose
#-v
# Be even more verbose (print client commands as well)
# -vv
# Start with a cap of 64 megs of memory. It's reasonable, and the daemon default
# Note that the daemon will grow to this size, but does not start out holding this much
# memory
-m <%= @memory %>
# Default connection port is 11211
-p <%= @port %>
#-U <%= @udp_port %>
# Run the daemon as root. The start-memcached will default to running as root if no
# -u command is present in this config file
-u <%= @user %>
# Specify which IP address to listen on. The default is to listen on all IP addresses
# This parameter is one of the only security measures that memcached has, so make sure
21 . 1
THE COOKBOOKS - FILES
› cookbook/app/ les/default/application-5.8.3.pm
› cookbook/app/ les/default/application-5.20.1.pm
cookbook_file "application.pm" do
path case node['platform']
when "centos","redhat"
"/usr/lib/version/1.2.3/dir/application.pm"
when "arch"
"/usr/share/version/core_version/dir/application.pm"
else
"/etc/version/dir/application.pm"
end
source "application-#{node['languages']['perl']['version']}.pm"
owner 'root'
group 'root'
mode '0644'
end
21 . 2
THE COOKBOOKS - FILES
1. /host-$fqdn/$source
2. /$platform-$platform_version/$source
3. /$platform/$source
4. /default/$source
5. /$source
host-foo.example.com/apache2_module_conf_generate.pl
ubuntu-10.04/apache2_module_conf_generate.pl
ubuntu-10/apache2_module_conf_generate.pl
ubuntu/apache2_module_conf_generate.pl
default/apache2_module_conf_generate.pl
22
THE COOKBOOKS – RESOURCES & PROVIDERS (LWRP / HWRP)
$ tree cookbooks/mdm/
cookbooks/mdm/
├── CHANGELOG.md
├── README.md
├── attributes
├── definitions
├── files
│   └── default
├── libraries
├── metadata.rb
├── providers
├── recipes
│  └── default.rb
├── resources
└── templates
└── default
Resources : Used to de ne a set of actions and attributes
Providers : Used to say to chef-client what to do foreach
de ned actions
user 'random' do
supports :manage_home => true
comment 'Random User'
uid 1234
gid 'users'
home '/home/random'
shell '/bin/bash'
password '$1$JJsvHslV$szsCjVEroftprNn4JHtDi'
end
23 . 1
THE COOKBOOKS – RESOURCES & PROVIDERS (LWRP)
› cookbooks/nginx/resources/app_site.rb
actions :create, :remove, :enable, :disable
default_action :create
attribute :app_name, :kind_of => String, :name_attribute => true, :required => true
attribute :cookbook, :kind_of => String, :default => "nginx"
attribute :template, :kind_of => String, :default => "reverse"
attribute :server_name, :kind_of => String
attribute :proxy_pass_url, :kind_of => String
attr_accessor :exists
attr_accessor :enabled
23 . 2
THE COOKBOOKS – RESOURCES & PROVIDERS (LWRP)
› cookbooks/nginx/providers/app_site.rb
23 . 3
THE COOKBOOKS – RESOURCES & PROVIDERS (LWRP)
23 . 4
THE COOKBOOKS – RESOURCES & PROVIDERS (LWRP)
› cookbooks/stash/recipes/nginx.rb
node.default['nginx']['default_site_enabled'] = false
include_recipe 'nginx'
nginx_app_site "stash" do
server_name node[:stash][:http_server][:virtual_host_name]
proxy_pass_url "http://localhost:#{node[:stash][:tomcat][:port]}"
cookbook "nginx"
template "reverse"
action [:create, :enable]
notifies :reload, 'service[nginx]', :delayed
end
nginx_app_site "stash-ssl" do
server_name node[:stash][:http_server][:virtual_host_name]
proxy_pass_url "http://localhost:#{node[:stash][:tomcat][:ssl_port]}"
cookbook "nginx"
template "reverse_ssl"
action [:create, :enable]
notifies :reload, 'service[nginx]', :delayed
end
24
THE COOKBOOKS – RESOURCES & PROVIDERS
(HWRP)
Create a resource with pure Ruby
Bypass the limits of Chef DSL
http://tech.yipit.com/2013/05/09/advanced-chef-writing-
heavy-weight-resource-providers-hwrp/
25 . 1
THE COOKBOOKS - DEFINITIONS
Used to de ne a set of actions with or without parameters
(function)
You can call your de nition many times in one or more
recipes
This is the same as a resource (LWRP) except that you may
not notify (trigger) other resources
25 . 2
THE COOKBOOKS - DEFINITIONS
define :host_porter, :port => 4000, :hostname => nil do
params[:hostname] ||= params[:name]
directory "/etc/#{params[:hostname]}" do
recursive true
end
file "/etc/#{params[:hostname]}/#{params[:port]}" do
content "some content"
end
end
host_porter node['hostname'] do
port 4000
end
host_porter "www1" do
port 4001
end
26 . 1
CUSTOM RESOURCES
Providers (LWRP/HWRP)
De nitions
DEPRECATED
26 . 2
CUSTOM RESOURCES
From Chef version 12.5
De nitions are useless. Advise you to use a resources instead.
Custom resources is a provider redesigned to be simpler.
Located only in "resources" directory
26 . 3
CUSTOM RESOURCES
exampleco/resources/site.rb
property :homepage, String, default: '<h1>Hello world!</h1>'
load_current_value do
if ::File.exist?('/var/www/html/index.html')
homepage IO.read('/var/www/html/index.html')
end
end
action :create do
package 'httpd'
service 'httpd' do
action [:enable, :start]
end
file '/var/www/html/index.html' do
content homepage
end
end
action :delete do
package 'httpd' do
action :delete
end
end
exampleco_site 'httpd' do
homepage '<h1>Welcome to the Example Co. website!</h1>'
action :create
end
27 . 1
THE COOKBOOKS - LIBRARIES
Allows to extend the Chef's classes or create your own Ruby lib
Do what Chef does not already do
Do external data processing to use in Chef
27 . 2
THE COOKBOOKS - LIBRARIES
27 . 3
THE COOKBOOKS - LIBRARIES
28
ROLES
Used to group one or more roles, attributes and recipes
{
"name": "ns",
"description": "The DNS Role",
"json_class": "Chef::Role",
"default_attributes": {
"bind": {
"dnssec-validation": "no",
"zone-statistics": "yes",
"zones": [ "company.local", "0.168.192.in-addr.arpa" ],
"acls": {
"slaves": [ "192.168.0.2", "192.168.0.3" ],
"mdm_network": [ "192.168.0.0/24" ]
},
"allow_query": ["localhost", "company_network"],
"allow_recursion": ["localhost", "company_network"],
"allow_query_cache": ["localhost", "company_network"],
"also_notify": true,
}
},
"override_attributes": {
"bind": {
"masters": ["192.168.0.1"],
"allow_transfer": ["slaves"]
},
"ntp": {
"is_server": true,
"servers": ["0.fr.pool.ntp.org", "1.fr.pool.ntp.org", "2.fr.pool.ntp.org", "3.fr.pool.ntp.org"],
"restrictions": ["192.168.0.0 mask 255.255.255.0 nomodify notrap"]
29
ENVIRONNEMENTS
{
"name": "staging",
"description": "The Staging environment",
"cookbook_versions": {
"box": "= 0.3.8",
"nginx": "= 0.5.3",
"web": "= 1.4.29",
"base": "= 0.3.4",
"tools": "= 0.4.2",
"users": "= 1.7.5",
"db": "= 0.3.5",
"ambari": "= 0.7.1"
},
"json_class": "Chef::Environment",
"chef_type": "environment",
"default_attributes": {
"php-fpm": {
"session": {
"save_handler_type": "memcache",
"save_handler_path": "tcp://localhost:11211"
},
"log_level": "alert",
"emergency_restart_threshold": "10",
"emergency_restart_interval": "1m",
"process_control_timeout": "10s"
},
"nginx": {
"keepalive_timeout": "40s",
"client_body_timeout": "20s",
"client_header_timeout": "20s",
"server_tokens": "off",
"disable_access_log": "true",
"mdm_log_format": "true",
"fastcgi_buffers": "8 16k",
"fastcgi_buffer_size": "32k",
Sets behaviors or different attributes for a given environment
include_recipe 'sshd'
include_recipe 'sudo'
include_recipe 'users::system'
include_recipe 'users::admins'
include_recipe 'users::operators'
if node.chef_environment != 'prod'
include_recipe 'users::qa'
include_recipe 'users::devops'
end
case node[:platform_family]
when 'debian'
include_recipe 'apt'
when 'rhel', 'fedora'
include_recipe 'yum'
include_recipe 'yum-epel'
include_recipe 'selinux::disabled'
include_recipe 'iptables::disabled'
end
include_recipe 'timezone-ii'
include_recipe 'ntp'
include_recipe 'postfix'
include_recipe 'snmp'
include_recipe 'locales'
include_recipe 'line'
POLICYFILE
Policies are a new feature of Chef that combine the very best
parts of Roles, Environments and cookbook dependency
resolvers (Berkshelf) into a single easy to use work ow.
It is associated with a group of nodes, cookbooks, and
settings. When these nodes run, they run the recipes
speci ed in the Policy le run-list
Resolves real-world problems of team work ow
30 . 130 . 2
POLICYFILE
De ne in Policy le:
› Run list
› Cookbook dependencies with version constraint and source
› Attributes overriding
name "jenkins-master"
run_list "java", "jenkins::master", "recipe[policyfile_demo]"
default_source :supermarket, "https://mysupermarket.example"
cookbook "policyfile_demo", path: "cookbooks/policyfile_demo"
cookbook "jenkins", "~> 2.1"
cookbook "mysql", github: "chef-cookbooks/mysql", branch: "master"
default['java']['version'] = '8'
30 . 3
POLICYFILE
Create your policy le into your cookbook
$ cd cookbooks/app
$ chef generate policyfile
$ ls -l
-rw-r--r-- 1 mlopez wheel 596 12 oct 21:57 Policyfile.rb
Create your policy le into policies directory
$ ls -l
total 24
-rw-r--r-- 1 mlopez wheel 70 12 oct 22:16 LICENSE
-rw-r--r-- 1 mlopez wheel 1546 12 oct 22:16 README.md
-rw-r--r-- 1 mlopez wheel 1067 12 oct 22:16 chefignore
drwxr-xr-x 4 mlopez wheel 136 12 oct 22:16 cookbooks
drwxr-xr-x 4 mlopez wheel 136 12 oct 22:16 data_bags
drwxr-xr-x 4 mlopez wheel 136 12 oct 23:13 policies
$ chef generate policyfile policies/web
30 . 4
Create the lock le
$ chef install [path/to/policyfile.rb]
Update lock le after modi cation
$ chef update [path/to/policyfile.rb]
Upload cookbook and policy le
$ chef push POLICY_GROUP PATH/TO/POLICYFILE.rb
→ If POLICY_GROUP doesn't exists it will be created
Assign policy name and policy group into client.rb's node or in
node con guration
31 . 1
DATA BAGS
Stock & share data
Read the desired data from a recipe or from your desktop (knife)
Write data collected during a run
$ knife data bag create DATA_BAG_NAME [ITEM]
knife data bag create users toto
knife data bag from file users data_bags/users/toto.json
31 . 2
DATA BAGS
› knife data bag create users myUser
{
"id": "myUser",
"ssh_keys": [
"ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDakMx4mjWYMko2r085yq/vq0Ey2DiVWXeJ
],
"groups": [
"mdm_qa"
],
"uid": 4001,
"shell": "/bin/bash",
"comment": "My User",
"action": "remove"
}
› knife data bag show users myUser –Fj > data_bags/users/myUser.json
31 . 3
DATA BAGS
admins = []
search(:admins, "*:*").each do |admin|
login = admin[‘id’]
admins << login
home = "/home/#{login}"
user login do
uid admin['uid']
gid admin['gid']
shell admin['shell']
home home
comment admin['comment’]
supports :manage_home => true
end
end
31 . 4
Write in a data bag
DATA BAGS
sam = {
"id" => "sam",
"Full Name" => "Sammy",
"shell" => "/bin/zsh"
}
databag_item = Chef::DataBagItem.new
databag_item.data_bag("users")
databag_item.raw_data = sam
databag_item.save
sam = data_bag_item("users", "sam")
sam["Full Name"] = "Samantha"
sam.save
31 . 5
What should I do if I have sensitive data ?
DATA BAGS
→ Encrypt !!
$ knife data bag create passwords postgresql –secret-file <path>/encrypted_data_bag_secret</path>
$ knife data bag show passwords postgresql -Fj
{
"id": "postgresql",
"password": {
"encrypted_data": "nu0GFIaJuzefK1iCgmYxWbRO64tvEezZJA/7iOUT87NLg=n",
"iv": "LWK$u1omaWHHNfzfDcYN45g==n",
"version": 1,
"cipher": "aes-256-cbc"
},
"databases": {
"encrypted_data": "lG4EULs9UQKKwjfzef8/WrccoGilQO2m7O6JNnIeMu199jGIT2l+/MvR+bX6dnk2U/_dwn
"iv": "UmtXyR9m0ornADWbiayPyw==n",
"version": 1,
"cipher": "aes-256-cbc"
}
}
32 . 1
CHEF VAULT
Whithout Chef Vault, nodes share a shared secret key le
→ Not good for security
With Chef Vault, nodes and workstation use their own
keypair to decrypt data bag. Chef administrators de ne
which node or admin can access to the encrypted data
32 . 2
or
CHEF VAULT
gem install chef-vault
Create a Vault
Allow mlopez, vaubert and nodes with web role assigned to decrypt content
$ knife vault create credentials database -A mlopez,vaubert -M client -S ‘roles:web' 
-J '{“db_password”:”some_password”}'
$ knife encrypt create credentials database --json '{“db_password”:”some_password”}' 
--search 'role:web' 
--admins mlopez, vaubert --mode client
32 . 3
or
CHEF VAULT
From an Unauthorized admin's workstation
$ knife vault show credentials database
db_password:
cipher: aes-256-cbc
encrypted_data: dsiBtADAV8Sbis89yKuYBvbdNXPpu8bQfJrS20op7zoysfR8roFlzpVHyoaG2
4yb3
iv: +0siNLzFHHqEkP07k6JhYw==
version: 1
id: database
Authorized users see the decrypted content
$ knife vault show credentials database
$ knife decrypt credentials database --mode client
db_password: some_password
id: database
32 . 4
CHEF VAULT
Content of database_keys
$ knife data bag show credentials database_keys
admins:
mlopez
vaubert
clients:
web-01
web-02
id: database_test_keys
mlopez: SOME KEY
vaubert: SOME KEY
web-01: SOME KEY
web-02: SOME KEY
32 . 5
CHEF VAULT
Add a new admin workstation
$ knife vault update credentials database -A mhue
Rotate all keys
$ knife vault rotate all keys
32 . 6
CHEF VAULT
Use Vault in recipe
chef_gem 'chef-vault' do
compile_time true if respond_to?(:compile_time)
end
require 'chef-vault'
# Or just include chef_vault coobkook
include_recipe 'chef-vault'
case ChefVault::Item.data_bag_item_type('credentials', 'database')
when :normal
...
when :encrypted
...
when :vault
...
end
33 . 1
OHAI
Ohai is a tool that is used to detect attributes on a node, inventory the
system (platform, cpu, memory...), and then provide automatically these
attributes to the chef-client at the start of every chef-client run.
Check Automatic attributes previously seen. You can list node attributes
by running "ohai" command
$ ohai
{
"hostname": "node-01",
"machinename": "node-01",
"fqdn": "node-01.my.local",
"domain": "my.local",
"network": {
"interfaces": {
"lo": {
"encapsulation": "Loopback",
"addresses": {
"127.0.0.1": {
"family": "inet",
"prefixlen": "8",
"netmask": "255.0.0.0",
...
33 . 2
OHAI - CUSTOM PLUGIN
You can create your own Ohai plugin to collect data before
the run and set them into Ohai as an attribute
Use Ohai cookbook and put your plugin into ' les' directory
› cookbooks/ohai/ les/default/plugins/haproxy.rb
# Encoding: utf-8
# Get current version of Haproxy
Ohai.plugin(:Haproxy) do
provides 'haproxy'
collect_data(:linux) do
haproxy Mash.new
[['dpkg-query -W haproxy | awk '{print $2}' | sed 's/(^[1-9].[1-9]).*/1/'',
:installed_version]].each do |cmd, property|
so = shell_out(cmd)
haproxy[property] = so.stdout.delete("n")
end
end
end
34
TEAM WORKING
Git -> Create branches
Test locally before uploading his cookbook
Vagrant / Virtualbox, Kitchen CI...etc.
Chef solo/zero/local
Bump version of your cookbook and upload it into environment
Freeze uploaded version and apply version constraint into environnements
Knife Spork – Plugin allow you to bump, upload and promote a cookbook more easily.
Noti cation plugin, auto git add …
$ knife cookbook upload nginx [--freeze] [--force] [-E <environment>]
$ knife spork omni nginx –l minor –e qa_group</environment>
35 . 1
TEST HIS JOB
Syntax
Logical tests with Foodcritic
Obsolescence of resources used, invalid search queries, syntax best-
practice…
Unit tests with ChefSpec
$ knife cookbook test nginx
package 'foo'
require 'chefspec'
describe 'example::default' do
let(:chef_run) { ChefSpec::SoloRunner.converge(described_recipe) }
it 'installs foo' do
expect(chef_run).to install_package('foo')
end
end
35 . 2
TEST HIS JOB
Integration tests with Kitchen CI
Check if a run runs without errors (converge)
Include unit tests (Rspec, Bats…)
Run many tests suite in one time (client / server) on many platforms
Check which nodes use your cookbook
Simulate an execution of a run
knife preflight web::ws
knife search node -i "recipes:web::ws »
chef-client --why-run
knife ssh ‘name:srv-01.pp’ ‘sudo chef-client –W’
36
DEBUG – WHY IT DOESN'T WORK ?
chef-client –l debug
Generate logs
Chef::Log.debug(« Doesn’t work »)
puts myVariable
Raise Exceptions
Chef::Log.fatal!('Deployment failure...')
raise
37 . 1
ADVANCED
Override a run-list (-o "recipe[]")
Override a community Cookbook
knife ssh 'name:srv-01.dev' 'sudo chef-client –o "recipe[firefox]"'
include_recipe 'nginx'
resources("template[/etc/nginx/nginx.conf]").cookbook 'myNginx')
37 . 2
ADVANCED
Noti cations by chef-handler
Knife diff (/*)
38
SOURCES & BOOKS
Chef Starter
Chef Infrastructure Automation Cookbook
https://docs.chef.io
http://tech.yipit.com/2013/05/09/advanced-chef-writing-heavy-
weight-resource-providers-hwrp/
http://dougireton.com/blog/2013/02/03/knife-tricks
http://www.pburkholder.com/blog/2015/04/23/emulating-
cookbook-semver-build-numbers-with-chef-policy les/
https://yolover.poise.io/
SOURCES & BOOKS
› Chef Vault
https://github.com/chef/chef-vault
https://blog.chef.io/2016/01/21/chef-vault-what-is-it-and-what-can-it-do-for-you/
https://blog.chef.io/2013/09/19/managing-secrets-with-chef-vault/
http://hedge-ops.com/chef-vault-tutorial/

Chef - industrialize and automate your infrastructure

  • 1.
    1 CHEF INDUSTRIALIZE & AUTOMATEYOUR INFRASTRUCTURE Created by Michael Lopez - © Maisons du Monde
  • 2.
    2 TO DO WHAT? Con gure a machine in minutes without action Deploy software or application Ensure that all your machines are identical Gain time !
  • 3.
    3 . 1 HOWDOES IT WORKS ?
  • 4.
    HOW DOES ITWORKS ?
  • 5.
    3 . 24 CHEFVS PUPPET/ANSIBLE PUPPET Language : DSL Ruby/Json Approach : Execution by dependency and chained action Stored data : YAML Agent : Yes ANSIBLE Language : Python Approach : Hierarchical Execution Stored data : YAML Agent : No CHEF Language : Full Ruby & DSL Approach : Hierarchical Execution Stored data : JSON Agent : Yes
  • 6.
    PUPPET package { 'openssh-server': ensure=> present, before => File['/etc/ssh/sshd_config'], } file { '/etc/ssh/sshd_config': ensure => file, mode => 600, source => 'puppet:///modules/sshd/sshd_config', require => Package['openssh-server'], } service { 'sshd': ensure => running, enable => true, subscribe => File['/etc/ssh/sshd_config'], }
  • 7.
    5 . 15. 2 PUPPET Chained Actions package { 'openssh-server': ensure => present, } -> # and then: file { '/etc/ssh/sshd_config': ensure => file, mode => 600, source => 'puppet:///modules/sshd/sshd_config', } ~> # and then: service { 'sshd': ensure => running, enable => true, }
  • 8.
    5 . 3 PUPPET VirtualResources @user {'deploy': uid => 2004, comment => 'Deployment User', group => www-data, groups => ["enterprise"], tag => [deploy, web], } realize User['deploy'], User['zleslie'] User <| tag == web |>
  • 9.
    6 . 1 ANSIBLE Playbook --- #This role installs HAProxy and configures it. - name: Download and install haproxy and socat apt: name={{ item }} state=present with_items: - haproxy - socat - name: Configure the haproxy cnf file with hosts template: src=haproxy.cfg.j2 dest=/etc/haproxy/haproxy.cfg notify: restart haproxy - name: Start the haproxy service service: name=haproxy state=started enabled=yes
  • 10.
    6 . 2 ANSIBLE -apt: name=foo update_cache=yes - apt: name=foo state=absent - apt: name=foo=1.00 state=present - apt: deb=/tmp/mypackage.deb - file: path=/etc/foo.conf owner=foo group=foo mode=0644 - file: src=/tmp/{{ item.path }} dest={{ item.dest }} state=link with_items: - { path: 'x', dest: 'y' } - { path: 'z', dest: 'k' } - file: path=/etc/foo.conf state=touch mode="u=rw,g=r,o=r"
  • 11.
    7 CHEF VS PUPPET/ANSIBLE AdvantagesDisadvantages Chef Fast & powerful Dev Oriented => Flexible Full Ruby Search Crypted data Flexibility -> Complexity Puppet Mature Large Community Lot of tools Slow Complex Language Complex execution order Ansible Agentless « No code » Multilingual plugins Immaturity: Small community and few tools Data stored in les
  • 12.
    8 . 1 CHEFSERVER Free/Basic Version Free & no node limit In your Infrastructure › No access to "premium" features
  • 13.
    8 . 2 CHEFSERVER
  • 14.
    8 . 3 CHEFAUTOMATE Deliver a continuous deployment pipeline for infrastructure and applications. Gain insight into operational, compliance, and work ow events. Identify compliance issues, security risks, and outdated software with customizable reports
  • 15.
    9 NOT ONLY CHEFSERVER... Chef Solo No Server Deploy & Run recipes directly on the node › No search & dedicated attributes on a node › Need to push cookbooks on each nodes › Not compatible with version constraint Chef Zero Chef Server instance in memory Chef Local mode (-z) embedded chef-zero (faster)
  • 16.
    10 INSTALLATION MODES Package Gem (ifruby already exists on the system) Bootstrap (client)
  • 17.
    11 KNIFE – CHEF'SSWISS KNIFE Management of chef environment Search, SSH (Executing commands in parallel) Plugins (VMware, solo, spork...)
  • 18.
    12 CLIENTS AND NODES Aclient is a registered machine with the server A Node is a client that executes one or more recipe → A node is a client but a client is not necessarily a node
  • 19.
    13 CHEF REPOSITORY › gitclone git://github.com/chef/chef-repo.git && cd chef-repo
  • 20.
    14 THE COOKBOOKS –WHAT WE WILL COOK TODAY ? Create a cookbook Structure Metadata Recipes Attributes Files & Templates Resources & Providers LWRP & HWRP De nitions Library
  • 21.
    15 THE COOKBOOKS -CREATE Three options to create a cookbook: Create your own Get a cookbook from Chef Supermarket Get a cookbook from Github (or other) knife cookbook create nginx chef generate cookbook nginx knife cookbook site install nginx
  • 22.
    16 THE COOKBOOKS -STRUCTURE $ tree cookbooks/mdm/ cookbooks/mdm/ ├── CHANGELOG.md ├── README.md ├── attributes ├── definitions ├── files │   └── default ├── libraries ├── metadata.rb ├── providers ├── recipes │   └── default.rb ├── resources └── templates └── default
  • 23.
  • 24.
    18 THE COOKBOOKS -RECIPES › cookbook/elasticsearch/recipes/default.rb include_recipe 'java' include_recipe 'elasticsearch::repo' include_recipe 'elasticsearch::sysctl' package 'elasticsearch' template '/etc/default/elasticsearch' do source 'elasticsearch.default.erb' owner 'root' group 'root' mode 0644 notifies :restart, 'service[elasticsearch]', :delayed end template node[:elasticsearch][:conf_file] do source 'elasticsearch.yml.erb' owner 'root' group 'root' mode 0644 notifies :restart, 'service[elasticsearch]', :delayed end include_recipe 'elasticsearch::plugins' service 'elasticsearch' do supports :status => true, :restart => true action [:enable, :start] end › cookbook/elasticsearch/recipes/repo.rb apt_repository "elasticsearch" do uri "http://packages.elasticsearch.org/elasticsearch/1.4/debian" distribution "stable" components ['main'] arch "amd64" key "https://packages.elasticsearch.org/GPG-KEY-elasticsearch" action :add end
  • 25.
    19 . 1 THECOOKBOOKS - ATTRIBUTES › memcache/attributes/default.rb default['memcached']['memory'] = 64 default['memcached']['port'] = 11211 default['memcached']['udp_port'] = 11211 default['memcached']['listen'] = '0.0.0.0' default['memcached']['maxconn'] = 1024 default['memcached']['max_object_size'] = '1m' default['memcached']['logfilename'] = 'memcached.log' case node['platform_family'] when 'ubuntu' default['memcached']['user'] = 'memcache' default['memcached']['group'] = 'memcache' when 'debian' default['memcached']['user'] = 'nobody' default['memcached']['group'] = 'nogroup' else default['memcached']['user'] = 'nobody' default['memcached']['user'] = 'nogroup' end
  • 26.
    19 . 2 THECOOKBOOKS - ATTRIBUTES case node['platform'] when "debian" case when node['platform_version'].to_f < 6.0 # All 5.X default['postgresql']['version'] = "8.3" when node['platform_version'].to_f < 7.0 # All 6.X default['postgresql']['version'] = "8.4" else default['postgresql']['version'] = "9.1" end default['postgresql']['client']['packages'] = ["postgresql-client-#{node['postgresql']['version']}","libpq-dev"] default['postgresql']['server']['packages'] = ["postgresql-#{node['postgresql']['version']}"] default['postgresql']['contrib']['packages'] = ["postgresql-contrib-#{node['postgresql']['version']}"] when "fedora" ...
  • 27.
    19 . 3 THECOOKBOOKS - AUTOMATIC ATTRIBUTES
  • 28.
    19 . 4 THECOOKBOOKS - ATTRIBUTES
  • 29.
    20 . 1 THECOOKBOOKS - TEMPLATES template '/etc/memcached.conf' do source 'memcached.conf.erb' owner 'root' group 'root' mode '0644' variables( :listen => node['memcached']['listen'], :logfilename => node['memcached']['logfilename'], :user => node['memcached']['user'], :port => node['memcached']['port'], :udp_port => node['memcached']['udp_port'], :maxconn => node['memcached']['maxconn'], :memory => node['memcached']['memory'], :max_object_size => node['memcached']['max_object_size'] ) notifies :restart, 'service[memcached]' end
  • 30.
    20 . 2 THECOOKBOOKS - TEMPLATES › memcached/templates/default/memcached.conf.erb # Run memcached as a daemon. This command is implied, and is not needed for the # daemon to run. See the README.Debian that comes with this package for more # information. -d # Log memcached's output to /var/log/memcached logfile /var/log/<%= @logfilename %> # Be verbose #-v # Be even more verbose (print client commands as well) # -vv # Start with a cap of 64 megs of memory. It's reasonable, and the daemon default # Note that the daemon will grow to this size, but does not start out holding this much # memory -m <%= @memory %> # Default connection port is 11211 -p <%= @port %> #-U <%= @udp_port %> # Run the daemon as root. The start-memcached will default to running as root if no # -u command is present in this config file -u <%= @user %> # Specify which IP address to listen on. The default is to listen on all IP addresses # This parameter is one of the only security measures that memcached has, so make sure
  • 31.
    21 . 1 THECOOKBOOKS - FILES › cookbook/app/ les/default/application-5.8.3.pm › cookbook/app/ les/default/application-5.20.1.pm cookbook_file "application.pm" do path case node['platform'] when "centos","redhat" "/usr/lib/version/1.2.3/dir/application.pm" when "arch" "/usr/share/version/core_version/dir/application.pm" else "/etc/version/dir/application.pm" end source "application-#{node['languages']['perl']['version']}.pm" owner 'root' group 'root' mode '0644' end
  • 32.
    21 . 2 THECOOKBOOKS - FILES 1. /host-$fqdn/$source 2. /$platform-$platform_version/$source 3. /$platform/$source 4. /default/$source 5. /$source host-foo.example.com/apache2_module_conf_generate.pl ubuntu-10.04/apache2_module_conf_generate.pl ubuntu-10/apache2_module_conf_generate.pl ubuntu/apache2_module_conf_generate.pl default/apache2_module_conf_generate.pl
  • 33.
    22 THE COOKBOOKS –RESOURCES & PROVIDERS (LWRP / HWRP) $ tree cookbooks/mdm/ cookbooks/mdm/ ├── CHANGELOG.md ├── README.md ├── attributes ├── definitions ├── files │   └── default ├── libraries ├── metadata.rb ├── providers ├── recipes │  └── default.rb ├── resources └── templates └── default Resources : Used to de ne a set of actions and attributes Providers : Used to say to chef-client what to do foreach de ned actions user 'random' do supports :manage_home => true comment 'Random User' uid 1234 gid 'users' home '/home/random' shell '/bin/bash' password '$1$JJsvHslV$szsCjVEroftprNn4JHtDi' end
  • 34.
    23 . 1 THECOOKBOOKS – RESOURCES & PROVIDERS (LWRP) › cookbooks/nginx/resources/app_site.rb actions :create, :remove, :enable, :disable default_action :create attribute :app_name, :kind_of => String, :name_attribute => true, :required => true attribute :cookbook, :kind_of => String, :default => "nginx" attribute :template, :kind_of => String, :default => "reverse" attribute :server_name, :kind_of => String attribute :proxy_pass_url, :kind_of => String attr_accessor :exists attr_accessor :enabled
  • 35.
    23 . 2 THECOOKBOOKS – RESOURCES & PROVIDERS (LWRP) › cookbooks/nginx/providers/app_site.rb
  • 36.
    23 . 3 THECOOKBOOKS – RESOURCES & PROVIDERS (LWRP)
  • 37.
    23 . 4 THECOOKBOOKS – RESOURCES & PROVIDERS (LWRP) › cookbooks/stash/recipes/nginx.rb node.default['nginx']['default_site_enabled'] = false include_recipe 'nginx' nginx_app_site "stash" do server_name node[:stash][:http_server][:virtual_host_name] proxy_pass_url "http://localhost:#{node[:stash][:tomcat][:port]}" cookbook "nginx" template "reverse" action [:create, :enable] notifies :reload, 'service[nginx]', :delayed end nginx_app_site "stash-ssl" do server_name node[:stash][:http_server][:virtual_host_name] proxy_pass_url "http://localhost:#{node[:stash][:tomcat][:ssl_port]}" cookbook "nginx" template "reverse_ssl" action [:create, :enable] notifies :reload, 'service[nginx]', :delayed end
  • 38.
    24 THE COOKBOOKS –RESOURCES & PROVIDERS (HWRP) Create a resource with pure Ruby Bypass the limits of Chef DSL http://tech.yipit.com/2013/05/09/advanced-chef-writing- heavy-weight-resource-providers-hwrp/
  • 39.
    25 . 1 THECOOKBOOKS - DEFINITIONS Used to de ne a set of actions with or without parameters (function) You can call your de nition many times in one or more recipes This is the same as a resource (LWRP) except that you may not notify (trigger) other resources
  • 40.
    25 . 2 THECOOKBOOKS - DEFINITIONS define :host_porter, :port => 4000, :hostname => nil do params[:hostname] ||= params[:name] directory "/etc/#{params[:hostname]}" do recursive true end file "/etc/#{params[:hostname]}/#{params[:port]}" do content "some content" end end host_porter node['hostname'] do port 4000 end host_porter "www1" do port 4001 end
  • 41.
    26 . 1 CUSTOMRESOURCES Providers (LWRP/HWRP) De nitions DEPRECATED
  • 42.
    26 . 2 CUSTOMRESOURCES From Chef version 12.5 De nitions are useless. Advise you to use a resources instead. Custom resources is a provider redesigned to be simpler. Located only in "resources" directory
  • 43.
    26 . 3 CUSTOMRESOURCES exampleco/resources/site.rb property :homepage, String, default: '<h1>Hello world!</h1>' load_current_value do if ::File.exist?('/var/www/html/index.html') homepage IO.read('/var/www/html/index.html') end end action :create do package 'httpd' service 'httpd' do action [:enable, :start] end file '/var/www/html/index.html' do content homepage end end action :delete do package 'httpd' do action :delete end end exampleco_site 'httpd' do homepage '<h1>Welcome to the Example Co. website!</h1>' action :create end
  • 44.
    27 . 1 THECOOKBOOKS - LIBRARIES Allows to extend the Chef's classes or create your own Ruby lib Do what Chef does not already do Do external data processing to use in Chef
  • 45.
    27 . 2 THECOOKBOOKS - LIBRARIES
  • 46.
    27 . 3 THECOOKBOOKS - LIBRARIES
  • 47.
    28 ROLES Used to groupone or more roles, attributes and recipes { "name": "ns", "description": "The DNS Role", "json_class": "Chef::Role", "default_attributes": { "bind": { "dnssec-validation": "no", "zone-statistics": "yes", "zones": [ "company.local", "0.168.192.in-addr.arpa" ], "acls": { "slaves": [ "192.168.0.2", "192.168.0.3" ], "mdm_network": [ "192.168.0.0/24" ] }, "allow_query": ["localhost", "company_network"], "allow_recursion": ["localhost", "company_network"], "allow_query_cache": ["localhost", "company_network"], "also_notify": true, } }, "override_attributes": { "bind": { "masters": ["192.168.0.1"], "allow_transfer": ["slaves"] }, "ntp": { "is_server": true, "servers": ["0.fr.pool.ntp.org", "1.fr.pool.ntp.org", "2.fr.pool.ntp.org", "3.fr.pool.ntp.org"], "restrictions": ["192.168.0.0 mask 255.255.255.0 nomodify notrap"]
  • 48.
    29 ENVIRONNEMENTS { "name": "staging", "description": "TheStaging environment", "cookbook_versions": { "box": "= 0.3.8", "nginx": "= 0.5.3", "web": "= 1.4.29", "base": "= 0.3.4", "tools": "= 0.4.2", "users": "= 1.7.5", "db": "= 0.3.5", "ambari": "= 0.7.1" }, "json_class": "Chef::Environment", "chef_type": "environment", "default_attributes": { "php-fpm": { "session": { "save_handler_type": "memcache", "save_handler_path": "tcp://localhost:11211" }, "log_level": "alert", "emergency_restart_threshold": "10", "emergency_restart_interval": "1m", "process_control_timeout": "10s" }, "nginx": { "keepalive_timeout": "40s", "client_body_timeout": "20s", "client_header_timeout": "20s", "server_tokens": "off", "disable_access_log": "true", "mdm_log_format": "true", "fastcgi_buffers": "8 16k", "fastcgi_buffer_size": "32k", Sets behaviors or different attributes for a given environment include_recipe 'sshd' include_recipe 'sudo' include_recipe 'users::system' include_recipe 'users::admins' include_recipe 'users::operators' if node.chef_environment != 'prod' include_recipe 'users::qa' include_recipe 'users::devops' end case node[:platform_family] when 'debian' include_recipe 'apt' when 'rhel', 'fedora' include_recipe 'yum' include_recipe 'yum-epel' include_recipe 'selinux::disabled' include_recipe 'iptables::disabled' end include_recipe 'timezone-ii' include_recipe 'ntp' include_recipe 'postfix' include_recipe 'snmp' include_recipe 'locales' include_recipe 'line'
  • 49.
    POLICYFILE Policies are anew feature of Chef that combine the very best parts of Roles, Environments and cookbook dependency resolvers (Berkshelf) into a single easy to use work ow. It is associated with a group of nodes, cookbooks, and settings. When these nodes run, they run the recipes speci ed in the Policy le run-list Resolves real-world problems of team work ow
  • 50.
    30 . 130. 2 POLICYFILE De ne in Policy le: › Run list › Cookbook dependencies with version constraint and source › Attributes overriding name "jenkins-master" run_list "java", "jenkins::master", "recipe[policyfile_demo]" default_source :supermarket, "https://mysupermarket.example" cookbook "policyfile_demo", path: "cookbooks/policyfile_demo" cookbook "jenkins", "~> 2.1" cookbook "mysql", github: "chef-cookbooks/mysql", branch: "master" default['java']['version'] = '8'
  • 51.
    30 . 3 POLICYFILE Createyour policy le into your cookbook $ cd cookbooks/app $ chef generate policyfile $ ls -l -rw-r--r-- 1 mlopez wheel 596 12 oct 21:57 Policyfile.rb Create your policy le into policies directory $ ls -l total 24 -rw-r--r-- 1 mlopez wheel 70 12 oct 22:16 LICENSE -rw-r--r-- 1 mlopez wheel 1546 12 oct 22:16 README.md -rw-r--r-- 1 mlopez wheel 1067 12 oct 22:16 chefignore drwxr-xr-x 4 mlopez wheel 136 12 oct 22:16 cookbooks drwxr-xr-x 4 mlopez wheel 136 12 oct 22:16 data_bags drwxr-xr-x 4 mlopez wheel 136 12 oct 23:13 policies $ chef generate policyfile policies/web
  • 52.
    30 . 4 Createthe lock le $ chef install [path/to/policyfile.rb] Update lock le after modi cation $ chef update [path/to/policyfile.rb] Upload cookbook and policy le $ chef push POLICY_GROUP PATH/TO/POLICYFILE.rb → If POLICY_GROUP doesn't exists it will be created Assign policy name and policy group into client.rb's node or in node con guration
  • 53.
    31 . 1 DATABAGS Stock & share data Read the desired data from a recipe or from your desktop (knife) Write data collected during a run $ knife data bag create DATA_BAG_NAME [ITEM] knife data bag create users toto knife data bag from file users data_bags/users/toto.json
  • 54.
    31 . 2 DATABAGS › knife data bag create users myUser { "id": "myUser", "ssh_keys": [ "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDakMx4mjWYMko2r085yq/vq0Ey2DiVWXeJ ], "groups": [ "mdm_qa" ], "uid": 4001, "shell": "/bin/bash", "comment": "My User", "action": "remove" } › knife data bag show users myUser –Fj > data_bags/users/myUser.json
  • 55.
    31 . 3 DATABAGS admins = [] search(:admins, "*:*").each do |admin| login = admin[‘id’] admins << login home = "/home/#{login}" user login do uid admin['uid'] gid admin['gid'] shell admin['shell'] home home comment admin['comment’] supports :manage_home => true end end
  • 56.
    31 . 4 Writein a data bag DATA BAGS sam = { "id" => "sam", "Full Name" => "Sammy", "shell" => "/bin/zsh" } databag_item = Chef::DataBagItem.new databag_item.data_bag("users") databag_item.raw_data = sam databag_item.save sam = data_bag_item("users", "sam") sam["Full Name"] = "Samantha" sam.save
  • 57.
    31 . 5 Whatshould I do if I have sensitive data ? DATA BAGS → Encrypt !! $ knife data bag create passwords postgresql –secret-file <path>/encrypted_data_bag_secret</path> $ knife data bag show passwords postgresql -Fj { "id": "postgresql", "password": { "encrypted_data": "nu0GFIaJuzefK1iCgmYxWbRO64tvEezZJA/7iOUT87NLg=n", "iv": "LWK$u1omaWHHNfzfDcYN45g==n", "version": 1, "cipher": "aes-256-cbc" }, "databases": { "encrypted_data": "lG4EULs9UQKKwjfzef8/WrccoGilQO2m7O6JNnIeMu199jGIT2l+/MvR+bX6dnk2U/_dwn "iv": "UmtXyR9m0ornADWbiayPyw==n", "version": 1, "cipher": "aes-256-cbc" } }
  • 58.
    32 . 1 CHEFVAULT Whithout Chef Vault, nodes share a shared secret key le → Not good for security With Chef Vault, nodes and workstation use their own keypair to decrypt data bag. Chef administrators de ne which node or admin can access to the encrypted data
  • 59.
    32 . 2 or CHEFVAULT gem install chef-vault Create a Vault Allow mlopez, vaubert and nodes with web role assigned to decrypt content $ knife vault create credentials database -A mlopez,vaubert -M client -S ‘roles:web' -J '{“db_password”:”some_password”}' $ knife encrypt create credentials database --json '{“db_password”:”some_password”}' --search 'role:web' --admins mlopez, vaubert --mode client
  • 60.
    32 . 3 or CHEFVAULT From an Unauthorized admin's workstation $ knife vault show credentials database db_password: cipher: aes-256-cbc encrypted_data: dsiBtADAV8Sbis89yKuYBvbdNXPpu8bQfJrS20op7zoysfR8roFlzpVHyoaG2 4yb3 iv: +0siNLzFHHqEkP07k6JhYw== version: 1 id: database Authorized users see the decrypted content $ knife vault show credentials database $ knife decrypt credentials database --mode client db_password: some_password id: database
  • 61.
    32 . 4 CHEFVAULT Content of database_keys $ knife data bag show credentials database_keys admins: mlopez vaubert clients: web-01 web-02 id: database_test_keys mlopez: SOME KEY vaubert: SOME KEY web-01: SOME KEY web-02: SOME KEY
  • 62.
    32 . 5 CHEFVAULT Add a new admin workstation $ knife vault update credentials database -A mhue Rotate all keys $ knife vault rotate all keys
  • 63.
    32 . 6 CHEFVAULT Use Vault in recipe chef_gem 'chef-vault' do compile_time true if respond_to?(:compile_time) end require 'chef-vault' # Or just include chef_vault coobkook include_recipe 'chef-vault' case ChefVault::Item.data_bag_item_type('credentials', 'database') when :normal ... when :encrypted ... when :vault ... end
  • 64.
    33 . 1 OHAI Ohaiis a tool that is used to detect attributes on a node, inventory the system (platform, cpu, memory...), and then provide automatically these attributes to the chef-client at the start of every chef-client run. Check Automatic attributes previously seen. You can list node attributes by running "ohai" command $ ohai { "hostname": "node-01", "machinename": "node-01", "fqdn": "node-01.my.local", "domain": "my.local", "network": { "interfaces": { "lo": { "encapsulation": "Loopback", "addresses": { "127.0.0.1": { "family": "inet", "prefixlen": "8", "netmask": "255.0.0.0", ...
  • 65.
    33 . 2 OHAI- CUSTOM PLUGIN You can create your own Ohai plugin to collect data before the run and set them into Ohai as an attribute Use Ohai cookbook and put your plugin into ' les' directory › cookbooks/ohai/ les/default/plugins/haproxy.rb # Encoding: utf-8 # Get current version of Haproxy Ohai.plugin(:Haproxy) do provides 'haproxy' collect_data(:linux) do haproxy Mash.new [['dpkg-query -W haproxy | awk '{print $2}' | sed 's/(^[1-9].[1-9]).*/1/'', :installed_version]].each do |cmd, property| so = shell_out(cmd) haproxy[property] = so.stdout.delete("n") end end end
  • 66.
    34 TEAM WORKING Git ->Create branches Test locally before uploading his cookbook Vagrant / Virtualbox, Kitchen CI...etc. Chef solo/zero/local Bump version of your cookbook and upload it into environment Freeze uploaded version and apply version constraint into environnements Knife Spork – Plugin allow you to bump, upload and promote a cookbook more easily. Noti cation plugin, auto git add … $ knife cookbook upload nginx [--freeze] [--force] [-E <environment>] $ knife spork omni nginx –l minor –e qa_group</environment>
  • 67.
    35 . 1 TESTHIS JOB Syntax Logical tests with Foodcritic Obsolescence of resources used, invalid search queries, syntax best- practice… Unit tests with ChefSpec $ knife cookbook test nginx package 'foo' require 'chefspec' describe 'example::default' do let(:chef_run) { ChefSpec::SoloRunner.converge(described_recipe) } it 'installs foo' do expect(chef_run).to install_package('foo') end end
  • 68.
    35 . 2 TESTHIS JOB Integration tests with Kitchen CI Check if a run runs without errors (converge) Include unit tests (Rspec, Bats…) Run many tests suite in one time (client / server) on many platforms Check which nodes use your cookbook Simulate an execution of a run knife preflight web::ws knife search node -i "recipes:web::ws » chef-client --why-run knife ssh ‘name:srv-01.pp’ ‘sudo chef-client –W’
  • 69.
    36 DEBUG – WHYIT DOESN'T WORK ? chef-client –l debug Generate logs Chef::Log.debug(« Doesn’t work ») puts myVariable Raise Exceptions Chef::Log.fatal!('Deployment failure...') raise
  • 70.
    37 . 1 ADVANCED Overridea run-list (-o "recipe[]") Override a community Cookbook knife ssh 'name:srv-01.dev' 'sudo chef-client –o "recipe[firefox]"' include_recipe 'nginx' resources("template[/etc/nginx/nginx.conf]").cookbook 'myNginx')
  • 71.
    37 . 2 ADVANCED Notications by chef-handler Knife diff (/*)
  • 72.
    38 SOURCES & BOOKS ChefStarter Chef Infrastructure Automation Cookbook https://docs.chef.io http://tech.yipit.com/2013/05/09/advanced-chef-writing-heavy- weight-resource-providers-hwrp/ http://dougireton.com/blog/2013/02/03/knife-tricks http://www.pburkholder.com/blog/2015/04/23/emulating- cookbook-semver-build-numbers-with-chef-policy les/ https://yolover.poise.io/
  • 73.
    SOURCES & BOOKS ›Chef Vault https://github.com/chef/chef-vault https://blog.chef.io/2016/01/21/chef-vault-what-is-it-and-what-can-it-do-for-you/ https://blog.chef.io/2013/09/19/managing-secrets-with-chef-vault/ http://hedge-ops.com/chef-vault-tutorial/