Dr. D. Y.

Patil Pratishthan’s
D. Y. Patil Institute of Master of Computer Applications and Management
(Approved by AICTE, New Delhi & Affiliated to Savitribai Phule Pune University)
Dr. D. Y. Patil Educational Complex, Sector 29, Pradhikaran, Akurdi, Pune – 411 044
Tel No: (020)27640998, Website: www.dypimca.ac.in, E-mail : director@dypimca.ac.in
---------------------------------------------------------------------------------------------------------------------
-

MONOGRAPH

Subject Code: IT-41 DevOps

Unit 3. Chef for Configuration Management
Configuration Management
 Managing multiple virtual machines requires proper configuration—loading them with
the right software and making sure that software can run.
 Solution - To implement a policy as code based automation solution that keeps
environments consistent.
 Chef enables DevOps teams to create pipelines that can cross both internal and external
boundaries, standardizing environments and processes locally within the data center and
up in the cloud.
 As a result, get a dynamic environment that’s stable no matter how complicated your
configurations are.
 When your application deployment and infrastructure changes move at the same pace,
your entire IT organization functions better.
 A DevOps team that turns configuration into code can leverage the same tools and
processes you use on your applications to efficiently and successfully prepare
environments to run applications.

Common Chef Terminology

Workstation
Reduce risks by iterating on policy changes before pushing them to production.
Workstation includes:
• Chef Tools: Chef Infra Client, Chef InSpec, Chef Habitat, Chef Cookstyle, and knife
• Chef Language: Pre-built resources for managing systems and helpers that make
authoring and distributing cookbooks easy.
Client
The chef-client is the name for both the command line tool that runs on your local
computer and the program that runs on nodes.
The chef-client allows you to ensure the configuration compliance of your systems
through policy code.
You use the chef-client command line tool from your local computer to send instructions
to the program on the node.
 You install the chef-client on nodes with the bootstrap process from your computer and
then you configure it to run on an interval to ensure continuous configuration compliance.
Server –
• The Chef Infra Server acts as a hub for configuration data.
• The Chef Infra Server stores cookbooks, the policies that are applied to nodes, and
metadata that describes each registered node that is being managed by Chef Infra Client.
• Nodes use Chef Infra Client to ask the Chef Infra Server for configuration details, such as
recipes, templates, and file distributions.
Repository-
The repository structure in which cookbooks are authored, tested, and maintained.

Pull Vs. Push Configuration

Pull Configuration: In this type of Configuration Management, the nodes poll a
centralized server periodically for updates. These nodes are dynamically configured so
basically they are pulling configurations from the centralized server. Pull configuration is
used by tools like Chef, Puppet etc.
Push Configuration: In this type of Configuration Management, the centralized Server
pushes the configurations to the nodes. Unlike Pull Configuration, there are certain
commands that have to be executed in the centralized server in order to configure the
nodes. Push Configuration is used by tools like Ansible.
 Chef Architecture

Workstation

• The Workstation is the location from which all of Chef configurations are managed.
• This machine holds all the configuration data that can later be pushed to the central Chef
Server.
• These configurations are tested in the workstation before pushing it into the Chef Server.
• A workstation consists of a command-line tool called Knife, that is used to interact with
the Chef Server.
• There can be multiple Workstations that together manage the central Chef Server.
• Workstations are responsible for performing the below functions:
• Writing Cookbooks and Recipes that will later be pushed to the central Chef
Server
 • Managing Nodes on the central Chef Server

Recipes
• A Recipe is a collection of resources that describes a particular configuration or policy.
• It describes everything that is required to configure part of a system. The user writes
Recipes that describe how Chef manages applications and utilities (such as Apache HTTP
Server, MySQL, or Hadoop) and how they are to be configured.
• These Recipes describe a series of resources that should be in a particular state, i.e.
Packages that should be installed, services that should be running, or files that should be
written.
Cookbooks
• Multiple Recipes can be grouped together to form a Cookbook. A Cookbook defines a
scenario and contains everything that is required to support that scenario:
• Recipes, which specifies the resources to use and the order in which they are to be
applied
• Attribute values
• File distributions
• Templates
• Extensions to Chef, such as libraries, definitions, and custom resources

Workstations - two major components

• Knife utility: This command line tool can be used to communicate with the central Chef
Server from Workstation. Adding, removing, changing configurations of Nodes in a central
Chef Server will be carried out by using this Knife utility. Using the Knife utility,
Cookbooks can be uploaded to a central Chef Server and Roles, environments can also be
managed. Basically, every aspect of the central Chef Server can be controlled from
Workstation using Knife utility.

• A local Chef repository: This is the place where every configuration component of central
Chef Server is stored. This Chef repository can be synchronized with the central Chef
Server (again using the knife utility itself).
Chef Server
• The Chef Server acts as a hub for configuration data.
• The Chef Server stores Cookbooks, the policies that are applied to Nodes, and metadata
that describes each registered Node that is being managed by the Chef-Client.
• Nodes use the Chef-Client to ask the Chef Server for configuration details, such as
Recipes, Templates, and file distributions.
• The Chef-Client then does as much of the configuration work as possible on the Nodes
themselves (and not on the Chef Server).
• Each Node has a Chef Client software installed, which will pull down the configuration
from the central Chef Server that are applicable to that Node.
• This scalable approach distributes the configuration effort throughout the organization.

Chef Nodes
• Nodes can be a cloud based virtual server or a physical server in your own data center,
that is managed using central Chef Server.
• The main component that needs to be present on the Node is an agent that will establish
communication with the central Chef Server. This is called Chef Client.
• Chef Client performs the following functions:
• It is responsible for interacting with the central Chef Server.
• It manages the initial registration of the Node to the central Chef Server.
• It pulls down Cookbooks, and applies them on the Node, to configure it.
• Periodic polling of the central Chef Server to fetch new configuration items, if any.

Advantages of Chef
• Can automate an entire infrastructure using Chef. All tasks that were manually being
done, can now be done via Chef tool.
• Can configure thousands of nodes within minutes using Chef.
• Chef automation works with the majority of the public cloud offerings like AWS
• Chef will not only automate things, but will also keep the systems under consistent check,
and confirm that the system is in fact configured the way it is required (Chef
Agent/Client does this job). If somebody makes a mistake by modifying a file, Chef will
correct it.
 • An entire infrastructure can be recorded in the form of a Chef repository, that can be used
as a blueprint to recreate the infrastructure from scratch.

Steps to install Chef

• Install Chef DK (Development Kit) on Chef Workstation
• Setup a Chef Server
• Create a Recipe or a Cookbook / download a Cookbook from Chef Supermarket in
Workstation
• Upload a Cookbook on the Chef Server
• Connect A Node To The Chef Server
• Deploy the Cookbook from the Server to the Node

Workstation Setup
 Knife is Chef’s command-line tool to interact with the Chef server, for uploading
cookbooks.
 It provides an interface between the chefDK (Repo) on the local machine and the Chef
server.
 Knife provides a set of commands to manage Chef infrastructure.
 It helps in managing −
Chef nodes
Cookbook
Recipe
Environments
Cloud Resources
Cloud Provisioning
Installation on Chef client on Chef nodes
Knife Setup
• In order to set up knife, move to .chef directory and create a knife.rb inside the chef repo,
which tells knife about the configuration details.
Organization Setup
• A single instance of the Chef Infra Server can support many organizations.
• Each organization has a unique set of groups and users.
• Each organization manages a unique set of nodes, on which a Chef Infra Client is
installed and configured so that it may interact with a single organization on the Chef
Infra Server.

A user may belong to multiple organizations under the following conditions:

• Role-based access control is configured for each organization
• For a single user to interact with the Chef Infra Server using knife from the same chef-
repo, that user may need to edit their config.rb file before that interaction

Organization Setup
• Using multiple organizations within the Chef Infra Server ensures that the same toolset,
coding patterns and practices, physical hardware, and product support effort is being
applied across the entire company, even when:
 Multiple product groups must be supported—each product group can have its own security
requirements, schedule, and goals
 Updates occur on different schedules—the nodes in one organization are managed
completely independently from the nodes in another
 Individual teams have competing needs for object and object types—data bags,
environments, roles, and cookbooks are unique to each organization, even if they share the
same name.
Environments
  Chef helps in performing environment specific configuration. It is always a good idea to
have a separate environment for development, testing, and production.
 Chef enables grouping nodes into separate environments to support an ordered
development flow.
_default Environment
 Each organization will always start with at least a single environment called default
environment, which is always available to the Chef server.
 A default environment cannot be modified in anyway.
 Any kind of changes can only be accommodated in the custom environment that we create.
Environment Attributes
 An attribute can be defined in an environment and then used to override the default settings
in the node.
 When the Chef client run takes place, then these attributes are compared with the default
attributes that are already present in the node.
 When the environment attributes take precedence over the default attributes, Chef client
will apply these settings and values when the Chef client run takes place on each node.
Attribute
 An attribute is a specific detail about a node.
 Attributes are used by Chef Infra Client to understand:
 The current state of the node
 What the state of the node was at the end of the previous Chef Infra Client
run
 What the state of the node should be at the end of the current Chef Infra
Client run
 Attributes are defined by:
 The node as saved on the Chef Infra Server
 Attributes passed using JSON on the command line
  Cookbooks (in attribute files and/or recipes)
 Policyfiles

Attribute Type Description

A default attribute is automatically reset at the start of every chef-client run and has the
default
lowest attribute precedence. Use default attributes as often as possible in cookbooks.

Use the force_default attribute to ensure that an attribute defined in a cookbook (by an
force_default attribute file or by a recipe) takes precedence over a default attribute set by a role or an
environment.

A normal attribute is a setting that persists in the node object. A normal attribute has a
normal
higher attribute precedence than a default attribute.

A default attribute is automatically reset at the start of every chef-client run and
default has the lowest attribute precedence. Use default attributes as often as possible in
cookbooks.

Use the force_default attribute to ensure that an attribute defined in a cookbook

force_default (by an attribute file or by a recipe) takes precedence over a default attribute set by
a role or an environment.

A normal attribute is a setting that persists in the node object. A normal attribute
normal
has a higher attribute precedence than a default attribute.

Attribute Type Description

An override attribute is automatically reset at the start of every chef-client run and
has a higher attribute precedence than default, force_default,
override and normal attributes. An override attribute is most often specified in a recipe, but
can be specified in an attribute file, for a role, and/or for an environment. A
cookbook should be authored so that it uses override attributes only when required.

Use the force_override attribute to ensure that an attribute defined in a cookbook

force_override (by an attribute file or by a recipe) takes precedence over an override attribute set
by a role or an environment.

An automatic attribute contains data that is identified by Ohai at the beginning of

automatic every chef-client run. An automatic attribute cannot be modified and always has
the highest attribute precedence.
Order of Applying an Attribute

Test Node Setup

 A Node represents a server and is typically a virtual machine, container instance, or
physical server — basically, any compute resource in your infrastructure that is managed
by Chef.
 Knife preflight shows details about all the nodes which uses a certain cookbook before
uploading it to Chef server.

Check node details using knife

In order to get started, we need to have knife-preflight gem installed.
Step 1 − Define the path in the gem file.
:~/chef-repo $ subl Gemfile
source 'https://rubygems.org'
gem 'knife-preflight‘
Step 2 − Run bundler to install knife-preflight gem.
:~/chef-repo $ bundle install
 Fetching gem metadata from https://rubygems.org/
...TRUNCATED OUTPUT...
Installing knife-preflight (0.1.6)

Working Method
Run knife-preflight on the given cookbook to find out which nodes and roles have the given
cookbook in their expanded run lists.
~/chef-repo $ knife preflight ntp
Searching for nodes containing ntp OR ntp::default in their
expanded run_list...
2 Nodes found
www-staging.example.com
cms-staging.example.com
Searching for roles containing ntp OR ntp::default in their
expanded run_list...
3 Roles found
your_cms_role
your_www_role
your_app_role
Found 6 nodes and 3 roles using the specified search criteria

Cookbook execution on the node

 There are multiple ways for a cookbook to get executed on the node.
 You can assign the cookbook directly to a node by adding it to the node's run list.
 You can add a cookbook to the role and add the role to the node's run list.
 You can add the role to the run list of another role and add that other role to the node's run
list.
 A cookbook can be a dependency of another used cookbook.
 No matter how a cookbook ends up in a node's run list, the knife preflight command will
catch it as Chef stores all expanded lists of roles and recipes in node attributes. The knife
preflight command issues a search for exactly those node attributes.
knife role
A role is a way to define certain patterns and processes that exist across nodes in an
organization as belonging to a single job function.
Each role consists of zero (or more) attributes and a run-list.
Each node can have zero (or more) roles assigned to it.
When a role is run against a node, the configuration details of that node are compared
against the attributes of the role, and then the contents of that role’s run-list are applied to
the node’s configuration details.
When a Chef Infra Client runs, it merges its own attributes and run-lists with those
contained within each assigned role.
Use the knife role subcommand to manage the roles that are associated with one or more
nodes on a Chef Infra Server.
Create role
Use the create argument to add a role to the Chef Infra Server.
Role data is saved as JSON on the Chef Infra Server.
knife role create ROLE_NAME (options)
Eg. knife role create role1
In the $EDITOR enter the role data in JSON:
{
"name": "role1",
"default_attributes": {
},
"json_class": "Chef::Role",
"run_list": ["recipe[cookbook_name::recipe_name]",
"role[role_name]“ ],
"description": "",
"chef_type": "role",
 "override_attributes": {
}}
Run list
• A run-list defines all of the information necessary for Chef to configure a node into the
desired state.
• A run-list is:
o An ordered list of roles and/or recipes that are run in the exact order defined in the
run-list; if a recipe appears more than once in the run-list, Chef Infra Client will not
run it twice.
o Always specific to the node on which it runs; nodes may have a run-list that is
identical to the run-list used by other nodes
o Stored as part of the node object on the Chef server
o Maintained using knife and then uploaded from the workstation to the Chef Infra
Server, or maintained using Chef Automate
Role Operations
Delete role
knife role delete ROLE_NAME
Edit role
knife role edit role1
From file - Use the from file argument to create a role using existing JSON data as a template.
knife role from file FILE
List - Use the list argument to view a list of roles that are currently available on the Chef Infra
Server.
knife role list –w
Show - Use the show argument to view the details of a role.
knife role show ROLE_NAME
How Role Works?
1. Define a role in a Ruby file inside the roles folder of Chef repository.
2. A role consists of a name and a description attribute.
3. A role consists of role-specific run list and role-specific attribute settings.
4. Every node that has a role in its run list will have the role’s run list exacted into its own.
 5. All the recipes in the role’s run list will be executed on the node.
6. The role will be uploaded to Chef server using the knife role from file command.
7. The role will be added to the node run list.
8. Running Chef client on a node having the role in its run list will execute all the recipes
listed in the role.

Run-list Format
A run-list must be in one of the following formats: fully qualified, cookbook, or default.
"role[NAME]“
OR
"recipe[COOKBOOK::RECIPE]“
Use a comma to separate roles and recipes when adding more than one item the run-list:
recipe[COOKBOOK::RECIPE],COOKBOOK::RECIPE,role[NAME]"
Data Bags
 Data bags store global variables as JSON data. Data bags are indexed for searching and
can be loaded by a cookbook or accessed during a search.
 A data bag item may be encrypted using shared secretencryption.
 This allows each data bag item to store confidential information (such as a database
password) or to be managed in a source control system (without plain-text data appearing
in revision history).
 Each data bag item may be encrypted individually; if a data bag contains multiple
encrypted data bag items, these data bag items are not required to share the same encryption
keys.
Create a Data Bag
 A data bag can be created in two ways: using knife or manually.
 Using knife
knife data bag create DATA_BAG_NAME (DATA_BAG_ITEM)
knife data bag from file BAG_NAME ITEM_NAME.json
file in “admins” directory => changes to Chef Infra Server
knife data bag from file admins ITEM_NAME.json
  Manually
One or more data bags and data bag items can be created manually under the data_bags
directory in the chef-repo.
mkdir data_bags/data_bag_folder
vi data_bags/data_bag_folder/ITEM_NAME.json

Store Data in a Data Bag

When the chef-repo is cloned from GitHub, the following occurs:
 A directory named data_bags is created.
 For each data bag, a sub-directory is created that has the same name as the data bag.
 For each data bag item, a JSON file is created and placed in the appropriate sub-directory.
When deploying from a private repository using a data bag, use the deploy_key option to ensure
the private key is present:
{
'id': 'my_app',
... (truncated) ...
'deploy_key': 'ssh_private_key'
}
where ssh_private_key is the same SSH private key as used with a private git repository

Directory Structure
All data bags are stored in the data_bags directory of the chef-repo.
Example:
- data_bags
- admins – data_bag1
- admin1.json - data_item
- admin2.json
- db_users – data_bag2
- user1.json - data_item
- user2.json
- db_config – data_bag3
- small.json - data_item
 - medium.json
Data Bag Items
knife can encrypt and decrypt data bag items when the knife data bag subcommand is run
with the create, edit, from file, or show arguments and the following options:
Eg. knife data bag create passwords mysql --secret-file /tmp/my_data_bag_key
“passwords” - name of the data bag
“mysql” - name of the data bag item
“/tmp/my_data_bag_key” - path to the location in which the file that contains the secret-
key is located
Verify Encryption
When the contents of a data bag item are encrypted, they will not be readable until they are
decrypted
Eg. knife data bag show passwords mysql
It will return,
id: mysql
pass:
cipher: aes-256-cbc
encrypted_data:
JZtwXpuq4Hf5ICcepJ1PGQohIyqjNX6JBc2DGpnL2WApzjAUG9SkSdv75TfKSjX4
iv: VYY2qx9b4r3j0qZ7+RkKHg==
version: 1
Decrypt
An encrypted data bag item is decrypted with a knife command :
Eg. knife data bag show --secret-file /tmp/my_data_bag_key passwords mysql
It will return JSON output :
{
"id": "mysql",
"pass": "thesecret123",
"user": "fred"
}
 Edit a Data Bag with Knife
Use the edit argument to edit the data contained in a data bag.
If encryption is being used, the data bag will be decrypted, the data will be made available
in the $EDITOR, and then encrypted again before saving it to the Chef Infra Server.
Eg. knife data bag edit admins ITEM_NAME

Data bags for Creating Users

 Chef Infra Client can create users on systems based on the contents of a data bag.
 For example, a data bag named “admins” can contain a data bag item for each of the
administrators that will manage the various systems that each Chef Infra Client is
maintaining.
 A recipe can load the data bag items and then create user accounts on the target system