DevOps Tutorial {Technical Guftgu]
DevOps Tutorial {Technical Guftgu]
1- Prequisites for DevOps
2- What is DevOps , DevOps Stages , Agile
3- Basics of AWS Cloud required for DevOps
4 Everything about Linux from Scratch (ii)
Started in 1964 at Bell Laboratory, New JErsey for multi-user real time operating system
Withdrawn in 1969
Dennis Ritchie and Ken Thompson start UNICS (Uniplexed Information & Computing Services)
UNIX V6 was released in 1975 which became very popular
Different bodies produced different of their own flavours of UNICS (E.g. IBM-AIX, MAC OS , HP _ UX)
A student , Linus Torvalds produced LINUX in Helsinki University, 1991.Note that LINUX is not a UNIX
derivative but merely took inspiration from it
Professor Andrew Tanenbaum produced MINUX for teaching purposes
GNU , a product of the Free Software movement, produced associated software for Linux kernel to complete
Linux operating system
(Linux is origin
ally just kernel, not operaring system : The kernel refers to core software tjhat is responsible for
accessing and directly interacting with hardware components of the system)
Linux source code is free and open source , so anyone is free to produce their own version of operating system
Several companies made their own version of linux operating system (aka distributions) - some commercial
and some open source as well. e.g. RHEL , Fedora, Debian <ubuntu , CentOS, Amazon Linux ,etc.
(Operating systems can be operated either through CLI (Command Line Interface) or GUI (Graphical User
Interface))
Linux Features:
1. Open Source - The internal source code is openly available which allows a perspicacious insight into the inner
workings and mechanisms of the system from end user
2. Secure - Advanced access control , auditing of open source code and defence against viruses' self-replicating
mechanism
3. Simplified updates for all installed software - Effective package management and installation tools available
which are much
4. Light weight - Dynamic ally loads kernel modules. Takes less memory and disk resources. Less bloatware.
5. Multiuser and Multitasking - Advanced access control and advanced process control and resource sharing
mechanisms
**6. Variety of multiple distributions - ** Due to the open source nature, there is a great variety of distributions
available. Users can install distribution which most suits them
5 - Everything about Linux from Scratch (ii)
Windows Linux
Hardware interacts with OS Hardware interacts with kernel
Shell is medium between OS and user Shell is medium between kernel and user
Folder Directory
Administrator root user
Software package
The top level directory is /, which contains all files.
C:\ contains all system files - Default folders include
Root directories include /root , /home , /boot , /etc ,
'Program Files, Users , Programs(x86) and Perflogs
/usr and /bin
CLI tends to be faster and more accurate than doing the same thing in GUI mode
A 'folder' in Linuc is referred to as directory
File system hierarchy in Linux
1. /home - home directory of other users
2. /root - home directory for root user
3. /boot -Contains bootable files for linux
4. /etc - contains all configuration
5. /usr - software is installed in this directory by default
6. /bin - contains commands used by all users
7. /sbin - contains commands used by only root user
8. /opt - Optional application software packages
9. /dev - Device files . Includes terminal devices, USBs
POST (Power On Self Test) :
6 - Everything about Linux from Scratch (iii) - Linux commands and its
use cases
**How to create a file: **
1. cat
E.g.
cat > file1
1
Copy Copied
Here, file1 is the new file created. Anything entered into stdin is written into this file. To terminate input , the
keyboard shortcut Ctrl+D can be used which sends EOF (End-of-File character)
2. touch creates an empty file
3. vi / vim - one of the most popular CLI editors
4. nano - another popular CLI editor
sudo su - (superuser-do switch-user) changes from
NOTE : Everything in linux bash is case-sensitive, stick to smaller letters mostly
The cat command
One of the most universal tools . It essentially copies standard input to standard outputThere are three main use
cases :
1. Create file: Create single file
cat > file1
Hello
1
2
Copy Copied
2. Concatenate file: Concatenate the content of files
3. Copy files Copying content of one file to another
The touch command
Updates timestampUse cases of touch :
1. Create an empty file
touch file1
Copy Copied
(creates empty file file1 )
2. Create multiple empty files
touch file1
Copy Copied
(creates empty file file1 )
3. Change all timestamps of a file
touch file2
Copy Copied
4. Update only access time of file, modify time of file
touch -a file1
touch -m file2
1
2
Copy Copied
(creates empty file file1 )Types of timestamps:1.**Access time: ** ( Last time when user accesed)
2. **Change time: ** ( Last time when file metadata was changed)
3. **Modified time: ** (Last time when a file was modified)
The vi editor
A programmer text editor
Can be used to edit any plain text. Especially useful for editing programs;
:w Saves file
:wq / :x Saves file and quits editor
:q Quits file
:q: Forcefully quits, even if file is unsaved
The nano editor
nano file_a
Copy Copied
creates file_a if it doesn't already exist and opens it.
Contains instructions on keybaord shortcuts for saving /exiting/ etc.
Demonstration
7 - Everything about Linux from Scratch (iv)
mkdir command
Creates new directory
mkdir dir1 dir2
1
Copy Copied
creates new directories dir1 and dir2
mkdir dir1/dir2/dir3
Copy Copied
creates dir1 directory , a dir2 inside it and another dir2 inside it.
cd directory
Change directory command
cd dir2
Copy Copied
switches to directory dir2
cd ..
Copy Copied
changes directory to .. (i.e. parent directory)
pwd directory
print working directory/ Shows current working directory.
pwd
Copy Copied
8 - Everything about Linux from Scratch (v)
Any file/directory which begins with . is hidden, they can be shown using ls only with -a flag
ls -a
1
Copy Copied
cp command
Used for copying files
cp file1 file2
Copy Copied
Copies file1 to file2
mv command
Move files/directories
mv file1 dir1
Copy Copied
moves file1 to dir1
rmdir command
Command for removing specified directory (if empty)
rmdir -p
Copy Copied
Remove both mentioned directory and its parent directory
rm command
remove files/directories
rm -rf
Copy Copied
forcefully removes all contents of directory and its constituent subdirectories and files recursively
rm -rp
1
Copy Copied
recursively remove both directory and its parent directory.
Demonstration
9 - Everything about Linux from Scratch (vi)
hostname command
Reveals details about the currently running machine
hostname -i
1
2
Copy Copied
Demonstrates version name of OS
ifconfig command
Shows IP address of the machine.
yum package manager
YellowDog Update Modifier (YUM)
yum install httpd -y
Copy Copied
Installs httpd package to system. The -y flag auto-confirms every prompt
yum remove httpd
Copy Copied
Removes httpd packagr
yum update httpd
Copy Copied
Updates httpd package
yum list
Copy Copied
Show list of installed packages
System services
service httpd start
Copy Copied
Initiates httpd service
service httpd status
Copy Copied
Checks status of httpd service
chkconfig httpd on
chkconfig
chkconfig httpd off
echo command
returns whatever is sent as input as output.
echo "Hello"
Copy Copied
returns Hello to output.
echo "Welcome" >file1
Copy Copied
redirects input Welcome to file1 which is created if it doesn't already exist.
echo "Wilkommen" >> file1
1
Copy Copied
appends input Wilkommne to file
grep command
used for searching strings anywhere , applies regular expression matching
sort command
sorts output in alphabetic ally ascending order
Demonstration
10 - Everything about Linux from Scratch (vii)
User control commands
useradd command - Creates user
useradd bhupinder
Copy Copied
creates a new user named bhupinder . The users/ usergroups are saved in /etc/passwd
groupadd command - Creates user group
groupadd techguftgu
Copy Copied
creates a new usergroup called techguftgu . The usergroups are saved in /etc/group
gpasswd command - Adds user to usergroup
gpasswd -a aman techguftgu```
## Shortcut(symbolic link) commands
`ln` command - creates link
```bash
ln file2 backupfile2
2
3
4
5
6
Copy Copied
creates a hard link backupfile2 for file2 while
ln -s file1 softfile1
Copy Copied
creates a soft link softfile1 for file1
A soft link is identical to a shortcut in windows. Even if it is deleted, the origin
al file is unaffected. A hard link creates
a complete backup copy of the given file. It,however, doesn't reflect changes made to origin
al file.
File archiving commands
tar command - an archiver tool used to combine multiple files into one.
gzip command
tar -cvf dirx.tar dirx
Copy Copied
the flag -c , -v and -f stand for creak , verbose and forcefully respectively. The command creates a tar file
dirx.tar from the directory dirx .
gzip dirx.tar
Copy Copied
Compresses the contents of dirx.tar into dirx.tar.gz
gunzip dirx.tar.gz
Copy Copied
tar -xvf dirx.tar
Copy Copied
The -x flag stands for extraction
wget command
Non-interactive downloader tool
Demonstration
Everything about Linux from Scratch
Access controls in Linux
Access mode File Directory
r (4) To display content To list content
w (2) To modify To create/remove
x (1) To execute the file To enter into directory
drwxr_xr___ 1 root root 0 July 01 04:00PM dirx
The first character d signifies that the given file is a directory. If it was a symbolic link, it would be l ; _ if it just a
file.
The next three characters define permissions for owner/root user , the next three for user group and the last three
for others.
The permissions are defined by three characters : r stands for readable, w stands for writable and x stands for
executable.
Then follows the owner of the file/directory and the user group of file/directory and then the file size in bytes.
chmod command - change access permissions of file/directory
1 chmod 536 dirx
changes permissions od dirx to 536 ( equivalent of r_x_wxr_x)
Another way to alter these permissions :-
1 chmod u+x,g-w,o=rwx file1
This will add execute permissions to user (if they already dont have it), remove write permissions from group (if
they have it) and set others' access control to rwx.
chown command - change owner of file/directory
1 chown bhupinder file1
chgrp command - change group of file/directory
1 chgrp main file1
11 - What is GIT /GitHub
Source Code Management is necessary to ensure efficient collaboration for software projects.
Before the era of version control, a manager had to individually monitor code assigned to several different
personnel.
There are two forms of code management :
1. Centealized Version Control System
2. Distributed Version Control System
Note: Git is just a software/tool . Github and Gitlab are tools built on top of
12 - Three stages of Git - Working directory , staging area and local
repository
Git workflow
1. Create your project directory
2. Run git init inside project repository. This makes the directory a local repository and makes a .git file
3. This logic ally (as opposed to physic ally) converts your directory into three divisions
4. Working directory
5. Staging area
6. Local repo
7. The code you write first gets saved in working directory.
8. Staging area is where premature code can be saved. It is not necessary but recommended for organised
workflow. This can be done with git add .
9. git commit will save a snapshot of code as it was and changes from last commit to local repo.
What is Github? - Not to be confused with Git, Github provides a central repository. It is owned by Microsoft with
free public repository storage.
Since GitHub was acquired by a major technology giant, GitLab was launched as a completely free alternative.
git push is used to push changes to local repository into remote repositories (like those on GitHub).
git pull is used to pull changes on remote repository to local repository.
A code repository may have multiple branches, the default branch is known as master branch.
Every commit created produces a Commit ID and record of changes made then. Everything is kept transparent for
all collaborators to see.
Repository
Repository is where all code is stored as a directory in server.
The folder is particular to the product
The Server stores all repositories and contains all metadata. E.g. Microsoft stores all GitHub data
All files can be physic ally seen and worked upon in the working directory/ workspace.
l
In other CVCS , there is no staging area but direct commit-ing only from
Commit ID -
Reference to identify each change.
Identifies who changed file.
Tags:
These assign a meaninful name with a specific version in the repository. Once a tag is created for a particular
save
Snapshots
Represens data state at particular time.
It is incremental: instead of storing entire data, only changes are stored
Push:
Push operations copies changes from local repository to a remote / central repo. Changes are stored
permanently into git repository.
Pull:
Pull operation copies changes from a remote repository to local machine
Branch
Branches can be used to divide the workflow by different tasks. E.g. someone is doing CSS styling and
backend dvelopment is handled separately. They can handle different branches, which can be merged later
when work is complete on both ends.
Each task can be given one separate branch
Finally all branches can be merged
Useful for parallel work.
Can create one branch through another branch.
Changes are personal to the particular branch.
Default branch is known as Master branch.
File created in workspace will be visib
le in any of the branches.
Once commited, changes will belong to the particular branch.
Advantages of Git
Free and Open ource
Fast and small in size
Effective Security
Powerful hardware is not necessary
Easy branching of code allows good compartmentalis ation
**Types of branches: **
1. Bare repositories
2. Non-bare repositories
Store and share only
A
Commit
Store changes in repository : This is asigned a Commit ID.
It consists of 40 alphanumeric characters
It uses SHA-1 checksum verification
Even if a single character is changed , Commit ID is altered as well.
Helps you track changes
Commit is also known as SHA-1 hash.
13 - How to install git on AWS Linux machine
14 - How to add, commit in git
There
15 - How to create branch in git
Branches enable parallel work separate from master branch.
A pro of paallel work is that master branch can be kept free from fatal errors which may occur whilst working on
other branches.
While we make a branch, all data is copied as it is on the main branch it split from.
When wanted, work done in the branch can be merged back into master/parent branch and work can further
continue.
We can make as many branches as per task requirements
File created in workspace is visib
le from all branches.
1 git branch
Shows list of all branches, and current branch
1 git log ---oneline
Shows log of commits in one line
1 git branch <branch-name>
Creates new branch with branch-name
1 git checkout <branch-name>
switch to branch-name
1 git branch -d <branch-name>
Delete branch-name branch
1 git merge <branch-name
Merge branch-name with master branch
1 git log
Verify message of git
1 git push origin master
Push to central repo like GitHub
Git conflict
When same files having different content in different branches are merged, a conflict emerges.
Git stash
Set aside files/changes for working upon later
1 git stash
stashes an item
1 git stash list
See stashed items
1 git stash apply stash @[0]
Apply stashed items
1 git stash clear
Clear stashed items
1 git reset
Demonstration
16 - What is git revert/ gitlab
17 - What is Chef in DevOps(i)
Used to be handled by system engineer earlier. Chef is used for automation of sysadmin tasks before,
eThere are two types of congifure management tools :-
1. Oush vased tools that use push configuration servers that pushes configuration o the nodes. Infrastructure as
code services. Server notifies about updates being rolled out.
Ansible and are push based tools
2. Pull based tools - Pull configuraion nodes. Reuces fault toreance.
E.g. Chef. Puppet
Toolused for devops engineer developed in ruby and erlang.
Used to be called Mario
nette.
Made open source in 2019
Automates all tasks done by sysadmins earlier
Configuration management tools automate all sysadmin tasks
Code becomes repeatable, testable and versionable.
-Advnatges of such tools
1. Complete automation
2. Reduced downtimes
3. Imorive performance
4. Ensure compliance
5. Prevent errors
6. Reduce costs
Chef architecture
Consists of:
1. Workstation - Code is worked on here. Code grouped together may be termed as recipes. Cookbook is a
collection of recipes.
2. Chef server - Cookbooks are stored here. It id connected to chef server by a CLI tool known as knife
3. Mode server -
**chef suoermarket ** contains several chef cookbooks.
0 Workstations are personal compyters
Devops engineers operate from chef server.
Midleman between node server and workstation.
Node are where codes are actually applied.
Ohai fetches current state of node.
18 - What is Chef in DevOps(ii)
How to create cookbook and Recipes
Create a Linux Machine in AWS
Use PuTTy
1 which chef
1 chef --version
check which version of chef you have
Cookbooks
It is a collection of recipes and some other files and folders
Contents of a cookbook:-
1. .chefignore - Similar to functions of .gitignore
2. Kitchenyml
3. Metadata.rb
4. Readme.md
5. Recipes
6. Specs
7. Tests -
1 chef generate cookbook
1 generate-recipe <name>
Ruby recipe code structure:
1 File do
2 content
3 action Create
4 end
Ruby recipe for software installation could be
1 package 'tree' do
2 action install
3 end
Chef does not overwrite anything if a recipe is run multiple times.
1 chef-client -zr "recipe[test-cookbook "recipe-name"]"
1 chef exec ruby -c <path of chef recipe>
Resource
Basic Component of a recipe used to manage the infrastructure with different kind of states. There can be multiple
resources in a recipe, which will help in configuring and managing infrastructure.
Some resources are:
1. Package - Manages packages on a node
2. Service - Manages services on a node
3. User - Manages users on the node
4. Group - Manages groups
5. Template - Manages file with embedded ruby template
6. Cokkbook.file - Transfers the files from the files subdirectory in the cookbook to a location on the node
7. File - Manages the content of a file on the node
8. execute - executes a command on the node
9. cron - Edit an existing cron file on the node
10. directory - manages directory on a node
Demonstration
57c5945428bb341b0946f2c7ad5837f2.png
5412f89fcac8e1aa16eb394dd36536ba.png
19 - What is attribute in CHEF
Helps distinguish features of several different node servers running.
Attribute is a key-value pair whch represents a specific detail about a node
An attribute can help us find out
Current state of node
What the state of a node was after the end of previous chef-client run
what state of a node should be at the end of the current chef-client run
Types of attribute (n ncasing priorty)
default
force=default
normal
Override
force-override
automatic
Attributes defined by Ohai have the highest priorty, followed by attributes , followed by those defined by attribute
files
20 - Chef recipes
21 - Bootstrapping in CHEF
22 - Roles in CHEF
23 - Introduction to Docker
24 - OS Level virtualisation in Docker
Docker is an open source centralised platform
Docker perofmrs OS-level virtualis ation while VMware performs hardware-level virtualis ation
Virtualisation in VMWare schema : Hypervisor is built upon Hardware. The VM takes all resources from
hardware.
Containerization in Docker : Docker is installed on top of host OS. Based upon containers rather than virtual
machines which contain UNIX based operating systems. Container images can be pulled from an online repository
called Docker Hub. Most code of UNIX like distros are similar, the container-specific code is obtained from Docker
Hub. A container is very light weight compared to a virtual machine as only marginal operating system files
necessary for booting and running the system is required.
25 - Docker - architecture, pros and cons
Advantages of Docker:
No pre-allocation of RAM - Unlike virtual machines. When n image is run, it is referred to as a container.
Increased efficiency of Continuous Integration - Docker removes dependency on platform and hardware
Lower costs - Since there is no pe-allocation of RAM
Light weight
Can
In order to run Docker on windows , you will need to enable virtualis ation and hypervisor. Docker is mainly
designed for Linux systems.
Changes to a container can be saved to images.
Disadvantages of Docker
Not great for cross-platform compatib
ility.
Not a good solution for applications requiring GUI
Suitable when dveloping OS and testing OS is same
Docker architecture
A dockerfile plays the same role which is played by cookbook in Chef. The dockerfile makes an image and installs
dependencies and requirements as mentioned. A container is produced from the image. The container may be
published to Docker Hub (a.k.a. registry) as an image. It can be pulled by other users as required.
A container has a layered file system, which has its processes run layer-by-layer.
Docker ecosystem
1. Docker Client - The end user of Docker
2. Docker daemon - executes commands and produces images
3. Docker Hub - Remote storage for keeping images
4. Docker Images - templates for running containers
5. Docker compose - Managing and running several images at once
Docker Host - used to provide an environment to execute and run applications . Contains the whole docker
ecosystem.
Docker registry manages and stores docker images.There are two types of registry in Docker:
1. Public Registry
2. Private registry - For use within enterprise
Docker Images are binary read-only templates required to run containers.
There are three ways to create images:
1. Take image from Docker Hub
2. Create your own image from Dockerfile
3. Create image from existing docker container
Docker Container - a copy of the image it was created from. Hold all essential packages in order to run an
application.
26 - Using docker - running/stopping containers
27 - Dockerfile creation
docker diff command
1 docker diff <Container1> <Container2>
Find the difference between <Container1> and <Container2>
Outoput shown shows three types of changes:-
1. C (Change)
2. A (Append)
3. D (Delete)
1 docker commit <ContainerName> <ImageName>
Creates new image of container.
1 docker run -it --name <ContainerName> <UpdatedContainer> /bin/bash
Dockerfile
A text file containing a set of instructions
Automates docker image creation
Docker componenets
1. FROM
Base image on top of which image is being created. A dockerfile always starts with this
2. RUN
Executes commands. This also produces layers in image
3. MAINTAINER
Provides information about author and contributors
4. COPY
Copy files from local system to container
5. ADD
Similar function to COPY , but also capable of downloading off internet and unpacking/extracting archived
directories
6. EXPOSE
Exposes ports for networking
7. WORKDIR
Set work directory for container
8. CMD
Execute commands during container creation
9. ENTRYPOINT
Like CMD, but with higher priorty. Ensures sequence of commands being executed
10. ENV
set environment variables
1 docker build
Used for building images from dockerfile
Demo
28 - Docker Volume
Volume is just a directory where container runtime information may be persistently stored. It enables sharing of
information among multiple containers.
A directory is just declared as a volume, then the volume can be shared
Even if container is stopped, volume is accessible.
Information from a container stored in a volume can be shared with any container
Volumes can't be created from existing containers
Volume is not included when an image is updated.
Volumes can be mapped in two ways: Container <-> Container, Host <-> Container
Volume will act as simply just another directory if it is not linked up with concerned container.
Benefits of using volumes
Decoupling containers from storage
Share volume among different containers
Attach volumes to container
Upon deleting containers, volumes are not deleted
1 docker
29 - Docker port mapping, Docker expose
IP addressing basics
Every networking process has a port assigned to it to uniquely identify it.
Docker IP networking
Ports can be mapped from host to containe
To accomplish this, certain ports of a container are set as exposed
1 docker run -td --name techserver -p 80:80 Ubuntu
would map port number 80 of host to the techserver ubuntu container.
1 docker port techserver - 80
2
3
4 # 30 - What is docker hub and how it works
5
6 # 31 - Ansible introduction and work
ing
7
8 # 32 -
9
10 # 33 -
11
12 # 34 -
13
14 # 35 -
15
16 # 36 -
17
18 # 37 -
19
20 # 38 - Jenkins and Maven basics
21
22 # 39 -
23
24 # 40 -
25
26 # 41 -
27
28 # 42 -
29
30 # 43 - Nagios Continuous Monitoring Tool
31
32 # 44 - Nagios setup on Linux
33
34 # 45 - Kubernetes introduction
35
36 # 46 - Kubernetes architecture
37
38 # 47 -
39
40 # 48 -
41
42 # 49 -
43
44 # 50
45
46 # 51 - Kubernetes Networking
47
48 # 52 -
49
50 # 53 -
51
52 # 54 -
53
54 # 55 -
55
56 # 56 -
57
58 # 57 - What is Helm | HELM chart
59
60 # 58 - Helm installation and com
mands
