INDERPRASTHA ENGINEERING COLLEGE

63 Site IV, Sahibabad Industrial Area,

Surya Nagar Flyover Road Sahibabad, Ghaziabad – UP

Department of Electronics & Communication Engineering

Vision of the Department: To be recognized as a Center of Excellence in Electronics and Communication Engineering by providing
valuable resources to the students for the purpose of nurturing their knowledge and skills to serve the nation by solving the
technological problems of modern society in the field of Electronics and Communication.

LIST OF STUDENTS
S.NO
. ROLLNO STUDENT NAME
1 1703031062 MANSI GOEL
2 1703031063 MAYANK BHATIA
3 1703031064 MAYANK LAKHERA
4 1703031065 MD FARHAN
5 1703031066 MUKUL
6 1703031067 MUSTAQUEEMUDDIN
7 1703031068 NAVDEEP GUPTA
8 1703031069 NELSON DIVYAM LOBO
9 1703031070 NIKHIL GUPTA
10 1703031071 NISHANT RANA
11 1703031072 NISHU TYAGI
12 1703031073 NITISH KUMAR
13 1703031074 OJASHVI GUPTA
14 1703031075 PIYUSH PURI
15 1703031076 PRANSHU SHARMA
16 1703031077 PRAVIGYA SRIVASTAVA
17 1703031078 PRIYAM BAJAJ
18 1703031079 PRIYANSHU JAYANT
19 1703031080 PULKIT SINGH
20 1803031901 ASHISH KUMAR SHARMA (DIPL-18)
21 1703031083 RAHUL RANA
22 1703031085 RAMA PANDEY
23 1703031086 RENU ANAND
24 1703031088 RISHABH SHUKLA
25 1703031089 RITESH PAL SINGH
26 1703031091 SHIVAM GUND
27 1703031092 SHIVAM TYAGI
28 1703031094 SHRADDHA GOEL
29 1703031095 SHRUTI SHARMA
30 1703031096 SHUBHAM JAIN
31 1703031097 SHUBHAM SHARMA
32 1703031098 SIMRAN SOLANKI
33 1703031099 SPARSH VIJ
34 1703031100 SRAVIKA SINGH
35 1703031101 SUMIT BISHT
36 1703031102 SUMIT
37 1703031103 SUNIDHI GOEL
38 1703031104 SUNNY JAIN
39 1703031105 SURYA KANT CHOUDHARY
40 1703031106 SUSHANT GARG
41 1703031108 TARUN JAIN
42 1703031110 UDIT AGGARWAL
43 1703031111 UTSAV BABULKAR
 INDERPRASTHA ENGINEERING COLLEGE
63 Site IV, Sahibabad Industrial Area,
Surya Nagar Flyover Road Sahibabad, Ghaziabad – UP

Department of Electronics & Communication Engineering

Vision of the Department: To be recognized as a Center of Excellence in Electronics and Communication Engineering by providing
valuable resources to the students for the purpose of nurturing their knowledge and skills to serve the nation by solving the
technological problems of modern society in the field of Electronics and Communication.

44 1703031112 VAIBHAV RUSTAGI

45 1703031113 VAIBHAV SHARMA
46 1703031114 VAIBHAV TYAGI
47 1703031115 VANSHIKA GUPTA
48 1703031116 VIKAS CHAUHAN
49 1703031117 VINI JAIN
50 1703031118 VISHESH GOYAL
51 1703031119 VISHESH JAIN
52 1703031121 YASHASWI VERMA
53 1703031122 YASHVI GUPTA

Sign of Faculty Signature of

HOD
 INDERPRASTHA ENGINEERING COLLEGE
63 Site IV, Sahibabad Industrial Area,
Surya Nagar Flyover Road Sahibabad, Ghaziabad – UP

Department of Electronics & Communication Engineering

Vision of the Department: To be recognized as a Center of Excellence in Electronics and Communication Engineering by providing
valuable resources to the students for the purpose of nurturing their knowledge and skills to serve the nation by solving the
technological problems of modern society in the field of Electronics and Communication.

REC-071 Information Theory & Coding 3L:1T:0P 4 Credits

Unit Topics Lectures

I Entropy: Entropy, Joint Entropy and Conditional Entropy, Relative Entropy 8
and Mutual Information, Relationship Between Entropy and Mutual
Information, Chain Rules for Entropy, Relative Entropy, and Mutual
Information, Jensen’s Inequality and Its Consequences, Log Sum Inequality
and Its Applications, Data-Processing Inequality, Sufficient Statistics,
Fano’s Inequality.
II Asymptotic Equipartition Property: Asymptotic Equipartition Property 8
Theorem,
Consequences of the AEP: Data Compression, High-Probability Sets and
the Typical Set.
Data Compression: Examples of Codes, Kraft Inequality, Optimal Codes,
Bounds on the Optimal Code Length, Kraft Inequality for Uniquely
Decodable Codes, Huffman Codes, Some Comments on Huffman Codes,
Optimality of Huffman Codes, Shannon–Fano–Elias Coding.
III Channel Capacity: Examples of Channel Capacity, Symmetric Channels, 8
Properties of Channel Capacity, Preview of the Channel Coding Theorem,
Definitions, Jointly Typical Sequences, Channel Coding Theorem.
IV Block Codes: Digital communication channel, Introduction to block codes, 8
Single-parity check codes, Product codes, Repetition codes, Hamming
codes, Minimum distance of block codes, Soft-decision decoding,
Automatic-repeat-request schemes.
Linear codes: Definition of linear codes, Generator matrices, Standard array,
Parity-check matrices, Error.
V Convolution codes: Encoding convolutional codes, Generator matrices for 8
convolutional codes, Generator polynomials for convolutional codes,
Graphical representation of convolutional codes, Viterbi decoder.

Text /Reference Books:

1. Bose, Information Theory, Coding and Cryptography, McGraw hill Education.
2. Joy A. Thomas, Thomas M. Cover, “Elements of information theory”, Wiley-Interscience;
2 edition (July 18, 2006)
3. S. Gravano, “Introduction to Error Control Codes” OUP Oxford (24 May 2001)
4. Robert B. Ash, “Information Theory”, Dover Publications (November 1, 1990)
5. Todd k Moon, “Error Correction Coding: Mathematical Methods and Algorithms” Wiley,
2005.
Course Objective:
1. To learn basic of Entropy. 2. To learn Asymptotic Equipartition Property. 3. To learn Channel Capacity. 4.
To learn the implementation of Block Codes 5. To learn the Convolution codes
 INDERPRASTHA ENGINEERING COLLEGE
63 Site IV, Sahibabad Industrial Area,
Surya Nagar Flyover Road Sahibabad, Ghaziabad – UP

Department of Electronics & Communication Engineering

Vision of the Department: To be recognized as a Center of Excellence in Electronics and Communication Engineering by providing
valuable resources to the students for the purpose of nurturing their knowledge and skills to serve the nation by solving the
technological problems of modern society in the field of Electronics and Communication.

Mapping of Course outcomes with Program

Outcome and Program Specific Outcome

Information Theory & coding (REC-071) Year of Study: 2020-21

Course Outcome

CO1
Model the Entropy, Joint Entropy and Conditional Entropy, Relative Entropy and Mutual Information,
Relationship Between Entropy and Mutual Information.
CO2
Design Data Compression, Examples of Codes, Kraft Inequality, Optimal Codes, Bounds on the Optimal Code
Length.
CO3
Identify the Examples of Channel Capacity, Symmetric Channels, Properties of Channel Capacity, Preview of
the Channel Coding Theorem.
CO4
Analyze Introduction to block codes, Single-parity-check codes, Product codes, Repetition codes, Hamming
codes.
CO5 Design Generator matrices for convolutional codes, Generator polynomials for convolutional codes.

Information Theory & coding (REC-071) Year of Study: 2020-21

CO PO1 PO2 PO3 PO4 PO5 PO6 PO7 PO8 PO9 PO10 PO11 PO 12
CO1 2
CO2 3 3 3 2
CO3 3 3 2
CO4 3 3 3 3
CO5 3 3 3 3
CO 3 3 3 3

Information Theory & coding (REC-071) Year of Study: 2020-21

CO PSO1 PSO2 PSO3
CO1 2
CO2 3
CO3 3
CO4 3
CO5 3
CO 3
 INDERPRASTHA ENGINEERING COLLEGE
63 Site IV, Sahibabad Industrial Area,
Surya Nagar Flyover Road Sahibabad, Ghaziabad – UP

Department of Electronics & Communication Engineering

Vision of the Department: To be recognized as a Center of Excellence in Electronics and Communication Engineering by providing
valuable resources to the students for the purpose of nurturing their knowledge and skills to serve the nation by solving the
technological problems of modern society in the field of Electronics and Communication.

LECTURE PLAN

B.Tech (ECE) – VII SEM

SUBJECT: Information Theory & coding (CODE: REC-071)

Total
Name of Date of Date of
SEM SECTION Name of Faculty Lectures
Course Commencement Conclusion
Planned

Information VII B Mr. Vishal Gupta 04/08/2020

Theory & coding

PREREQUISITE
1. Basic concept of Probability,
2. Concept of digital communication etc.
Lecture Reference Proposed Mode of
Contents Actual Date
No. Books Date Delivery

UNIT#4

Lecture 1 Introduction of Syllabus Text-Book 2 04/08/2020

Lecture 2 Digital communication channel Text-Book 2 05/08/2020

Lecture 3 Introduction to block codes Text-Book 2 06/08/2020

Lecture 4 Single-parity-check codes Text-Book 2 10/08/2020

Lecture 5 Product codes Text-Book 2 12/08/2020

 INDERPRASTHA ENGINEERING COLLEGE
63 Site IV, Sahibabad Industrial Area,
Surya Nagar Flyover Road Sahibabad, Ghaziabad – UP

Department of Electronics & Communication Engineering

Vision of the Department: To be recognized as a Center of Excellence in Electronics and Communication Engineering by providing
valuable resources to the students for the purpose of nurturing their knowledge and skills to serve the nation by solving the
technological problems of modern society in the field of Electronics and Communication.

Lecture 6 Repetition code, Hamming codes Text-Book 2 13/08/2020

Minimum distance of block codes,

Lecture 7 Text-Book 2 17/08/2020
Soft-decision decoding

Lecture 8 Automatic repeat request schemes Text-Book 2 18/08/2020

Lecture 9 Definition of linear codes Text-Book 2 19/08/2020

Generator matrices, Standard

Lecture 10 Text-Book 2 20/08/2020
array

Lecture 11 Parity check matrices Text-Book 2 24/08/2020

Lecture 12 Error Text-Book 2 25/08/2020

Assignment 1

UNIT#5

Convolution codes: Encoding

Lecture 13 Text-Book 2 26/08/2020
convolutional codes

Generator matrices for

Lecture 14 Text-Book 2 27/08/2020
convolutional codes

Generator polynomials for

Lecture 15 Text-Book 2 31/08/2020
convolutional codes

Graphical representation of
Lecture 16 Text-Book 2 02/09/2020
convolutional codes

Lecture 17 Viterbi Decoder Text-Book 2 07/09/2020

 INDERPRASTHA ENGINEERING COLLEGE
63 Site IV, Sahibabad Industrial Area,
Surya Nagar Flyover Road Sahibabad, Ghaziabad – UP

Department of Electronics & Communication Engineering

Vision of the Department: To be recognized as a Center of Excellence in Electronics and Communication Engineering by providing
valuable resources to the students for the purpose of nurturing their knowledge and skills to serve the nation by solving the
technological problems of modern society in the field of Electronics and Communication.

Assignment 2

UNIT#1

Entropy, Joint Entropy and

Lecture 18 Text-Book 1 14/09/2020
Conditional Entropy

Relative Entropy and Mutual

Lecture 19 Text-Book 1 15/09/2020
Information

Relationship Between Entropy and

Lecture 20 Text-Book 1 16/09/2020
Mutual Information

Lecture 21 Chain Rules for Entropy Text-Book 1 17/09/2020

Relative Entropy, and Mutual

Lecture 22 Text-Book 1 21/09/2020
Information

Jensen’s Inequality and Its

Lecture 23 Text-Book 1 22/09/2020
Consequences

Log Sum Inequality and Its

Lecture 24 Text-Book 1 23/09/2020
Applications

Lecture 25 Data-Processing Inequality Text-Book 1 24/09/2020

Sufficient Statistics, Fano’s

Lecture 26 Text-Book 1 29/09/2020
Inequality

Assignment 3

UNIT#2

Lecture 27 Examples of Codes Text-Book 1 01/10/2020

 INDERPRASTHA ENGINEERING COLLEGE
63 Site IV, Sahibabad Industrial Area,
Surya Nagar Flyover Road Sahibabad, Ghaziabad – UP

Department of Electronics & Communication Engineering

Vision of the Department: To be recognized as a Center of Excellence in Electronics and Communication Engineering by providing
valuable resources to the students for the purpose of nurturing their knowledge and skills to serve the nation by solving the
technological problems of modern society in the field of Electronics and Communication.

Lecture 28 Kraft Inequality Text-Book 1 05/10/2020

Lecture 29 Optimal Codes Text-Book 1 06/10/2020

Bounds on the Optimal Code

Lecture 30 Text-Book 1 07/10/2020
Length

Kraft Inequality for Uniquely

Lecture 31 Text-Book 1 08/10/2020
Decodable Codes

Huffman Codes, Some Comments

Lecture 32 Text-Book 1 12/10/2020
on Huffman Codes

Optimality of Huffman Codes,

Lecture 33 Text-Book 1 13/10/2020
Shannon–Fano–Elias Coding

Asymptotic Equipartition Property

Lecture 34 Text-Book 1 14/10/2020
Theorem

Data Compression, High -

Lecture 35 Probability Sets and the Typical Text-Book 1 15/10/2020
Set

Assignment 4

UNIT#3

Lecture 36 Examples of Channel Capacity Text-Book 1 19/10/2020

Lecture 37 Properties of Channel Capacity Text-Book 1 20/10/2020

Preview of the Channel Coding

Lecture 38 Text-Book 1 21/10/2020
Theorem

Definitions, Jointly Typical

Lecture 39 Text-Book 1 22/10/2020
Sequences
 INDERPRASTHA ENGINEERING COLLEGE
63 Site IV, Sahibabad Industrial Area,
Surya Nagar Flyover Road Sahibabad, Ghaziabad – UP

Department of Electronics & Communication Engineering

Vision of the Department: To be recognized as a Center of Excellence in Electronics and Communication Engineering by providing
valuable resources to the students for the purpose of nurturing their knowledge and skills to serve the nation by solving the
technological problems of modern society in the field of Electronics and Communication.

Lecture 40 Channel Coding Theorem Text-Book 1 28/10/2020

Assignment 5
 INDERPRASTHA ENGINEERING COLLEGE
63 Site IV, Sahibabad Industrial Area,
Surya Nagar Flyover Road Sahibabad, Ghaziabad – UP

Department of Electronics & Communication Engineering

Vision of the Department: To be recognized as a Center of Excellence in Electronics and Communication Engineering by providing
valuable resources to the students for the purpose of nurturing their knowledge and skills to serve the nation by solving the
technological problems of modern society in the field of Electronics and Communication.

Topics Beyond Syllabus

Execution
UNIT NO Topic References Planned Date Remark
Date

PEDAGOGY:
Sessions will be interactive, rich with practical problem solving approaches. Students are required to understand
the real life scenario with parallel approach
EVALUATION PARAMETERS
Class Test (CT) Teacher’s Assessment End Sem Total
Attendance (AT) [10]
[30] (TA)[10] Exam Marks
1. Assignments (5)
Sessional-I – 15 Based on Class Attendance
2. Class participation (2) 100 150
Sessional-II - 15 ( As perUniversity Norms)
3. Quiz (1)
Text Books :
T1: Joy A. Thomas, Thomas M. Cover, “Elements of information theory”, Wiley-Interscience; 2 edition (July 18,
2006).
T2: S. Gravano, “Introduction to Error Control Codes” OUP Oxford (24 May 2001).
Reference Books:
R1: Bose, Information Theory, Coding and Cryptography, McGraw hill Education.
R2: Robert B. Ash, “Information Theory”, Dover Publications (November 1, 1990).
Web and other Reference:
1. NPTEL
2. Mitopencourseware