TOC 1 | Introduction to Theory of Computation

Introduction to TOC
CSE 2233
Mohammad Imam Hossain | Lecturer, Dept. of CSE | UIU

Introduction
2
Mathematics:
What can be computed?
And what can’t be computed?
Electrical Engineering:
How can we build computers?
Computer Science
TOC - Comprises the fundamental
mathematical properties of computer HW,
SW, and other applications.

Theory of Computation - TOC
Theory of Computation tries to answer the following questions:
• What are the mathematical properties of computer hardware and software?
• What is a computation and what is an algorithm? Can we give rigorous mathematical definitions of
these notions?
• What are the limitations of computers? Can everything be computed?
Purpose of TOC >> Develop formal mathematical models of computation that reflect real-world computers.
Our main purpose is to determine:
• what can and can’t be computed?
• how quickly?
• with how much memory?
• on which type of computational model?
3

Central Areas - TOC
Theory of Computation has 3 central areas:
▸ Complexity Theory
▸ Computability Theory
▸ Automata Theory
They are linked by the question:
What are the fundamental capabilities and limitations of computers?
In each of the three areas
— Automata, Computability, and Complexity —
this question is interpreted differently, and the answers vary according to the interpretation.
4

1 - Complexity Theory
- Though it has been intensively researched for more than 40 years but we don’t know the answer to it.
- One important achievement by exploring this question and some of its ramifications:
Researchers have discovered an elegant scheme for classifying problems according to their
computational difficulty(easy or hard) even if there exists no proof.
▸ Informally, a problem is called “easy” if it is efficiently solvable.
Ex - sorting a sequence of , say, 1000000 numbers
▸ On the other hand, a problem is called “hard”, if it can’t be solved efficiently, or if we don’t know
whether it can be solved efficiently.
Ex - factoring a 300 digit integer into its prime factors
Motivation of Complexity Theory >> Classify problems according to their degree of “computational
difficulty” by giving a rigorous proof that certain problems that seem to be “hard” are really “hard”.
5
What makes some problems computationally hard and other problems easy?

2 - Computability Theory
In the 1930’s Gödel, Turing, and Church discovered that some of the fundamental mathematical
problems can’t be solved by a “computer”.
Ex - Determining whether a mathematical statement is true or false!!
It seems like a natural for solution by computer but no computer algorithm can perform this task.
The theoretical models that were proposed in order to understand solvable and unsolvable problems led
to the development of real computers.
Motivation of Computability Theory >> Classify problems as being solvable or unsolvable.
6
Which problems can be solved by computers and which ones cannot?

3 - Automata Theory
Automata Theory deals with the definitions and properties of different types of “mathematical models of
computation”
For example,
▸ Finite Automata - These are used in text processing, compilers, and hardware design.
▸ Context-Free Grammars - These are used to define programming languages and in Artificial
Intelligence.
▸ Turing Machines - These form a simple abstract model of a “real” computer, such as your PC at
home.
Motivation of Automata Theory >> Determine the power of different computational models or, which
model can solve more problems than the other.
7

TOC - History
8
1930s • Alan Turing studies Turing Machines
• Decidability
• Halting problem
1940-1950s • Finite automata machines studied
• Noam Chomsky proposes the “Chomsky Hierarchy” for formal languages
1969 • Cook introduces “intractable” problems or “NP-Hard” problems
1970 - • Modern computer science: compilers, computation and complexity theory
evolve

Alan Turing - Facts
9
Dr. Alan Turing(1912-1954) is one of the
founders of Computer Science (he was an
English Mathematician).
Important facts:
▸ “Invented” Turing machines.
▸ “Invented” the Turing Test.
▸ Broke the German submarine transmission
coding machine “Enigma”.
▸ Was arraigned for being gay and committed
suicide soon after.

Why this course?
▸ This course is about the fundamental capabilities and limitations of computers. These topics form the
core of computer science.
▸ It is about mathematical properties of computer hardware and software.
▸ To design new programming languages, compilers, string searching, pattern matching, computer
security, artificial intelligence etc. this course will provide you the basics.
▸ This course helps you to learn problem solving skills. Theory teaches you how to think, prove, argue,
solve problems, express, and abstract.
▸ This theory simplifies the complex computers to an abstract and simple mathematical model, and
helps you to understand them better.
▸ This course is about rigorously analyzing capabilities and limitations of systems.
10

Terminology – Mathematical
Sets - Set, element (a.k.a member), subset, proper subset, power set, multiset, infinite set, natural
numbers, integers, empty set, union of sets, intersection of sets, complement of a set, Venn diagrams.
Sequences and Tuples - Sequence, k-tuple, ordered pair, Cartesian product (a.k.a. cross product).
Functions and Relations - Function (a.k.a. mapping), domain, range, onto function, arguments of a
function, k-ary function, unary function, binary function, infix notation, prefix notation, predicate (a.k.a.
property), relation, k-ary relation, binary relation, equivalence relation (reflexive, symmetric and
transitive).
11

Terminology – Graphs , Boolean
Graphs - Undirected graph, nodes (a.k.a. vertices), edges, degree of a node, labeled graph, subgraph,
path, simple path, connected graph, cycle, simple cycle, tree, root of a tree, leaves of a tree, directed
graph, outdegree, indegree, directed path, strongly connected graph.
Boolean Logic - Boolean logic, boolean values (True and False), boolean operations, negation (a.k.a.
NOT), conjunction (a.k.a. AND), disjunction (a.k.a. OR), operands of an operation, exclusive OR (a.k.a
XOR), equality, implication, distributive law.
12

Terminology – Strings
▸ An alphabet is a finite, nonempty set of symbols.
Conventionally, we use the symbol Σ for an alphabet. For example,
i. Σ = 0, 1 , the binary alphabet.
ii. Σ = { 𝑎, 𝑏, 𝑐, … … … , 𝑧}, the set of all lower-case letters.
▸ A string over an alphabet Σ is a finite sequence of symbols, where each symbol is an element of Σ
- The string w where w = w1w2w3 … … wn represents a string of length n where each wi ϵ ∑
- The reverse of w is represented as wR
- The length of w denoted by |w|, is the number of symbols contained in w.
- The empty string, denoted by 𝜀, is the string having length zero.
Ex: if the alphabet Σ is equal to {0, 1}, then 10, 1000, 0, 101, and 𝜀 are strings over Σ, having
lengths 2, 4, 1, 3, and 0 respectively.
13

Concatenation of Strings:
Let x and y be strings. Then xy denotes the concatenation of x and y, that is, the string formed
by making a copy of x and following it by a copy of y.
More precisely, if x is the string composed of i symbols x = a1a2. . .ai and y is the string composed of j
symbols y = b1b2. . .bj , then xy = a1a2. . .aib1b2. . .bj
For example,
if x = 01101 and y = 110, then xy = 01101110
if x = 01101, then 𝜀x = x𝜀 = x holds.
14

Powers of an Alphabet: Σ 𝑘
is the set of strings of length exactly k, each of whose symbols is in Σ
For example,
Σ0
={𝜀} regardless of Σ
if Σ = { 0, 1 }, then
Σ1
={ 0, 1 }
Σ2
={ 00, 01, 10, 11 }
Σ3
={ 000, 001, 010, 011, 100, 101, 110, 111 }
The set of all strings over an alphabet Σ is conveniently denoted as Σ∗
, where
Σ∗
= Σ0
∪ Σ1
∪ Σ2
∪ Σ3
∪ …
For example, { 0, 1 }* = { 𝜀, 0, 1, 00, 01, 10, 11, 000, … }
The set of all strings except the empty string is denoted as Σ+
, where
Σ+
= Σ1
∪ Σ2
∪ Σ3
∪ …
15

Terminology – Languages
Language:
A set of strings all of which are chosen from some Σ∗
, where Σ is a particular alphabet, is
called a language.
If Σ is an alphabet, and L ⊆ Σ∗
, then L is a language over Σ
For example,
▸ Language, L
= set of strings of 0’s and 1’s with an equal number of each
= {𝜀, 01, 10, 0011, 0101, 1001, 0110, … … }
▸ Σ∗
is a language for any alphabet Σ
▸ ∅ , the empty language, is a language over any alphabet
▸ {𝜀}, the language consisting of only the empty string. Notice that, ∅ ≠ {𝜀}
16

Terminology – Definition, Theorems, and Proofs
Definition - It describes the objects and notions that we use.
Mathematical Statement - It expresses that some object has a certain property.
Proof - It is a convincing logical argument that a statement is true.
Theorem - It is a mathematical statement proved true.
Lemma - Proved mathematical statements that assist in the proof of another more significant statement.
Corollaries - The statements easily concluded from a theorem or its proof.
17

Terminology – Types of Proofs
Proof by Construction:
Many theorem state that a particular type of object exists. One way to prove such a theorem is
by demonstrating how to construct the object. This technique is a proof by construction.
Proof by Contradiction:
First we will assume that the theorem is false and then show that this assumption leads to an
obviously false consequence, called a contradiction.
Proof by Induction:
Useful for proving statements for infinite sets. It consists of two parts: the basis and the
induction step. Each part is an individual proof on its own.
• The basis proves that P(1) is true.
• The induction step proves that for each i>=1, if P(i) is true then so is P(i+1).
18

19
THANKS!
Any questions?
You can find me at imam@cse.uiu.ac.bd
References:
Chapter 1, Introduction to the Theory of Computation, 3rd Edition by Michael Sipser

TOC 1 | Introduction to Theory of Computation

More Related Content

What's hot

Similar to TOC 1 | Introduction to Theory of Computation

More from Mohammad Imam Hossain

Recently uploaded

In this document

TOC 1 | Introduction to Theory of Computation