Lecture 01
Introduction to Natural
Language Processing (NLP)
Natural Language Processing
COSC-3121
Ms. Humaira Anwer
humaira.anwer@kfueit.edu.pk
Lecture 01 Introduction to NLP 1
Introduction About Instructor
• Ms. Humaira Anwer
• MS (CSE) - College of E&ME, NUST, Islamabad.
• PhD (CSE)- College of E&ME, NUST, Islamabad. (on-
going)
• Can be reached at:
• Humaira.anwer@kfueit.edu.pk
• Student Meeting Hours (Mon-Fri 0800 Hrs. to 1000 Hrs. )
Lecture 01 Introduction to AI 2
Evaluation Criteria
• Mid Term 25%
• Final Term 50%
• Sessional 25%
–Quizzes 10%
–Assignments 5%
–Term Project/Presentations 10%
– Class behavior
• Attendance must be maintained as per rule (>=75%)
Lecture 01 Introduction to NLP 3
Ground Rules
• Attendance in lecture is compulsory
• If you decide to attend the lecture then:
―Do not talk with your friends during lecture
―Switch off your mobiles!!
―Ask Relevant questions but please put your hand up
―No Plagiarism Whatsoever!!!
Lecture 01 Introduction to NLP 4
Agenda
• Introduction to NLP
• History of NLP
• Forms of Natural Language
• Challenges in Developing NLP
• NLP an Interdisciplinary Field
• Buzz Words in NLP
• NLP Applications
Lecture 01 Introduction to NLP 5
Recommended Book
Speech and Language
Processing 3rd Edition
Danial Jurafsky
James H. Martin
Practical Implementation & Reference Book
Natural Language
Processing with Python
Steven Bird
Ewan Klein
Edward Loper
Lecture 01 Introduction to NLP 6
Introduction to NLP
• Subfield of Computer Science
• Subfield of Artificial Intelligence
• Subfield of Linguistics
• Enables computers to understand and process human
language
Lecture 01 Introduction to NLP 7
Natural Language Processing
AI Linguistics NLP
• HUMAN-COMPUTER INTERACTION
• Goal- “to achieve human-like language processing.”
• Encompasses the automation of linguistic forms, activities
or methods of communication.
• Presently implemented using machine learning algorithms
• Previously-Decision trees
• Now-Statistical methods
• Future-Unsupervised, Semi-supervised algorithms
Processing
Communicate
Lecture 01 Introduction to NLP 8
Natural Language Processing
• Natural Language Processing (NLP) refers to:
• AI method of communicating with intelligent
systems using a natural language such as English
• The field of NLP involves
making computers to perform
useful tasks with the natural
languages humans use.
Lecture 25- Natural Language Processing 9
Introduction to NLP
Definition:
“Natural Language Processing (NLP) is the sub-field of
Computer Science especially Artificial Intelligence (AI) and
Computational Linguistics that is concerned about enabling
computers to understand and process human language.”
Lecture 01 Introduction to NLP 10
Natural Language Processing
• Processing of Natural Language is required:
• When you want an intelligent
system like robot to perform
as per your instructions,
• When you want to hear
decision from a dialogue
based clinical expert system,
etc.
Lecture 25- Natural Language Processing 11
History of NLP
• Four Phases:
• First Phase- Machine Translation Phase- Late 1940s-Late 1960s
• Second Phase- AI Influenced Phase- Late 1960s-Late 1970s
• Third Phase- Grammatico Logical Phase- Late 1970s-Late 1980s
• Fourth Phase- Lexical & Corpus Phase- the 1990s
Lecture 01 Introduction to NLP 12
First Phase (Late 1940s to late 1960s)
• Machine Translation(MT) Phase
• This phase was a period of enthusiasm and optimism.
• Late 1940s to late 1960s
• Weaver’s memorandum-1949
• The first international conference on Machine
Translation (MT)-1952
• Georgetown-IBM experiment-1954
• Publication of the Journal MT-1954
• Second International Conference on MT-1956.
Lecture 01 Introduction to NLP 13
Second Phase (Late 1960s to late 1970s)
• Also known as AI-flavoured phase
• Work done was majorly related to world knowledge
• Focused mainly on problems of addressing and
constructing data or Knowledge Base (KB)
• BASEBALL question-answering system was developed
• Restricted inputs
• Simple language processing
Lecture 01 Introduction to NLP 14
Third Phase ( Late 1970s to late 1980s)
• Also known as grammatico-logical phase
• Focused on use of logic for knowledge
representation and reasoning in AI.
• Development of SRI’s Core Language Engine
• A powerful general-purpose sentence processor
• Development of some practical resources & tools
• Parsers
• Alvey Natural Language Tools
Lecture 01 Introduction to NLP 15
Fourth Phase (The 1990s)
• Lexical & corpus phase
• Introduction of machine learning algorithms for
language processing
Lecture 01 Introduction to NLP 16
I/O of NLP
• NLP includes both algorithms that take:
• human-produced text as input, and
• produce natural looking text as outputs.
• The input and output of an NLP system can
be:
• Speech
• Written Text
Lecture 25- Natural Language Processing 17
Forms of Natural Language
• We will mostly concerned with written text (not
speech).
• To process written text, we need:
• lexical, syntactic, semantic knowledge about the
language
• discourse information, real world knowledge
• To process spoken language, we need everything
required to process written text, plus the
challenges of speech recognition and speech
synthesis.
Lecture 25- Natural Language Processing 18
Challenges in Developing NLP
• Computers need structured data
• Human speech is unstructured
• Human speech is ambiguous in nature.
Lecture 01 Introduction to NLP 19
NLP - an inter-disciplinary Field
• NLP borrows techniques and insights from several
disciplines.
• Linguistics: How do words form phrases and sentences?
What constraints the possible meaning for a sentence?
• Computational Linguistics: How is the structure of
sentences are identified? How can knowledge and
reasoning be modeled?
• Computer Science: Algorithms for automatons, parsers.
• Engineering: Stochastic techniques for ambiguity resolution.
• Psychology: What linguistic constructions are easy or
difficult for people to learn to use?
• Philosophy: What is the meaning, and how do words and
sentences acquire it?
Lecture 01 Introduction to NLP 20
Some Buzz-Words
• NLP – Natural Language Processing
• CL – Computational Linguistics
• SP – Speech Processing
• HLT – Human Language Technology
• NLE – Natural Language Engineering
• SNLP – Statistical Natural Language Processing
• Other Areas:
• Speech Generation, Text Generation, Speech Understanding,
Information Retrieval,
• Dialogue Processing, Inference, Spelling Correction, Grammar
Correction,
• Text Summarization, Text Categorization,
Lecture 01 Introduction to NLP 21
Some NLP Applications
• Machine Translation – Translation between two natural
languages.
• See the Babel Fish translations system on Alta Vista.
• Information Retrieval – Web search (uni-lingual or
multi-lingual).
• Query Answering/Dialogue – Natural language
interface with a database system, or a dialogue system.
• Report Generation – Generation of reports such as
weather reports.
• Some Small Applications –
• Grammar Checking, Spell Checking, Spell Corrector
Lecture 01 Introduction to NLP 22
