What is NLP?

NLP stands for Natural Language Processing, which is a part of Computer Science, Human
language, and Artificial Intelligence. It is the technology that is used by machines to
understand, analyse, manipulate, and interpret human's languages. Natural language
processing strives to build machines that understand and respond to text or voice data—
and respond with text or speech of their own—in much the same way humans do.

NLP combines computational linguistics—rule-based modelling of human language—with

statistical, machine learning, and deep learning models. Together, these technologies enable
computers to process human language in the form of text or voice data and to ‘understand’
it’s full meaning, complete with the speaker or writer’s intent and sentiment.

NLP drives computer programs that translate text from one language to another, respond to
spoken commands, and summarize large volumes of text rapidly—even in real time. There’s
a good chance you’ve interacted with NLP in the form of voice-operated GPS systems, digital
assistants, speech-to-text dictation software, customer service chat-bots, and other
consumer conveniences.

Advantages of NLP

 NLP helps users to ask questions about any subject and get a direct response within
seconds.
 NLP offers exact answers to the question.
 It does not offer unnecessary and unwanted information.
 NLP helps computers to communicate to humans (Any languages).
 It is very time efficient.

Disadvantages of NLP

 Misinterpretation due to ambiguous or unclear input.

 The system is built for a single and specific task only; it is unable to adapt to new
domains and problems because of limited functions.
  A voice interface might need training to get the software to recognise what the user is
saying.

Components of NLP

There are the following two components of NLP –

NLP=NLU + NLG

1) Natural Language Understanding (NLU)

Understanding involves the following tasks −

 Mapping the given input in natural language into useful representations.

 Analysing different aspects of the language.

Difficulties in NLU

There can be different levels of ambiguity −

 Lexical ambiguity − It is at very primitive level such as word-level.

For example, treating the word “board” as noun or verb?

 Syntax Level ambiguity − A sentence can be parsed in different ways.

For example, “He lifted the beetle with red cap.” − Did he use cap to lift the beetle
or he lifted a beetle that had red cap?

 Referential ambiguity − Referring to something using pronouns. For example, Sita

went to Meena. She said, “I am tired.” − Exactly who is tired?

2) Natural Language Generation (NLG)

It is the process of producing meaningful phrases and sentences in the form of natural
language from some internal representation.
It involves −
 Text planning − It includes retrieving the relevant content from knowledge base.
 Sentence planning − It includes choosing required words, forming meaningful
phrases, setting tone of the sentence.
 Text Realization − It is mapping sentence plan into sentence structure.
Phases of NLP
There are the following five phases of NLP:

1. Lexical Analysis and Morphological

The first phase of NLP is the Lexical Analysis. This phase scans the source code as a stream
of characters and converts it into meaningful lexemes. It divides the whole text into
paragraphs, sentences, and words.

2. Syntactic Analysis (Parsing)

Syntactic Analysis is used to check grammar, word arrangements, and shows the relationship
among the words.

Example: Agra goes to the Poonam.

In the real world, Agra goes to the Poonam, does not make any sense, so this sentence is
rejected by the Syntactic analyser.

3. Semantic Analysis

Semantic analysis is concerned with the meaning representation. It mainly focuses on the
literal meaning of words, phrases, and sentences. The semantic analyser disregards sentence
such as “hot ice-cream”.
4. Discourse Integration

The meaning of any sentence depends upon the meaning of the sentence just before it and
after it. In addition, it also brings about the meaning of immediately succeeding sentence.

5. Pragmatic Analysis

Pragmatic is the fifth and last phase of NLP. It helps you to discover the intended effect by
applying a set of rules that characterize cooperative dialogues.

For Example: "Open the door" is interpreted as a request instead of an order.

NLP use cases (Applications)

Natural language processing is the driving force behind machine intelligence in many modern
real-world applications. Here are a few examples:

 Spam detection: You may not think of spam detection as an NLP solution, but the
best spam detection technologies use NLP's text classification capabilities to scan
emails for language that often indicates spam or phishing. These indicators can
include over use of financial terms, characteristic bad grammar, threatening
language, inappropriate urgency, misspelled company names, and more. Spam
detection is one of a handful of NLP problems that experts consider 'mostly solved'
(although you may argue that this doesn’t match your email experience).
 Machine translation: Google Translate is an example of widely available NLP
technology at work. Truly useful machine translation involves more than replacing
words in one language with words of another. Effective translation has to capture
accurately the meaning and tone of the input language and translate it to text with
the same meaning and desired impact in the output language. Machine translation
tools are making good progress in terms of accuracy. A great way to test any
machine translation tool is to translate text to one language and then back to the
original. An oft-cited classic example: Not long ago, translating “The spirit is
willing but the flesh is weak” from English to Russian and back yielded “The vodka
is good but the meat is rotten.” Today, the result is “The spirit desires, but the flesh
is weak,” which isn’t perfect, but inspires much more confidence in the English-to-
Russian translation.
 Virtual agents and chatbots: Virtual agents such as Apple's Siri and Amazon's
Alexa use speech recognition to recognize patterns in voice commands and natural
language generation to respond with appropriate action or helpful comments. Chat-
bots perform the same magic in response to typed text entries. The best of these also
learn to recognize contextual clues about human requests and use them to provide
even better responses or options over time. The next enhancement for these
applications is question answering, the ability to respond to our questions—
anticipated or not—with relevant and helpful answers in their own words.
 Social media sentiment analysis: NLP has become an essential business tool for
uncovering hidden data insights from social media channels. Sentiment analysis can
analyze language used in social media posts, responses, reviews, and more to extract
attitudes and emotions in response to products, promotions, and events–information
companies can use in product designs, advertising campaigns, and more.
 Text summarization: Text summarization uses NLP techniques to digest huge
volumes of digital text and create summaries and synopses for indexes, research
databases, or busy readers who don't have time to read full text. The best text
summarization applications use semantic reasoning and natural language generation
(NLG) to add useful context and conclusions to summaries.