Natural language processing
Section 7
Machine Translation
1
Definition
Process of using artificial intelligence to automatically
translate text from one language to another without
human involvement.
Modern machine translation goes beyond simple
word-to-word translation to communicate the full
meaning of the original language text in the target
language.
2
Examples of Machine Translation
Google translate ,Bing translate ,Microsoft translator
and Amazon translator are examples of machine
translation.
3
How does Google Translate work?
It is able to scan text using the device and translate it instantly.
The system automatically identifies foreign languages and translates
them.
But there is some ambiguity in the case of speech translation because
of misunderstanding spelling of some words.
4
Idea
Using Machine Translation to translate text from English to
French.
5
Illustrating This Idea
This idea Depends on Encoder
and Decoder gates :
An encoder: is a
combinational network that
performs the reverse
operation of the decoder. An
encoder has 2n or less
numbers of inputs and n
output lines.
A decoder: is a combinational
circuit that converts n bits of
binary information of input
lines to a maximum of 2n 6
What is LSTM? - Introduction
Long Short-Term Memory Networks is a deep learning,
sequential neural network that allows information to
persist.
It is a special type of Recurrent Neural Network which
is capable of handling the vanishing gradient problem
faced by RNN.
LSTM was designed by Hochreiter and Schmidhuber
that resolves the problem caused by traditional rnns
and machine learning algorithms. LSTM Model can be
What is LSTM? - RNN problem
● The Problem of Long-Term Dependencies
● vanishing gradient problem
What is LSTM? - RNN problem
How do LSTMs work?
● STMs try to add long term
memory to remember
certain hidden states
more than others. This
allows them to retain
knowledge over longer
sequences.
● They have 2 outputs
instead of 1, the hidden
state and the cell state.
Their computation is a bit
LSTM - Architecture
An LSTMs architecture consists of 3 gates - Forget
gate, Input gate, Output gate
• Tanh acts as a squashing function while Sigmoid acts
as a decision function (gate)
• Cell state is a channel that runs along the LSTM chain
carrying information from one time-step to another
freely
LSTM - Architecture
Acquiring Data
• As we need to open and read files, we use pandas and os
libraries .
• Then read the file “fra.txt” in csv format.
import numpy as np
import pandas as pd
import os
df = pd.read_csv('fra.txt', delimiter='\t')
print(df.info())
13
Prepare data
We use keras and tensorflow as we iterate with
deep learning and apply our model on.
from tensorflow.keras.models
import Model from tensorflow.keras.layers
import Input,LSTM,Dense
batch_size=64
epochs=100
latent_dim=56
num _samples=1000
14
loading our data
We need to load our data:
kaggle kernels output akshat0007/machine-translation-english-to-french-rnn-lstm -p
/path/to/dest
15
Detecting input and output
input_texts = []
target_texts = []
input_characters = set()
target_characters = set()
with open(data_path, 'r', encoding='utf-8') as f:
lines = f.read().split('\n')
for line in lines[: min(num_samples, len(lines) - 1)]:
input_text, target_text= line.split('\t')
target_text = '\t' + target_text + '\n'
input_texts.append(input_text)
target_texts.append(target_text)
for char in input_text:
if char not in input_characters:
input_characters.add(char)
for char in target_text:
if char not in target_characters:
target_characters.add(char) 16
Applying Encoder and
Decoder Techniques
encoder_input_data = np.zeros(
(len(input_texts), max_encoder_seq_length, num_encoder_tokens),
dtype='float32')
decoder_input_data = np.zeros(
(len(input_texts), max_decoder_seq_length, num_decoder_tokens),
dtype='float32')
decoder_target_data = np.zeros(
(len(input_texts), max_decoder_seq_length, num_decoder_tokens),
dtype='float32')
17
Applying Encoder and Decoder
Techniques (.cont)
encoder_inputs = Input(shape=(None, num_encoder_tokens))
encoder = LSTM(latent_dim, return_state=True)
encoder_outputs, state_h, state_c = encoder(encoder_inputs)
encoder_states = [state_h, state_c]
decoder_inputs = Input(shape=(None, num_decoder_tokens))
decoder_lstm = LSTM(latent_dim, return_sequences=True, return_state=True)
decoder_outputs, _, _ = decoder_lstm(decoder_inputs,
initial_state=encoder_states)
decoder_dense = Dense(num_decoder_tokens, activation='softmax')
decoder_outputs = decoder_dense(decoder_outputs)
18
Training Model
model = Model([encoder_inputs, decoder_inputs], decoder_outputs)
model.compile(optimizer='rmsprop', loss='categorical_crossentropy',
metrics=['accuracy'])
model.fit([encoder_input_data, decoder_input_data], decoder_target_data,
batch_size=batch_size,
epochs=epochs,
validation_split=0.2)
19
Translated Model
for seq_index in range(100):
input_seq = encoder_input_data[seq_index: seq_index + 1]
decoded_sentence = decode_sequence(input_seq)
print('-')
print('Input sentence:', input_texts[seq_index])
print('Decoded sentence:', decoded_sentence)
20
https://colab.research.google.com/drive/
1WxycXjTkeemHPZMFwf0qm8KpEZJCt1c5#scrollTo=3-sWPmxyiHZU
21
Thank You
22
