School of Science and Engineering

Advanced Driver Assistance Systems Using AI

-Capstone Project-

Submitted in

Spring 2021

By

Omar El Idrissi El Hassani

Supervised by

Rachid Lghoul, Al Akhawayn University

1
Student Statement:

I have used the knowledge and information imparted by and about the company we are
working with, in addition to public information shared by the government or else, that I made sure
to cite. Additionally, I held the public’s and environment’s safety in the highest regard during the
design process and selection of the final design.

_____________________________________________________

Omar El Idrissi El Hassani

Approved by the Supervisor(s)

_____________________________________________________

Dr. Rachid Lghoul

2
Acknowledgment

I could not have done this work without the lots of help I received cheerfully from Pr.
Rachid Ghoul, so I would like to express my sincerest thanks to Pr. Rachid Lghoul. His
encouragement and motivation helped me overcome every difficulty we encountered during this
Capstone. Despite this pandemic situation, he showed me support, and he knew how to guide me
through positive results. He put me in touch with the right people with a large variety of skills that
helped me complete this Capstone Project.

I would also like to thank my parents and my family who showed me support during these
rough times.

3
Abstract in English

This final report is for a capstone project carried out under Pr. Rachid Lghoul 's oversight
and direction. The purpose of this Capstone Project is to get familiar with image detection
techniques and how to detect in real-time objects. Also, the goal of this work is to design,
implement and validate a circular road sign detection application of circular road signs. The
approach adopted in this work is based on a detection module that relies on the radial symmetry
detector to detect areas of the scene that are likely to contain road signs. Tests carried out on a set
of real road traffic images show the performance of the performance of the system currently
developed.

4
LIST OF ACRONYMS

ADAS – Advanced Driver Assistance Systems

AI – Artificial Intelligence

CNN – Convolutional Neural Network

YOLO – You Look Only Once

ReLU – Rectified Linear Unit

ROI – Region of Interest

SVM – Support Vector Machine

SIFM – Scale Invariant Feature Transform

TSR – Traffic Sign Recognition

5
List of Figures

Figure 1: Car accidents

Figure 2: The Engineering Design Process

Figure 3: Hough Transform

Figure 4: Cascade Classifier

Figure 5: Importing python libraries

Figure 6: Defining constants

Figure 7: Number of categories

Figure 8: Visualizing the different signs

Figure 9: Defining load data function

Figure 10: splitting the dataset

Figure 11: Creating the machine learning

Figure 12: flattening the layer and adding dense layer

Figure 13: Compiling the model

Figure 14: Fitting the model

Figure 15: model’s accuracy

Figure 16: code for plotting accuracy

Figure 17: Training and validation accuracy and loss

Figure 18: Test Data accuracy

6
Table of Contents

Student Statement: ....................................................................................................................................... 2

Acknowledgment .......................................................................................................................................... 3
Abstract in English......................................................................................................................................... 4
LIST OF ACRONYMS ....................................................................................................................................... 5
List of Figures ................................................................................................................................................ 6
Table of Contents .......................................................................................................................................... 7
1 Introduction: ......................................................................................................................................... 9
2 The Engineering Design Process ......................................................................................................... 12
3 FEASIBILITY STUDY .............................................................................................................................. 14
3.1 TECHNICAL FEASIBILITY ..................................................................................................................... 14
3.1.1 RESOURCES ................................................................................................................................ 14
3.1.2 IMPLEMENTATION ..................................................................................................................... 14
3.2 TIME FEASIBILITY............................................................................................................................... 14
3.3 FINANCIAL FEASIBILITY...................................................................................................................... 14
4. Related work ...................................................................................................................................... 16
5 Traffic Sign Recognition System (TSR)................................................................................................. 18
5.1 Detection..................................................................................................................................... 18
5.2 Recognition ................................................................................................................................ 18
5.3 Time tracking............................................................................................................................... 18
6 AI background and Algorithms............................................................................................................ 20
6.1 Machine learning ........................................................................................................................ 20
6.2 Cascade Classifier ........................................................................................................................ 20
6.3 Supervised learning ..................................................................................................................... 21
6.4 Unsupervised learning ................................................................................................................ 22
6.5 Reinforcement learning .............................................................................................................. 22
6.6 SVM Classifier.............................................................................................................................. 22
6.7 Neural network classifier ............................................................................................................ 23
6.8 Road sign descriptors .................................................................................................................. 24
6.8.1 The SIFT descriptor.............................................................................................................. 24
7 STEEPLE ANALYSIS ............................................................................................................................... 25

7
 Societal: ................................................................................................................................................... 25
Technological: ......................................................................................................................................... 25
Environmental:........................................................................................................................................ 25
Ethical:..................................................................................................................................................... 25
Political:................................................................................................................................................... 26
Legal: ....................................................................................................................................................... 26
Economical .............................................................................................................................................. 26
8 Implementation .................................................................................................................................. 27
8.1 Importing libraries....................................................................................................................... 27
8.2 Finding the Total Number of categories ..................................................................................... 28
8.3 Visualizing all the different Signs ................................................................................................ 28
8.4 Creating the Model ..................................................................................................................... 30
8.5 Plotting the Accuracy and Loss values ........................................................................................ 34
8.6 Prediction .................................................................................................................................... 36
9 Conclusion ........................................................................................................................................... 37
10 References ...................................................................................................................................... 38

8
1 Introduction:

Nowadays, Artificial Intelligence is one of the most interesting topics in computer science.
It is evolving every day; it never stops to surprise programmers. There is a huge potential behind
this technology since it is used in a large amount of domains. Artificial intelligence is such a vast
topic that it has several sub-domains such as Machine learning, Deep learning which themselves
are composed of subdomains. Also, many other field are created through AI like Big Data for
example.

For this project, our main concern is how we can use this technology with cars. Here, the
objective is to create a program that will detect every possible obstacle that a car can encounter.
Therefore, we are going to uses AI to enhance object detection algorithms.

The purpose behind this project is to prevent drivers from road accidents as much as
possible especially for seniors. Studies have shown that the elderly have more chances to have a
car accident than other drivers. This phenomenon is due to the fact that due to age and aging of the
body, the driver lacks attention when driving. Also, his reflexes are greatly reduced. In order to
prevent accidents caused by the previous mentioned reason, the development of software that
makes communication between the driver and the car is necessary. This software will essentially
detect obstacles or road panels via an object detection algorithm, and it will alert its presence to
the driver.

Figure 1: Car accidents

9
 Nowadays, the rate of traffic accidents resulting in human casualties is constantly is
increasing. Many governments aim to reduce this rate and different measures have been taken such
as measures have been taken such as traffic regulations, banning the use of distracting devices such
as cell phones. That said, technology has been used to save human lives. Many devices have been
built for this purpose, and Advanced Driver Assistance Systems (ADAS) are a good example.
These systems were designed as an aid to the driver of a vehicle, with the aim of avoiding
dangerous to avoid dangerous situations. Thus, a driver assistance system is an active safety system
driver information or assistance system designed to:

- avoid the occurrence of a dangerous situation that could lead to an accident

- free the driver from a certain number of tasks that could reduce his vigilance

- assist the driver in his perception of the environment (overtaking detectors, risk of frost,
pedestrian freezing risk, pedestrian, etc.)

- allow the vehicle to perceive the risk and to react in an anticipated way with respect to
the driver's reflexes.

A Traffic Sign Recognition (TSR) module is a highly desirable component of any traffic
assistance system. TSR is a very desirable component in any commercial mobile driver assistance
system. commercial mobile driver assistance system.

The purpose of this module is to recognize traffic signs in a video stream from a camera
installed in a vehicle and then provide this information to other ADAS modules. Such information
is needed for many tasks, such as knowing the current speed, guiding the driver through traffic,
and general knowledge of potentially dangerous areas. Common architectural implementations of
a traffic sign recognition module have divided this process into two main steps: a detection step
and a recognition step. In general, the detection step is performed to identify the region of interest
(ROI) using color segmentation, followed by some form of pattern identification. The detected
candidate signs are then either recognized or rejected during the recognition phase. The recognition
stage is performed on the candidate traffic signs, where some candidates may not contain traffic
signs. This step is performed with some machine learning techniques such as artificial neural
network (ANN), support vector machine (SVM).

10
The detection and recognition of traffic signs can be face one or more of the following difficulties:

- discolored signs,
- the presence of obstacles in the scene
- similar objects in the scene or similar background color,
- damaged road signs.

11
2 The Engineering Design Process

1. Define the Problem

The problem is that the rate of traffic accidents resulting in human casualties is constantly
is increasing, so we need a solution to stop this augmentation.

2. Do Background Research

There are several measures offered by the government and car industry, but it does not
affect drastically traffic accidents rate.

3. Specify Requirements

The possible solution should be durable and available for everyone with as maximum
efficiency as possible.

4. Brainstorm Solutions

- sensors that warn the driver about some danger

- camera that constantly informs the user of incoming information (what’s in front of the
driver).

- integrated speed limiter

5. Choose the Best Solution

A camera that constantly informs the user of incoming information (what’s in front of the
driver).

6. Develop the Solution

7. Build a Prototype

8. Test and Redesign

9. Communicate Results

12
Figure 2: The Engineering Design Process

13
3 FEASIBILITY STUDY

3.1 TECHNICAL FEASIBILITY

3.1.1 RESOURCES
In terms of technical resources, all the technologies I used are open source and the testing
will be carried out using my personal computer. Also, I may use a raspberry card to embed my
program into it. In terms of resources, the capacity of my personal computer does not matter
because the tests and simulations will be done in the cloud proposed by google colab which uses
its own resources. the platform of google colab uses the cloud technology in order to avoid the
problems resulting from the lack of resources. one can say that the platform is independent. Thus,
this project is feasible in terms of resources.

3.1.2 IMPLEMENTATION

In the implementation we will use the following open-source libraries and Frameworks:

• Python (3.9)

• GoogleColab

3.2 TIME FEASIBILITY

Because of my lack of expertise in designing a software that acts in real-time, I will have
to devote a considerable amount of time to researching the new technologies that allow such
achievement. Nevertheless, creating a model that will learn from a training set and respond if given
an input is affordable in terms of time and my knowledge in python.

3.3 FINANCIAL FEASIBILITY

Both tools and systems used will be free and open source. When we are working on
hardware, the issue of financial feasibility is addressed. Materials such as the Raspberry Pi and the
Kinect camera are expensive. Also, the Kinect camera requires an adaptor in order to work in real-
time with a computer. It will be difficult to realize such a device in real life unless we obtained all

14
of these components, but because our primary goal is to build a model, the main goal is financially
attainable.

15
4. Related work

When it comes to automating a car in order to recognize traffic panels, the topic that cannot
be avoided is that of image recognition. Image recognition and text detection are part of the top
trend topic of the last decade. Therefore, a lot of work has been conducted using these concepts.
As scientist always tries to go forward, they have been trying to improve different field with this
technology. the case of car automation has not escaped the desire of evolving through this
technology. Two major works have been conducted: Adaptive traffic road sign panels text
extraction and Detection of text on road signs from video.

- Adaptive traffic road signs panels color extraction:

The detection of traffic signs can be done according to their color. color. Since the color of
traffic signs is standardized, this type of segmentation is possible, but color perception in a camera
does not account for sign degradation due to weathering and sunlight. Color-based methods are
generally fast and invariant to translation, rotation, and scaling. Since color can be easily affected
by lighting conditions, the main difficulty with color-based methods is how to be invariant to
different lighting conditions. These methods tend to follow a common pattern: the image is
transformed into a color space and then smeared. Some authors perform this thresholding directly
in the RGB (Red Green Blue) space, even though it is very sensitive to changes in lighting. To
overcome this, simple formulas for the red, green and blue components are used. A segmentation
into related components based on a color model is performed. The regions of interest are then
validated by a recognition algorithm or an appearance model. These methods are the fastest but
also the least robust to variations in lighting conditions.

- Detection of text on road signs from Shape:

The use of a specialized shape detector to identify traffic signals depending on their shape.
This is possible thanks to most countries' standardization of traffic sign forms. They are more
stable than photometric ones in general because they handle color gradients and can process
grayscale images. Shape-based techniques, on the other hand, employ Haar-like features in frames
inspired by the famous Viola-Jones detector, where the direction and strength of image gradients
in frames inspired by the generalized Hough transform.

16
 The Hough transform is a process used for the extraction of geometric primitives in binary
images. There are two main types of Hough transforms, the "m to 1" transform (for "many to one"),
and the "1 to m" transform (for "one to many"). In these expressions, "m" represents both the
number of parameters that define a single curve, and also the number of points needed to determine
this curve. Therefore, m is the dimension of the representation space of these curves (the Hough
space). Then, there is the "m to1" transformation. It consists, from a large number of m - uplets of
points, appropriately chosen in the image, to represent the corresponding parametric curves in the
Hough space. Also there is the "1 to m" transformation. It consists in considering, for each of the
characteristic points, the curves which can then to connect the most relevant points through these
curves.

Figure 3: Hough Transform

17
5 Traffic Sign Recognition System (TSR).

A TSR is usually based on an on-board camera on the front of the vehicle. Its principle
consists in analyzing the video stream provided by the camera in order to recognize the road signs
present in the scene. The system is based on two main steps: Detection and Recognition (also
called classification). The first step locates the signs in the images based on their visual
characteristics, such as such as color and shape. The second step recognizes the type of each
detected sign by its pictogram. In general, recognized signs are reported to the driver or forwarded
to another ADAS. It is possible to add a Time Tracking step in order not to re-signal to the driver
the same recognized signs. Indeed, the signs appear on several images before they disappear from
the camera's field of view, which allows their tracking.

5.1 Detection

Locates the signs in the images. This step exploits the visual characteristics of the signs,
such as color and shape, to distinguish them from other objects in the scene. However, in real life
situations the Detection is never perfect. In addition to the signs (True Positives), false detections
(False Positives) can be generated. Indeed, the road scene can be quite cluttered and the signs can
be obscured or very close to other objects, which does not facilitate the Detection task and
generates FPs. Detection has to take into account different factors such as the diversity of sign
categories (prohibition signs, hazard signs, mandatory signs, directional signs, yield signs and
intersection signs; the change in lighting and the low contrast which make it difficult to distinguish
between signs and background.

5.2 Recognition

After Detection, there is Recognition. This step recognizes the type of each detected sign
by its pictogram based on a classifier that has already been trained on the different classes of signs,
so it can assign the appropriate class to the signs in question. In general, recognized signs are
reported to the driver or forwarded to another ADAS.

5.3 Time tracking

18
 In some TSRs, there is a temporal tracking step. This step is integrated often after the
Recognition step, the Temporal Tracking identifies the already recognized panels in order not to
re-classify them which reduces the processing time. This identification is ensured by a data
association step that relates the known panels (previously recognized) with the perceived panels
(detected at the current time). current time). The association also allows not to re-signal the same
signs to the driver in order not to disturb him. With the help of a filtering step, the Time Tracking
can also provide a priori on the positions and resolutions of the signs in the following images thus
facilitating their Detection.

19
6 AI background and Algorithms

6.1 Machine learning

On an example basis, a classifier (cascade, SVM, neural networks) is prepared. It is used

on a sliding window that moves through the picture at various sizes.

These techniques merge geometry and photometry, but they can be time-consuming to
compute. They necessitate the development of a training base based on the form of panel, which
is a time-consuming process when the amount of objects to be detected is high.

6.2 Cascade Classifier

The Cascade procedure is used to classify a detected indication as volumetric or non-

volumetric. It results from the successive application of several discriminatory criteria

- the reflectivity, or amplitude of the echo,

- the directional reflectivity, or study of the reflectivity as a function of the angle of

incidence and the direction of the ultrasound beam,

- the shape of the echo, or its form

- the dynamic behavior of the echo, or its behavior when the transducer moves over the
defect. The procedure stops as soon as one of the criteria is met. The classification thus goes
through the following five steps:

- step 1: avoid taking into account poorly reflecting indications,

- step 2: consider as non-volumetric defects all indications that are strongly reflective.

reflective indications.

- step 3: characterize the lack of fusion on the chamfer, the lack of fusion between passes,
or the inclusionary

passes, or inclusionary bonding.

- step 4 : characterize some inclusions

20
 - step 5: characterizes the branched or strongly roughened cracks.

Figure 4: Cascade Classifier

6.3 Supervised learning

In the field of artificial intelligence there are different types of artificial learning.
Supervised learning uses a learning base containing annotated examples. From this base and these
annotations, the system is able to classify a new example that does not belong to the base. In this
case, the classifier makes binary decisions, i.e. "the object belongs to the class or the object does
not belong to the class". The databases will therefore be made up of positive examples (one panel)
and negative examples (something else). The objective is to define a rule allowing to assign any
unknown pattern x of a certain space X to a given class l from a priori knowledge. This knowledge
is provided in the form of a learning base, representative of the classification task. It represents the
memory, or experience, of the system. In particular, it must contain a significant set of N patterns
xi, i ∈ [1, N] from each of the categories to be distinguished. If each pattern xi has been previously
associated with a class, labeled li, the classification algorithm aims at separating the space between
the L classes as well as possible. The learning is said to be supervised. On the other hand, if this
distribution is unknown, it is said to be unsupervised and the objective is to find the optimal number
of classes allowing the best separation according to the given criteria.

21
6.4 Unsupervised learning

For unsupervised learning, the learning set is only composed of examples, without any
examples, without any class label. It is up to the algorithm to find dependencies, structures between
the different examples. Clustering" or data partitioning is a set of unsupervised learning methods,
such as the K-means algorithm. Clusters are created by the algorithm, which groups together
objects with characteristics that are common to each other and different from objects that do not
belong to the same class. Let's take an example of classification by K-means. The algorithm is
randomly initialized with a number of clusters for which an average point, called barycenter, is
evaluated. At each iteration, the distance between each example at the different centroids is
computed; each example is then associated to the cluster whose distance to the distance to the
centroid is the closest. Then the centroids are re-evaluated. The algorithm ends when there is no
more change.

6.5 Reinforcement learning

This type of method is interactive learning. For each decision that the the algorithm makes,
it receives responses from the environment called reinforcement signals. called reinforcement
signals. It is an adaptive process that improves the solution according to the responses that it
receives. The "Q-learning" algorithm performs reinforcement learning. It produces a matrix Q with
each element Q (s, a) being a measure of the interest to perform action a when the system is in
state s. Moreover, theoretical results theoretical results guarantee, in specific cases, the
convergence of the algorithm towards optimal values of Q.

6.6 SVM Classifier

SVMs are a class of next-generation supervised learning methods focused on recent

mathematical learning theory advances. They are founded on the margin maximization theorem
(class separation).

SVM classifiers are mainly designed to solve discrimination problems, allowing to decide
which discrimination problems, allowing to decide to which class a sample belongs. However,
they can also solve regression problems aiming at predicting the numerical value of a variable.
That is to say, the prediction of a continuous variable according to other variables, as it is the case
22
for example in the prediction of electrical consumption as a function of the time of year,
temperature, etc.

The solution of these two problems is based on the construction of a function h which, to
an input vector x, corresponds an output y (y = h(x)). Its general principle is based on the following
steps:

1. Construction of a real-valued classifier

2. Splitting the problem into two sub-problems

3. Non-linear transformation of the inputs

4. Choice of an 'optimal' linear separation

In the following diagram, a hyperplane is determined which separates the two sets of
points. The closest points, which are the only ones used for the determination of the hyperplane,
are called support vectors.

6.7 Neural network classifier

Neural networks (NN) are mathematical and computer models of computational units
called "formal neurons" whose original inspiration was a model of the human nerve cell. Neural
networks are composed of simple elements (or neurons) working in parallel. These elements have
been strongly inspired by the biological nervous system. As in nature, the functioning of the
(neuron) network is strongly influenced by the connections of the elements between them. We can
train a neural network for a specific task (sign recognition for example) by adjusting the for
example) by adjusting the values of the connections (or weights) between the elements (neurons).
In general, neural networks are trained so that for a particular input presented to the network, a
specific target corresponds. The adjustment of weights is done by comparing the network response
(or output) with the target, until the output matches the target. For this type of supervised learning,
a large number of input/output pairs are used.

23
6.8 Road sign descriptors

The first phase of the recognition consists in extracting the characteristics of the detected signs.
For this, several descriptors have been proposed. In the following, we quote the most used ones in
TSR.

6.8.1 The SIFT descriptor

SIFT (Scale Invariant Feature Transform) is a model that combines a scale invariant interest point
detector and a descriptor based on the distribution of gradients computed over the region of
interest. First, a region of interest is defined around each point of interest. Then, the amplitude and
orientation of the gradient are computed for all the pixels of the region of interest. The amplitudes
obtained are weighted by a Gaussian that takes into account the distance of the pixels of the region
of interest from the center. Then, orientation histograms, where the vote is weighted by the
amplitudes, are generated from each subregion.

24
7 STEEPLE ANALYSIS

Societal:
From a sociocultural standpoint, there's no denying that driving is becoming more common.
Families all over the world are increasingly purchasing one or more automobiles; in reality, owning
one or more automobiles is already the standard in developed countries like the United States,
Canada, and most of the European Union. Part of this is a cultural phenomenon: it's not that many
of us couldn't get by on bicycles or taxis, but we prefer to drive motor vehicles because it's what's
expected of us. With vehicles equipped with this technology the need of buying a car will definitely
increase.

- Reduce car accident

Technological:
- Depending on the need we may design using python, GoogleColab.

- the program could be implemented to a car system using rasberycard

Environmental:
Collision with fauna and Flora will be drastically reduced. Trees and animals will no
longer suffer from crashes.

Ethical:
- When talking about car object detection, a famous scenario tends to emerge. Consider the
scenario in which while you are driving, you are faced with a horrible choice: swerve left and hit
an eight-year-old girl, or swerve right and hit an 80-year-old grandmother. Given the car's pace,
any victim will almost certainly die on impact. Both victims will be hit and killed if you do not
swerve, so there is reasonable reason to believe you can swerve one way or the other. What, on
the other hand, would be the ethically right decision? How will be the decision of the software?

- All information used has been acquired from ethical sources and practices and respect the
privacy of the people involved.

- The ethical implications of the project itself are positive taking into consideration all the other
implications aforementioned.

25
Political:
Cities under Smart City Initiative by Government & hence moving towards “Intelligent
Transport System” encourages Object Detection in Car’s.

Legal: N/A

Economical:

It should come as no surprise that demand for automobiles is steadily increasing. More
vehicles would be sold as a result of the increasing demand for automobiles especially automobiles
equipped with high level technology such as object detection, making the industry even more
lucrative for its members.

26
8 Implementation

After the theoretical part and the literature review, time comes to implement what we have
learned. This part will be essentially dedicated to the creation of the machine learning model that
will allow us to solve the problem addressed in the previous chapters. Besides the creation of the
model, an optimization must be conducted, so it can greatly enhance the model performance.

8.1 Importing libraries

First, importing the right libraries is very important for what is coming next. Some libraries
facilitate a lot the work that need to be done. In our case, sklearn make the construction of a
machine learning model really simple. After importing the libraries, I have set important constants
such as the paths where the data that I have downloaded is and the images dimensions.

Figure 5: Importing python libraries

Figure 6: Defining constants

27
8.2 Finding the Total Number of categories

An important step when it comes to data preprocessing is to see in how many categories
the data is divided. In our case there are 43 category of images.

Figure 7: Number of categories

8.3 Visualizing all the different Signs

Before going further, it is important to visualize the data. This way I can have an Idea on
how it is categorized.

28
 Figure 8: Visualizing the different signs

Now that we know the different types of signs. I need to categorize them and split them
into Train and Test set. I created the function load data to categorize data. Loading data from Train
folder. Returns a tuple `(images, labels)` , where `images` is a list of all the images in the train
directory, where each image is formatted as a numpy ndarray with dimensions IMG_WIDTH x
IMG_HEIGHT x 3. `labels` is a list of integer labels, representing the categories for each of the
corresponding `images`.

29
 Figure 9: Defining load data function

Figure 10: splitting the dataset

A one hot encoding allows the representation of categorical data to be more expressive. It
is the most common method, which works well until your categorical variable has a huge number
of values. So this approach is well justified. Then I started splitting the data into training and test
set. I chose to separate it this way 40% for test set and 60% for training set.

8.4 Creating the Model

Here I started creating my model. I opted for a sequential a three layered Convolutional
neural network. While CNN models have achieved great success on image recognition due to their
strong abilities in abstracting spatial information from multiple levels. In this work, I chose a
Sequential Convolutional Neural Network, denoted as SCNN, to extract effective spatial-temporal

30
features from videos, thus incorporating the strengths of both convolutional operation and
recurrent operation.

Figure 11: Creating the machine learning

31
 In order to add dense layer, I used ReLU activation function. The term "rectified linear
unit" refers to a form of activation function. It is mathematically defined as y = max (0, x). In
neural networks, especially CNNs, ReLU is the most widely employed activation function. ReLU
is a decent first choice if you're not sure what activation function to utilize in your network.

Figure 12: flattening the layer and adding dense layer

32
 Figure 13: Compiling the model
Then, I compiled my model and optimized it using Adam optimization algorithm The
Adam optimization approach is a stochastic gradient descent variant that has lately gained traction
in computer vision and natural language processing applications.

Figure 14: Fitting the model

33
 Figure 15: model’s accuracy

Finally, the model managed to achieve 98% of accuracy.

8.5 Plotting the Accuracy and Loss values

Now that we have seen the accuracy, I plotted it at different epochs for training set and
testing set as it is shown in figure [16].

Figure 16: code for plotting accuracy

34
Figure 17: Training and validation accuracy and loss

35
8.6 Prediction

When it comes to the model’s performance with the testing set, it is doing really well
with an accuracy of 92.9%.

Figure 18: Test Data accuracy

36
9 Conclusion

Object detection in images and more particularly sign detection is a problem addressed by
computer vision. Among its application domains, we can mention video surveillance, road safety,
detection assistance systems. In our work, we are interested in the detection of circular road signs.
A bibliographical study was carried out in order to determine the possible detection methods
adopted in this case. A particular interest was given to the development of a machine learning
model. This one is dedicated to give results in the case of a data entry. The model is trained in
advance through a training set that allows it to learn to distinguish the different types of signs. Our
work was supposed to implement or embed this model in a raspberry card, and to connect it with
a Kinect camera so it can utilize in real time. Since there were difficulties regarding the Kinect’s
functionality, we could not achieve this goal. Nevertheless, the model itself performs with a fairly
high degree of accuracy. Although the main objective was not complete due to technical issues,
the idea itself is still reachable. Developing such a tool would enhance the quality of transport and
safety in traffics. Also, while developing such a tool the emphasis should be on the accuracy and
the portability of the tool.

37
10 References

[1] A. V. Reina, R. J. L. Sastre, S. L. Arroyo, and P. G. Jiménez, “Adaptive traffic road sign
panels text extraction,” in Proc. 5th WSEAS ISPRA, 2006, pp. 295–300.

[2] Belaroussi, R., Foucher, P., Tarel, J. P., Soheilian, B., Charbonnier, P., & Paparoditis, N.
(2010, August). Road sign detection in images: A case study. In 2010 20th International
Conference on Pattern Recognition (pp. 484-488). IEEE.

[3] Brownlee, J. (2021, January 26). A Gentle Introduction to Object Recognition With Deep
Learning. Machine Learning Mastery. https://machinelearningmastery.com/objectrecognition-
with-deep-learning/

[4] C. Bahlmann, Y. Zhu, R. Visvanathan, M. Pellkofer, et T. Koehler. A system for traffic sign
detection, tracking, recognition using color, shape, and motion information. Dans proceedings of
IEEE intelligent vehicules symposium

[5] De la Escalera, A., Armingol, J. M., & Mata, M. (2003). Traffic sign recognition and analysis
for intelligent vehicles. Image and vision computing, 21(3), 247-258.

[6] Gonzalez, A., Bergasa, L. M., & Yebes, J. J. (2013). Text detection and recognition on traffic
panels from street-level imagery using visual appearance. IEEE Transactions on Intelligent
Transportation Systems, 15(1), 228-238.

[7] Mathias, M., Timofte, R., Benenson, R., & Van Gool, L. (2013, August). Traffic sign
recognition—How far are we from the solution?. In The 2013 international joint conference on
Neural networks (IJCNN) (pp. 1-8). IEEE.

[8] Romadi, M., Thami, R. O. H., Romadi, R., & Chiheb, R. (2014, May). Detection and
recognition of road signs in a video stream based on the shape of the panels. In 2014 9th
International Conference on Intelligent Systems: Theories and Applications (SITA-14) (pp. 1-5).
IEEE.

[9] Wen, S. (2019, December 11). Object Detection with YOLO | Bringing Vision to SelfDriving
Cars. Medium. https://towardsdatascience.com/object-detection-with-yolobringing-vision-to-
self-driving-cars-980295226830

38
[10] W. Wu, X. Chen, and J. Yang, “Detection of text on road signs from video,” IEEE Trans.
Intell. Transp. Syst., vol. 6, no. 4, pp. 378–390,

[11] Y. Y. Nguwi et A. Z. Kouzani. Detection and classification of road signs in natural

Environments. Neural computing and applications, 17, 265–289, 2008.

39