Complex & Intelligent Systems (2020) 6:621–634

https://doi.org/10.1007/s40747-020-00155-2

ORIGINAL ARTICLE

Machine learning based customer sentiment analysis

for recommending shoppers, shops based on customers’ review
Shanshan Yi1 · Xiaofang Liu2

Received: 11 March 2020 / Accepted: 13 May 2020 / Published online: 11 June 2020
© The Author(s) 2020

Abstract
Big data analytics plays a major role in various industries using computing applications such as E-commerce and real-time
shopping. Big data are used for promoting products and provide better connectivity between retailers and shoppers. Nowa-
days, people always use online promotions to know about best shops for buying better products. This shopping experience
and opinion about the shopper’s shop can be observed by the customer-experience shared across social media platforms. A
new customer when searching a shop needs information about manufacturing date (MRD) and manufacturing price (MRP),
offers, quality, and suggestions which can only be provided by the previous customer experience. The MRP and MRD are
already available in the product cover or label. Several approaches have been used for predicting the product details but
not providing accurate information. This paper is motivated towards applying Machine Learning algorithms for learning,
analysing and classifying the product information and the shop information based on the customer experience. The product
data with customer reviews is collected from benchmark Unified computing system (UCS) which is a server for data based
computer product lined up for evaluating hardware, support to visualization, software management. From the results and
comparison, it has been found that machine learning algorithms outperform than other approaches. The proposed HRS system
has higher values of MAPE which is 96% and accuracy is nearly 98% when compared to other existing techniques. Mean
absolute error of proposed HRS system is nearly 0.6 which states that the performance of the system is significantly effective.

Keywords Big data · Big data analytics · Machine learning · Feature extraction · Clustering and classification · Accuracy

Introduction play a vital role in analyzing and understanding the com-

munication happening across the transcripts.
Social media and networks have probably taken over the Sentiment analysis is referred as opinion mining or emo-
globe by storm and have shrunk the dimensions of the world. tional intelligence (EI). Sentimental analysis can be defined
There is a fair bit of communication happening between peo- as the art of gathering useful insight from unstructured and
ple across several social media platforms such as Twitter, unorganized textual contents from several social platforms
Facebook, WhatsApp etc. These communications in most and online sources such as chats happening across social
cases will be informal. These discussions convey the mood platforms such as Twitter, WhatsApp and Facebook, online
and sentiment of the persons involved in the discussion. This blogs and comments. Opinion mining involves developing
paves way for understanding the behavioural traits of people rule based automated systems that analyze the data based
involved in the discussion. Sentimental analysis also called on a set of predefined rules. Also, automated systems that
as opinion mining and natural language processing (NLP) use any of the machine learning principles are developed to
perform opinion mining. There are some scenarios where
both rule based and automated machine learning algorithms
* Shanshan Yi are combined together to develop a hybrid model for senti-
shanshanyi@126.com mental analysis. Sentimental analysis can also serve as an
effective means to study the positive or negative intent based
1
School of Nuclear Technology and Automation Engineering, on the textual contents. Sentimental analysis has served as a
Chengdu University of Technology, Chengdu 610000, China
reliable source for providing insightful opinion about several
2
Sichuan Aerospace Vocational College, Chengdu 610000, products rolled over in the market, innovative ideas, people
China

13
Vol.:(0123456789)
622 Complex & Intelligent Systems (2020) 6:621–634

opinion about new policies framed by government etc. At followed by data pre-processing which comprises of several
the same time, sentiment analysis also plays a pivotal role in tasks like removal of contents like punctuations, erroneous
understanding the shopping experience in a particular shop words, duplicate or redundant words and stop words. This
based on the remarks that is shared by the customer across in turn is a measure to enhance relevancy of data towards
social media platform. Sentimental analysis thus undermines opinion mining. The data pre-processing is followed by fea-
the significance in terms of understanding the customer’s ture extraction from the text data. Feature selection involves
opinion in terms of the shopping experience gained while searching for text that conveys the mood of the person in
purchasing a product in a particular shop. conversation and this feature selection is carried out using
One of the key techniques that are instrumental in pro- Simulated Annealing. A semantic word dictionary is formed
moting sentimental analysis is natural language processing to identify the classes of opinions. This is followed by apply-
(NLP). It is one of the sub-domains in artificial intelligence ing multiclass support vector machine as the classifier for
that empowers the computer aided system to understand the classifying the sentiments.
different languages as communicated or spoken by human. The contribution of this work is to develop an effec-
NLP predominantly focuses on understanding the unstruc- tive data pre-processing method for unwanted content
tured contents present in the social media and organizing removal and filtering only relevant text. The feature extrac-
the same to be made easier for sentimental analysis. NLP tion process involves extracting information pertaining to
mainly focuses on reading and interpreting the free form Manufacturing date (MFD), Manufacturing price (MRP),
text into an analysable format. The primary application area discounts and offers, quality ratings, and suggestions or
thus where NLP is a key component is opinion mining. Also, reviews. Finally, multiclass support vector machine classi-
another common application where NLP plays a pivotal role fier is deployed to promote an efficient and effective multi
is lending its help to search engines such as Google to tune class classification of different types of sentiments based
their search algorithms to understand different contexts and on the above extracted features. Extraction of feature has
to understand and interpret contents of different languages wide range adoptability in automatic inspection manufactur-
and produce relevant search results. ing systems. Image feature identification in manufacturing
NLP predominantly deploys several machine learning industries fail due to poor unsuitable identification of fea-
algorithms for text data classification. One common and tures, thus machine learning approaches such as principal
mostly preferred method deployed for classification is sup- component analysis (PCA) is useful in identifying suitable
port vector machine (SVM). It is a typical supervised learn- features from manufacturing images. This PCA method is
ing approach that uses machine learning algorithms to per- significant in reducing dimensionality for feature extraction
form effective regression analysis and data classification. In approaches and features related to identification of different
the case of typical SVM based classification, the categories patterns in manufacturing.
of classes are clearly defined and the model is well trained
to fit the data into the specified classes. SVM can be used
to perform linear classification as well as nonlinear clas- Literature review
sification. SVM plays a pivotal role in categorizing textual
contents and hyper textual contents according to the classes With the social media platforms such as Twitter, Face-
defined. Also, SVMs are typically used for image classifica- book, Instagram and WhatsApp taking the communica-
tion, text and hand written character recognition. A binary tion world by storm, it has become imperative that the
SVM is focussed towards classifying data under two buckets data residing across these social media platforms will
or classes. However, human sentiment is multidimensional convey insightful information about the opinion, mood
in nature and a simple binary SVM classifier may not suit and sentiment of the people over any product, idea or
sentimental analysis. This prompts for the use of multiclass policies. Several works have been performed earlier to
support vector machine classifier for studying and analyz- analyze the twitter contents and perform opinion mining
ing sentiment by reading through textual data. A multiclass over twitter data. The authors of [1] have proposed an
SVM is built by combining several binary SVMs. This approach that uses deep convolution neural network to
approach towards sentimental analysis using twitter data analyze the twitter feed. The feature set was integrated
uses MSVM for better classification of different types of into deep CNN for training and predicting sentiments
sentiments that can be extracted out of twitter data. by analyzing twitter data. When it comes to twitter data
The proposed approach is a machine learning based analysis, text pre-processing has a significant role to play.
approach which uses multiclass support vector machine Several text pre-processing methods have been analyzed
(MSVM) for classification of different classes of opinions and compared as part of [2]. Since sentiment analysis
and sentiments in twitter. The proposed approach comprises involves analyzing and interpreting different types of sen-
of importing the twitter data from the twitter source which is timents, multi class sentiment analysis is highly essential

13
Complex & Intelligent Systems (2020) 6:621–634 623

and is discussed in [3]. Authors of [4] have developed Limitations

a tool named SENTA that uses a pattern comparison
approach to perform multiclass sentiment analysis. As The data residing in twitter or any social media is mostly
discussed in [5], is to understand the evolving changes unstructured in nature and hence NLP has a pivotal role
of opinion mining, to the context and the dynamic events in structuring the unstructured data. At the same time the
that occur during the twitter conversations. dynamics behind the topic under discussion and the con-
The usage of twitter has kindled more research work text of discussion still remain a black box. This in turn
towards understanding the sentiments using twitter data. reduces the efficiency of opinion mining and the accuracy
One such work discussed in [6] uses a hybrid framework of sentiment analysis and prediction. It is highly difficult
that uses a genetic algorithm-based approach to perform for any sentimental analysis model to understand the exter-
sentiment analysis. The focus of this work was to enhance nal factors that may have an influence on the discussions
the system from a scalability perspective. Authors of [7] that happen over twitter. These external factors have a
have explored the variations of public sentiment over a direct bearing on the discussions and in turn impact the
given topic using a mathematical model to detect the fore- sentiment of people discussing about a particular topic.
ground topics and promote effective ranking of candi- Another important aspect that is found wanting is the need
dates. An interesting work related to sentimental analysis for a quick search solution to spot the essential words that
is to classify sentiment based on the topic of discussion. covers the mood and sentiment of the people discussions
Authors of [8] have provided an effective topic adap- over twitter.
tive sentiment classification over tweets. The different
challenges posed while performing multiclass sentiment
analysis has been discussed in [9] and the authors have
also developed a novel model that uses multiclass senti- Motivation
ment analysis over twitter data. Authors of [10] have pro-
posed an effective text to speech conversion based on the Since human sentiment is multidimensional in nature, a
sentences provided in twitter data. Authors of [11] have simple binary support vector machine may not provide
introduced a novel method for hierarchical topic model- a right classification when it comes to opinion mining.
ling using twitter data to perform Online Analytical Pro- Hence the proposed approach is motivated towards using
cessing (OLAP). To enhance the effectiveness of queries a machine learning based regression model to accommo-
related to analytics, large OLAP database such as Vertica date the different types of sentiments that can be extracted
[12], Greenplum [13] and Teradata DB [14] has been from the twitter data. This in turn promotes using more
suggested. Vertica [12] utilize projection to enhance per- classes which means that the classification accuracy is
formance of query. Instead of developing indexes which not compromised. Since there are five pivotal features
are conventional on columns, it retains the details about namely—Manufacturing date (MFD), Manufacturing
min/max ranges, leading to higher latency from lower price (MRP), discounts and offers, quality ratings and
pruning which are less efficient. Greenplum [13] and suggestions or reviews that will be used to define cus-
Teradata DB [14] approve store column wise and allow tomer shopping experience in a shop, regression based
specification of indexing in column by users. Moreover, collaborative recommendation has been chosen to be the
there exists, two drawbacks: initially, modification of col- best classifier for this system. Also, it is imperative that
umn indexes in write path which is excessive for every the searching speed has a pivotal role in determining the
column indexes; secondly, it requires more random I/Os overall efficiency of the opinion mining system. Using a
for queries relating to point-lookup. high-end machine learning classifier such as regression
From the above works it can be easily understood that based collaborative recommendation system ensures that
the contents of twitter data provide useful insights about the overall accuracy of the decision making is always high
any topic under discussion and also conveys the opinion while classifying the customer’s shopping experience
of people involved over the particular topic. The main from the shop perspective. Quantum computing refers to
advantage and the unique functionality of machine learn- the phenomena which uses quantum mechanism such as
ing algorithms is extracting the essence of the given prob- entanglement and superposition to illustrate computing.
lem. It provides complete information about the data and Computers used for analysing computation is known as
make the developer to think about choosing the appropri- quantum computers. It uses quantum circuits for quantum
ate algorithm which can be used for the problem. Some of analysis, circuits are on the basis of quantum bit. There
the common classes of the machine learning problems are exists two methods to implement physically which refers
clustering, classification, regression and rule extraction. as quantum computer known digital and analog. Analog

13
624 Complex & Intelligent Systems (2020) 6:621–634

methods utilise various sections such as simulation and Table 1  Customer – product matrix
annealing quantum. Digital quantum computes the logic
gates for evaluating, whereas both methods uses bits for
computing.
This paper integrates the collaborative filtering and
product-product similarity method for designing the new
recommendation system. Various types of recommendation
systems are illustrated in Fig. 1. The collaborative filter-
ing method applied for predicting the best shops and the
product-product similarity method applied for predicting the
best product. Both the products and the shops are predicted
based on the highest ratings given by the customer.

Recommendation system
represents the rating value given by the customer Ci on the
A recommender system is one of the information retrieval product Pj , where the value is from 1 to 5. If the product is
system which uses filtering method where it finds the not selected by the customer the value is 0, else it is 1. Hence
requirement by analysing and comparing the score/rank/ rat- the Aij value may be binary initially. In other cases, the value
ing of a product/item given by the customer. Then it predicts of Aij is from 1 to 5. Since, the customer and their option of
the high rating-based product for recommend to the new product selection is not compulsory, the customer-product
customers. This recommendation system makes the shops matrix is very sparse matrix. Means, all the cells have not
and shopper happy in terms of selling and buying. This rec- been filled, many of the cells are not having any values.
ommendation system recommends not only few products
or shops, but it used for wide range of products and shops Assumptions
added into its recommendation list. A recommendation sys-
tem mainly used in various concerns like movies, videos, 1. It is not compulsory for a customer to give rating value
music, news, websites, clothes, Twitter pages, hotels, res- for at least one product, and the matrix is very sparse
taurants, travels, tourist places and etc. Most of the major matrix.
enterprises are using recommendation system to elaborate 2. One customer cannot provide rating value to all the
their business, enrich customer experience. product.
The above Table 1 shows the relationship among the cus- 3. If the customer gives a rating value to one product, then
tomer Ci and the product Pj as a matrix. The rows in the it is assumed that the customer has experience on the
Table 1 represents the customer, the column represents the product.
product, and the cells represents the rating value Aij given by 4. Number of customer and the products is not limited.
the customer. It is assumed that n number of products and k 5. One shop can sell any number of products, and any
number of customers are involved in the paper. The term Aij shops can sell any product.

Fig. 1  Recommendation
systems

13
Complex & Intelligent Systems (2020) 6:621–634 625

Collaborative filtering is a procedure that is used to fil- customer who has not yet heard about the product. It is illus-
ter out components that a customer will like on reaction trated and can be understood easily from Fig. 2.
basis by user similarity. It is deployed on the basis of find- Among the various recommendation system, this paper
ing a larger set of people and searching a minimal group of focused on using collaborative filtering method. It is well-
customers with similar tastes to a particular user. known that the collaborative recommendation system rec-
Cognitive filtering is another term illustrating the ommends a product or shop based on the rating value given
content-based filtering; it suggests the user on the basis by the customer. For example, there are three shops, four
of items content and the profile of customers. The items products, three old customers and one new customer are
content is referred as a group of parameters. involved in the proposed scenario. All the three shops selling
Similarity based modelling is defined as a procedure four products, but one product is selling by only one shop.
wherein a normal procedure of system is performed for Similarly, three customers buying three products, but only
detecting the errors by examining their similarity to sys- one product is buying by only one customer. So, based on
tem’s normal states. that, product—three is recommended to the new customer.
Matrix factorization is a course of collaborating vari- It means one product is selling by more shops and buying by
ous filtering algorithms utilised in recommendation sys- more customer can be recommended to the new customer.
tem. Its works by neutralizing the interaction of user item This scenario is used for recommending a product, and it
into component of two different minimal dimensionality illustrates the collaborative filtering method.
matrices. The idea behind this method is, the shops and the cus-
For example, assume that one customer cannot provide tomer who have accepted the product in past and present can
even to 1% of total products. Hence, approximately 99% of agreed to the future as well. Here the first three products are
cells do not have any value. These empty cells are filled by accepted by the shops and the customer and is worth for buy-
dots “…”, means, it is not a number. In case, if n = 1-million, ing. If the entire system is true, then making a collaborative
and k = 10 k, then n * k = 1010, is a big number. If, an aver- system is easy and suitable for the application.
age number of customers gives rating to 5 products then, 5 *
1-million = 5 * 106 ratings. So, the customer-product matrix Product–product similarity
becomes a sparse matrix.
Due to the number of ratings filled in the matrix is com- This product–product (P-P) similarity construction is also
paratively less, this paper is mainly aimed to use the recom- similar to customer-customer similarity [15]. It is also an
mendation system, because the recommendation system can imagination by inspecting Fig. 5.
obtain the recommended product very fast than the other P-P similarity can identify the best products which can
systems. One of the types of recommendation systems is be recommended for new customers. To understand the
collaborative filtering where it focused on collaboration functionality of P-P similarity method, a similarity matrix
among the customer. If a greater number of shops are linked is constructed. Calculating P-P similarity is similar to cus-
with the products/items, they can be recommended to the tomer–customer similarity, but the customer–customer

Fig. 2  Collaborative recommen-

dation based product recom-
mendation

13
626 Complex & Intelligent Systems (2020) 6:621–634

Fig. 3  P-P similarity

similarity is very complicated to compare millions of cus- lower Euclidean distance amongst two customers has higher
tomers associated with the product. The product is recom- customer similarity whereas higher the distance defines the
mended by the customer, who has more experience. Figure 3 degree of similarity between tow customers if minimal.
shows the customer who is recommending the third product, The rows and the columns in the P-P matrix is repre-
where comparing with the other two, third one is the best in sented by products and customer. There are n number of
number of products and has a lesser cost. To find out simi- products given in each column and the m number of cus-
lar products based on the customer ratings, P-P similarity tomers given in each row. Each cell in the matrix shows the
matrix is constructed and it is given in Table 2. Similarity ratings Simij given by the customer Ci for the product Pj .
amongst two different products is the Euclidean distance in The symmetric matrix n × m is constructed using the rating
space on which its dimensions defines the products features. value, is the similarity between two products. The similar-
If there exists the minimal distance amongst the product ity is calculated using cosine similarity value between two
then the products similarity is higher, while higher distance products. Two products are selected based on the similar
will illustrate the minimal degree of similarity. Like product and high ratings given by the customer. In the cosine simi-
similarity, customer similarity also has the value where the larity values such as, ‘1’ signifies the maximum similarity

Table 2  P-P similarity matrix

13
Complex & Intelligent Systems (2020) 6:621–634 627

and ‘0’ signifies the lowermost similarity. Simij represents Data collection model of shoppers shop system
the ith customer ratings for jth product.
Cosine similarity is utilised as a parameter for calculat- Using the collaborative and P-P similarity calculation the
ing the distance where the vectors magnitude is not taken proposed approach processes the testing data and provide
into account. It is much easier for using it for data with the recommendation results. Social media and platform have
text demonstrated by word count. Error in testing is mini- now become the most utilized means to communicate any
mal using cosine normalization than the weight, layer and information among people. There is a huge impact of social
various other normalization. Cosine cantered normaliza- platforms such as Twitter, Facebook and WhatsApp across
tion further lesser the error in testing. This normalization the daily routine of a normal person. The trend has changed
using cosine is very stable when compared to other non- to such an extent that people share their experience, mood
normalization methods. Quantum computing makes sam- and current state of mind either in the form of a status or as
pling of matrix easy for making recommendations. Here a review comment. These status or review comments gets
in this paper, quantum method that samples from every shared across a chain of linked customers and in turn acts
row (U ̂h )j in a matrix mn . Normally, quantum algorithm as an indirect means of marketing the customer experience.
will not yield the row (U ̂h )j , which itself takes linear in Figure 4 depicts this scenario where the customer shares his
dimension n with time, but uses row for sampling. But it experience by posting his experience across social media
is perfectly what every recommendation requires: sample platform such as twitter or Facebook.
a greater value component of the row, somewhat clearly The proposed shoppers shop system is focussed towards
output the row entirely. More correctly, the method of gathering the insightful information pertaining to the cus-
quantum procedure is effectively described which states tomers shopping experience in a shop which may be a retail
that it takes input as a matrix, vector and a threshold con- outlet, super market, retail distributor shop, online purchase
straint and produces the corresponding quantum state for etc. Figure 5 depicts the block diagram of a shopper’s shop
vector projection onto the spanned space by singular row system. The customer has two modes of performing the
vectors of matrix whose value of singular is higher than shopping namely online mode where in the customer pur-
threshold. Using this procedure, the output is clear on how chases the product by placing orders through online web-
to produce the sample by calculating the quantum state on sites or a direct mode where in the customer visits the shop
the basis of computation. which may be a retail outlet, distributor stall, shopping mall
etc. The customer then shares his opinion or experience by
posting the same across any social media portals such as
Finding similar products using P‑P similar matrix Twitter, Facebook, Instagram, WhatsApp etc. as a status or
as comments or feedback or sometimes pictures with some
For example, the customer C10 likes the product P1, P7 and emotions. All these experiences are recorded through the
P15. Then the P-P matrix provides the similar products as, website in a social media database.
The initial step will be data extraction. Twitter or any
P1 = {P4 , P5 , P6 },
other social media database comprises of huge chunk of
P7 = {P4 , P8 , P9 }, data pertaining to all the customer transactions. The pri-
P15 = {P10 , P11 , P12 }. mary objective of data extraction is to extract the data
pertaining to the customer transactions related to shopping
Product P1 similar to P4, P5 and P6, P7 is similar to P4, alone, thus extracting only the essential data for further
P8 and P9, and P15 is similar to P10, P11 and P12. Now processing and discarding the remaining data from being
P4 is the common product where it is similar to P1 and P7, considered for further processing. The extracted shopping
hence P4 is recommend to customer C10. Similar to the related data are then subjected to pre-processing. Data pre-
above, the P-P similarity works. One of the major advan- processing comprises of considering the extracted data as
tages of P-P similarity is the ratings given for a product can- input and involves removing redundant, duplicated and
not be changed after predefined period. For example, con- unwanted noisy data to ensure that only data relevant to
sider the song “Believer”. In the initial period, most of the
people gave ratings of 3–4 out of 5. But after a month, peo-
ple realised that the “Believer” is a great song and they are
social Posting user
not able to change. As a rule of thumb, number of customers
platform experience
is more than products, if the product ratings do not modify
much over time after the period, then P-P similarity-based
recommendation system is highly preferred for shop-shop
Fig. 4  Current scenario where customer shares his experience in
based recommendation. social media

13
628 Complex & Intelligent Systems (2020) 6:621–634

Fig. 5  Overall functionality of

the proposed recommendation
system shopping Online Opinion

Direct Experience

social
website

web data

Data Extraction Pre- Normalization

using API Process

Feature Feature
Classification
selection extraction

the context percolates to the next stage. Thus, all the irrel- Feature extraction
evant data are completely removed and only required data
remains in the system for further processing. In the context Data extraction demonstrates some original transformation
of this work, pre-processing is highly essential in ensuring of features to produces certain other features that has more
that duplicate remarks from the same customer pertain- momentous. Feature extraction is normally utilised to con-
ing to a same shop and for the same product is removed. struct mean of linear combination ∝ Ry of unceasing fea-
Also, all the irrelevant information is removed as part of tures which consists of discriminant. It is mainly utilised in
this stage. the context of reducing complexity and forecast some useful
Pre-processing is followed by normalization. Normali- information from the gathered data. Here the shopping data
zation is a technique where in the data is transformed and gathered are enforced for data extraction as it may include
fitted into a scale that is highly suitable for the system. The some redundancy in it. Extraction techniques such as prin-
primary objective of normalization is to transform the data ciple component analysis may be used to analyse the draw-
into the best suitable form such that it enables easy data backs in the database and produce a solid data for further
analytics and easier processing. Data analytics in shop- investigation used the proposed model.
pers shop application comprises of feature extraction and Once, the feature extraction and feature selection are
feature selection. completed, the refined data is then subjected to classifica-
Feature extraction is the process of extracting essential tion. The classification model in this shopper’s shop system
features that are needed for data analysis. The data may uses machine learning based approach namely regression-
comprise of n number of features. However, there may based recommendation system to perform multi class clas-
be certain salient features using which the data can be sification. The classifier analyses the five selected features
completely reconstructed. In the case of shopper’s shop and based on the values, determines whether the shop where
system, the shopping data may consist of different features the product was purchased provided good, average or bad
and each feature may comprise of different dimensions. customer shopping experience.
Feature selection comprises of selecting only the essential
features from the extracted features of the data. Only five Experimental results and discussion
features namely Manufacturing date (MFD), Manufactur-
ing price (MRP), discounts and offers, quality ratings and The proposed machine learning approach is implemented,
suggestions or reviews are considered as salient features experimented, and the results are verified. Programming
and the information pertaining to the entire shopping and executing the proposed approach, the system con-
transactions in a shop can be depicted using these fea- figuration used is given in Table 3. The dataset is taken
tures and the customer experience can be portrayed using from various data publicly available in “https​://jmcau​ley.
these features. ucsd.edu/data/amazo​n/links​.html” [16, 17]. The complete

13
Complex & Intelligent Systems (2020) 6:621–634 629

Table 3  System requirement for Element Performance where, CID, PID and rating are integers. The rating range is
experiment between 1 and 5. In general, the ratings are given as stars. In
OS Windows this paper, to reduce the computational complexity, the num-
CPU Intel Core ber of stars is counted and an equal integer value is given.
i7, 7th Gen
Processor
For each product and the customer, it is essential to predict
@3.2 GHz the ratings given by the customer. The recommendation
Memory 8 GB RAM system is considered as regression problem. The machine
HDD 1 TB learning algorithm is used for minimising the RMSE.
Software MATLAB Modelling the recommendation system with the help of a
Data set Amazon— regression model using mapping function (f) with approxi-
product mation from the input variable (Y) to an output variable (X)
recommen- which is continuous. Real value is the variable in the output,
dation data
sets
for instance it may be integer or floating value. These are
represented often by quantities such as sizes and amounts.
Various models can be adopted for recommendation system
but linear regression is utilised for analysing the best hyper
explanation about the dataset is given in [18]. The dataset plane which makes the prediction of data effective.
comprises of various product information, reviews and
meta information taken from Amazon, Shop clues and Objectives
Flicker websites. The total number of reviews available
in the dataset is 142.8 million has been given up to 2014. It can be obtained only by predict the rating values given by
Including the data collected from [16–18], various product the customer to a product which is not already rated. One of
dataset given in [19] is also taken for experiment. From the main objectives of this experiment is to calculate the real
the experiment the performance of the proposed ML algo- rating value by finding the difference between actual rating
rithm is verified. Since, ML based predicting the ratings is (MAPE) and predicted rating (RMSE).
true, the error values are calculated for finding the incor-
rect recommendation output. It takes very less time and
fast, because the number of incorrect recommendations is Constraints
very less than the correct one. The error values are calcu-
lated like supervised learning methods discussed in [20, The application and the data have a defined interoperability.
21]. The performance comparison is more accurate since There is no recommended product with less latency have
the dataset has predefined recommendation result. It is not been precomputed in the existing researches.
assumed that there is a portion of recommendation results
is not correct. Using the difference between the correct Type of data
and in-correct answers is calculated and it is defined as
error value. The volume of data collected is more than 50,000 and
The main objective of this paper is to create a well suit- we obtained only good, complete data for the experi-
able and highly applicable recommendation system for shop- ment to avoid manual and computational errors with time
pers’ shop. To improve the efficiency and provide an optimal complexity.
solution for recommendation system, this proposed HRS use
the collaborative and P-P similarity algorithms for predicting 1. There are 35,500 unique product ID.
customer ratings-based product and shops. The problem is 2. There are 85,000 unique customer ID.
considered as the regression problem and it is implemented 3. Ratings with 1 to 5 values (it may be indicated in stars).
in MATLAB software and the results are verified. The vol- 4. Each product with the shop name, MRD, MRP, SP, qual-
ume of data used for training process is 100,400,000 ratings ity and ratings are collected.
where 400,000 customers given for 25,000 products. The
products include clothes, kitchen products, cosmetics, elec- Sample data (one row) from the data file are given in
trical and electronic items used in our daily life. The format the following Table 4. The data selected based on the best-
of the data (see Table 3) used in the training process is selling price, quality and ratings, the product and shop name
are given in Table 4, which is used for experiment. It says
< customer-ID(CID), product-ID(PID), rating, date, shop >
that MRD is the latest date, MRP is the original given by the
production company, SP is given by the shop after offer, the
quality of the product and the highest ratings.

13
630 Complex & Intelligent Systems (2020) 6:621–634

Table 4  Data size: category wise The data are clustered based on the categories, product
Product category Number of products Total number of ratings dataset has a greater number of products and sub items.

Electronics 6,498,196 27,824,482 Process on the data

Beauty 9,259,204 12,023,070
Food 1,671,760 19,297,156 Initially read the datafile, eliminate the unwanted spaces,
Clothes 19,503,384 5,748,920 trailing semicolon, and separate each field by comma. The
Total 36,932,544 64,893,628 time taken for compiling and executing each process of the
total proposed approach is given in the Tables 5 and 6 for
understanding the time complexity.
Table 5  Sample data Next, the basic statistics of the data are considered for
Sr. no CID PID Rating Date Shop
verifying the efficiency of the approach. The size of the total
data, data used in training process and testing process are
1 1209954 093456 5 2005–05–09 0942 computed and given in Table 7. The data size is verified
2 2263586 085632 4 2004–08–20 0452 from actual data collected, divided for training and testing
3 1009622 082843 4 2005–01–19 0942 process. It can determine the efficiency of the proposed
4 1910569 075634 3 2004–04–12 0942 approach. From Table 7, it is noticed that, the data loss
5 401047 081263 4 2005–06–03 0983 before and after processing is not high, and ML algorithm
saves the data.
Then the result obtained from the experiment is clustered
Table 6  Time complexity data. After clustering, the data are arranged (it is given for
Process Time (min)
the sample data taken from the actual data). The product is
clustered based on the product-categories such as cosmetics,
Reading and converting into known format 1.45 groceries, Medical, ready-food and etc. Some of the products
Preprocessing 3.87 are clustered under same cluster with high ratings are recom-
Clustering 7.34 mended. Then those products selling by similar companies
Product recommendation 2.9 are considered for recommendation. In this paper Table 8a
Shop recommendation 1.9 and b shows the products and the related shops are recom-
mended for the new customer (highlighted in the Table).

Table 7  Data size verification

13
Complex & Intelligent Systems (2020) 6:621–634 631

Table 8  a Product based

clustering, b Product – shop
based clustering

For example, the product id 093456, 082843 have testing process. Testing process is applying the same process
obtained rating value 5 and both the products is selling by applied in the training process and obtain the recommenda-
the shop amazon. Based on the rating value and the simi- tion result as “1” or “0”.
lar shop the shopper’s shop is amazon. Since the product The size of the total data and the ratings are high, it is
has high rating value and both the products are selling by not able to count it by simple addition or multiplication
Amazon, it is recommended by the hybrid recommenda- method. To obtain the accurate ratings initially the rating
tion system. Similarly, the entire dataset is processed. To distributed from 1 to 5 is calculated for all the products.
improve the quality of the recommendation 40% of the It makes easy to classify the ratings for a specific product
dataset is used for training process and the recommended and shop. Figure 6 shows the rating distribution of the
products are updated for future testing process. Whenever a training data set. The ratings have been categorized into
new product or customer looks for a product it is compared five values with 1 being the least rating and 5 being the
with the trained process and apply recommendation. If the maximum rating value across different products (Table 9).
product is not available in the list then it will be applied for

Fig. 6  Rating distribution Rating Distribution on Training Data

calculation
16

14
Rating Values (millions)

12

10

0
1 2 3 4 5
Range of Ratings (1 to 5)

13
632 Complex & Intelligent Systems (2020) 6:621–634

Table 9  Product versus recommendation of differences persisting between two continuous variables
Product category Total number of Number of while performing regression analysis. MAE provides the
products products recom- average vertical distance between a variable and identity
mended line. Figure 7 strikes a comparison of MAE values for the
proposed Hybrid Recommendation System (HRS) and its
Food 10,000 9899
contemporary methods. It can be observed that the MAE
Clothes 10,000 9945
values are significantly lower when compared to other
Electronics 10,000 9965
approach which is a clear indicator that the proposed HRS
Kitchen 10,000 9389
outperforms other existing approaches.
Mean absolute percentage error (MAPE) is used
to denote the prediction accuracy of any forecasting
The set of all performance measures used to determine approach. Greater the MAPE value, the accuracy is said
the accuracy of the proposed approach is MAE (Mean to be precise. Figure 8 strikes a comparison of MAPE
absolute error), MSE (Mean squared error) and MAPE values with existing methods and it can be observed that
(mean absolute percentage error), and are calculated using the proposed HRS system has higher values of MAPE and
the following equation. the accuracy is nearly 98%. Thus it can be observed that
Mean absolute error is used to estimate the inaccuracy the proposed Hybrid Recommendation System is highly
amongst combined findings expressing the similar phenom- accurate and provides accurate customer recommendations
enon. MAE is similar metric used for estimating the error when compared to other contemporary approaches such as
caused in forecasting time series analysis, sometimes utilised collaborative filtering with machine learning (CBF-ML),
in confusion having huge standard definition. smart recommender hybrid system of learning (SRHL) and
n Collaborative filtering with machine learning (CF-ML).
1 ∑| | Mean Square Error (MSE) is the deviation of the square
MAE = |d − ̂
di |
n 1 | i | of the estimated value from that of the actual value. This
is another key metric that determines the deviation and
n provides a clear indicator in terms of determining the
100 ∑ di − ̂di
MPAE = accuracy of the estimation technique. Lesser MSE value
n 1 di
indicates very minimal deviation between the estimated
and observed values. Figure 9 provides a comparison
1∑
n ( )2 of MSE values of the proposed HRS with other existing
MSE = di − ̂
di approaches. It can be observed that the MSE value for
n 1
HRS is 6 which is significantly lesser when compared to
where di represents the actual rating value, ̂di represents SRHL which has 7, CBF-ML has 13 and CF-ML has 17 as
the predicted rating value,n represents the total number of MSE value. Lower the MSE value higher the performance
products. Mean Absolute Error (MAE) denotes the measure of the recommendation system.

Fig. 7  MAE comparison MAE Comparison

1.6

1.4

1.2
Obtained MAE Value

0.8

0.6

0.4

0.2

0
CF + ML [Ref. 18] CBF + ML [Ref. 18] SRHL [Ref. 18] HRS
Various Methods

13
Complex & Intelligent Systems (2020) 6:621–634 633

Fig. 8  MAPE comparison MAPE

100
98
96

Obtained MAPRE Value

94
92
90
88
86
84
82
80
CF + ML [Ref. 18] CBF + ML [Ref. 18] SRHL [Ref. 18] HRS

Various Methods

Fig. 9  MSE comparison MSE

20

18

16
Obtained MSE Values

14

12

10

0
CF + ML [Ref. 18] CBF + ML [Ref. 18] SRHL [Ref. 18] HRS
Various Methods

Conclusion intervention of human element involved when it comes to

predicting the customer choice of shops.
This work was directed towards developing a suitable The performance of this Hybrid Recommendation System
hybrid recommendation system that can self-study the was evaluated using three metrics namely MAE (Mean abso-
customer shopping data that is available in the form of lute error), MSE (Mean squared error) and MAPE (mean
reviews, understand the pattern and predict the interest of absolute percentage error) and the results were compared
customers towards buying a particular product in selected and analyzed with other contemporary approaches. These
shop. Five essential features were extracted from the results that were discussed in the previous section show
customer data and using this, the Hybrid Recommenda- that the MAE value of HRS is significantly lower than the
tion System (HRS) was designed using machine learning existing approaches that were compared. Also, the MAPE
based regression model. This system has been found to be value for HRS was nearly 98% which is a clear indicator
instrumental in classifying the preferred choice of shops of a high degree of accuracy. Similarly, the MSE value for
based on the products purchased by the customer. The HRS showed minimal deviation which is another strong
salient feature of this HRS approach is that there is zero indicator of high degree of precision and accuracy. Thus, it

13
634 Complex & Intelligent Systems (2020) 6:621–634

can be concluded that the proposed Hybrid Recommenda- 8. Liu S, Cheng X, Li F, Li F (2015) TASC:topic-adaptive sentiment
tion System (HRS) clearly outperforms other contemporary classification on dynamic tweets. IEEE Trans Knowl Data Eng
27(6):1696–1709. https​://doi.org/10.1109/TKDE.2014.23826​00
approaches in terms of accurate prediction of customer sen- 9. Bouazizi M, Ohtsuki T (2019) Multi-class sentiment analysis on
timent with respect to shopping a product in a particular twitter: classification performance and challenges. Big Data Min
shop. Future work can be extending this approach towards Anal 2(3):181–194. https​://doi.org/10.26599​/BDMA.2019.90200​
gathering the customer interest for multiple products across 02
10. Trilla A, Alias F (2013) Sentence-based sentiment analysis for
different geographical locations. expressive text-to-speech. IEEE Trans Audio Speech Lang Pro-
cess 21(2):223–233. https:​ //doi.org/10.1109/TASL.2012.221712​ 9
Open Access This article is licensed under a Creative Commons Attri- 11. Yu D, Xu D, Wang D, Ni Z (2019) Hierarchical topic modeling
bution 4.0 International License, which permits use, sharing, adapta- of twitter data for online analytical processing. IEEE Access
tion, distribution and reproduction in any medium or format, as long 7:12373–12385. https​://doi.org/10.1109/ACCES​S.2019.28919​02
as you give appropriate credit to the original author(s) and the source, 12. Lamb AF, Varadarajan M, Tran R, Vandier N, Doshi BL, Bear C
provide a link to the Creative Commons licence, and indicate if changes (2012) The vertica analytic database: C-store 7 years later. arXiv​
were made. The images or other third party material in this article are :1208.4173
included in the article’s Creative Commons licence, unless indicated 13. Greenplum Database. green​plum.org/
otherwise in a credit line to the material. If material is not included in 14. Solutions TW (2002) Teradata Database technical overview, pp
the article’s Creative Commons licence and your intended use is not 1–7. http://www.terada​ ta.com/brochu​ res/Terada​ ta-Soluti​ on-Techn​
permitted by statutory regulation or exceeds the permitted use, you will ical-Overv​iew-eb302​5
need to obtain permission directly from the copyright holder. To view a 15. Data Driven Investor – Medium (2020) https:​ //medium ​ .com/datad​
copy of this licence, visit http://creat​iveco​mmons​.org/licen​ses/by/4.0/. riven​inves​tor. Retrieved 6 June 2020
16. Ni J, Muhlstein L, McAuley J (2019) Modeling heart rate and
activity data for personalized fitness recommendation. In:
WWW’19: proceedings of the 2019 World Wide Web confer-
References ence, San Francisco, CA, USA, May 2019
17. He R, Kang W-C, McAuley J (2017) Translation-based recom-
1. Jianqiang Z, Xiaolin G, Xuejun Z (2018) Deep convolution neural mendation. In: Proceedings of the eleventh ACM conference on
networks for twitter sentiment analysis. IEEE Access 6:23253– recommender systems, 2017
23260. https​://doi.org/10.1109/ACCES​S.2017.27769​30 18. McAuley J (2020) Recommender systems datasets. https​://csewe​
2. Jianqiang Z, Xiaolin G (2017) Comparison research on text pre- b.ucsd.edu/~jmcau​ley/datas​ets.html. Retrieved 6 June 2020
processing methods on twitter sentiment analysis. IEEE Access 19. Kretser A, Murphy D, Starke-Reed P (2017) A partnership for
5:2870–2879. https​://doi.org/10.1109/ACCES​S.2017.26726​77 public health: USDA branded food products database. J Food
3. Bouazizi M, Ohtsuki T (2018) Multi-Class sentiment analysis Compos Anal 64:10–12
in twitter: What if classification is not the answer. IEEE Access 20. Elahi M, Ricci F, Rubens N (2016) A survey of active learning
6:64486–64502. https​://doi.org/10.1109/ACCES​S.2018.28766​74 methods in collaborative filtering recommender systems. Comput
4. Bouazizi M, Ohtsuki T (2017) A pattern-based approach for multi- Sci Rev 20:29–50
class sentiment analysis in twitter. IEEE Access 5:20617–20639. 21. Nouh RM, Lee H-H, Lee W-J, Lee J-D (2019) A smart recom-
https​://doi.org/10.1109/ACCES​S.2017.27409​82 mender based on hybrid learning methods for personal well-being
5. Ebrahimi M, Yazdavar AH, Sheth A (2017) Challenges of senti- services. Sensors 19(431):1790–1801
ment analysis for dynamic events. IEEE Intell Syst 32(5):70–75.
https​://doi.org/10.1109/MIS.2017.37116​49 Publisher’s Note Springer Nature remains neutral with regard to
6. Iqbal F et al (2019) A hybrid framework for sentiment analysis jurisdictional claims in published maps and institutional affiliations.
using genetic algorithm based feature reduction. IEEE Access
7:14637–14652. https​://doi.org/10.1109/ACCES​S.2019.28928​52
7. Tan S et al (2014) Interpreting the public sentiment variations on
twitter. IEEE Trans Knowl Data Eng 26(5):1158–1170. https​://
doi.org/10.1109/TKDE.2013.116

13
Reproduced with permission of copyright owner. Further reproduction
prohibited without permission.