Implementation of Real Time Activity Sensing
Implementation of Real Time Activity Sensing
(2022) An
implementation of real-time activity sensing using wi-fi: identifying optimal machine
learning techniques for performance evaluation. IEEE Sensors Journal, 22(21), pp.
21127-21134.
For the purpose of open access, the author(s) has applied a Creative Commons
Attribution license to any Accepted Manuscript version arising.
https://eprints.gla.ac.uk/277562/
implemented using ambient Wi-Fi signals already present in of this work but removes the need for wearable devices.
many homes [13]. This means that the cost is inexpensive and The authors of [15] used RF sensing to detect four people
there is no intrusiveness of additional equipment in the home. performing activities. The work was able to have the highest
It has become acceptable for homes to have Wi-Fi systems accuracy of 91.25 % accuracy when using a single subject.
present. Currently, RF sensing research looks at how signals Our work will seek to implement real-time sensing using a
represent movements and how AI can classify movements. single subject. The paper [16] detailed how falls in the homes
There is a lack of research on implementing a real-time of elderly people can be detected by using Ultra Wide Band
working system. This paper looks to introduce the use of RF (UWB) radar technology. This, therefore, provides a contact-
sensing into a working monitoring system. less sensing fall detection system and the results of the paper
The contributions in this paper can be summarised in the show that this proposed approach can be applied in real-time
following points: by streaming incoming data from the UWB device. The paper
1) A real-time analysis of the monitoring system looking at aims to distinguish between normal activities and falling. The
the timing of machine learning algorithms. Fast classifi- system is beneficial for fall detection but is not designed
cation timing is essential in real-time AI applications as to provide broader monitoring of elderly people’s activities
the AI model needs to be able to perform classifications which may be indicative of a fall risk rather than providing
before the next sample of data is received from the notification of a fall that occurs when harm may have already
continuous sensing system so that warnings of critical been caused to the individual. In our paper, we seek to make
events can be reported in real-time. use of the cheaper more available WiFi technology compared
The development of a monitoring system that makes to this work using radar technology. Aziz Shah, Syed, et al.
use of RF signals to accurately sense if a person is [17] made use of contactless sensing using Wi-Fi for real-
present in a room and performing movements within the time classification of human activities picking up an object;
monitored room. Machine learning is used to recognise walking; sitting on a chair; jogging and leaving a room.
patterns of signal amplitude associated with the signal The work takes the amplitude of the Wi-Fi subcarriers and
reflection of the human body. This will serve to provide a converts them to scalogram images and passes these images
significant monitoring system that can be used to detect through deep learning algorithms. The deep learning provided
disruptions to elderly people’s expected activities. If an accuracy result of 91.1 %. The paper mentions the use
there are changes in expected activities, then this can be of real-time but does not detail the process of how a real-
indicative of further issues and health care professionals time approach is carried out. One of the main focuses of this
or family members can take further actions if required. paper is the privacy factor of activity classification. Our paper
2) A dashboard which displays the current classification expands on the ideas detailed in this work and implements a
from the real-time system. A dashboard is an important working real-time system to provide continuous monitoring.
feature of any monitoring system as it must communi- Zhang, Daqing, et al. [18] present a real-time fall sensing
cate the classification of the AI model to family mem- system using Channel State Information (CSI) of RF signals.
bers and healthcare professionals so that appropriate The classifications are made using Support Vector Machine
action can be taken to care for any irregularities detected (SVM). The set-up uses a camera to serve as ground truth
by the system. The dashboard is designed with simplicity while the CSI stream records the data being collected. The data
in mind. There are minimal elements that show only the is then labelled using the camera footage as a reference. This
important aspects required to report. The classification shows that the data is collected in real-time and classifications
is shown as a heading and there is an image of the RF are made on the real-time data. The paper does not detail
signal which can show how much disruption the event how long it takes for a classification to be made following
has caused to the RF signals. all the data processing steps detailed in the paper. Therefore
our paper seeks to provide an analysis of machine learning
algorithms to select an appropriate algorithm to provide a real-
II. R ELATED W ORK time classification of new incoming data. The authors of [19]
The authors in [14] developed a real-time activity moni- made use of RF signals to sense sit down, stand up and static
toring system using wristbands paired with mobile devices. stand activities in real-time. The paper made use of Linear
The work makes use of data fusion to combine data from Discriminant Analysis and SVM machine learning algorithms.
both wristbands and mobile devices. Data fusion provides a The authors of this paper considered the timing component
contingency for when users forget to wear the wristband, it is of the algorithms concerning the sample rate. The results
still possible that the user will have their mobile device on their concluded the AI model created could make classifications
person and vice versa. The work included a diverse dataset of within 300 milliseconds with an accuracy of 70 %. The work
many different activities which is possible by using the many of our paper will seek to increase the accuracy of a real-time
sensors included in the mobile device and the wristbands. system by reducing the complexity of detected movements and
The smartphone is also able to identify the location of the keeping the classification time in milliseconds.
person and thus provides the presence or non-presence of the The paper [20] made use of Wi-Fi to sense that social
person. However, the paper still relies on wearable devices distancing is present in elevators and office space with an
and will be ineffective without users wearing the devices. accuracy of 92 % for the elevator and 97 % in the office space.
Our proposed system seeks to provide the monitoring aspect This is achieved by using the CSI to calculate the number
AUTHOR et al.: PREPARATION OF PAPERS FOR IEEE TRANSACTIONS AND JOURNALS (FEBRUARY 2017) 3
of occupants within an elevator and office space. The paper is processed for machine learning classification and output.
made use of description features of the CSI amplitude like the Figure 2 details the system architecture and data flow including
work of our paper. The features they used to describe the CSI data collection, data preprocessing, classification and output
amplitude were mean, standard deviation, MAD, min, max, display on the web interface.
skewness, kurtosis, and entropy. The paper did not compare
the difference in results between using describing features A. Hardware Setup
against the full CSI amplitude data which the work of our
This work makes use of an X300 Universal Software-
paper looks to add to current literature. The authors of [21]
defined Radio Peripheral (USRP). As seen in Figure 3, the
recorded the timings of each algorithm in the application
person performs activities in front of the USRP and the CSI
of detecting diseases by use of medical records as datasets.
is transferred to the computer using 1 Gigabyte Ethernet. Table
Algorithms tested like this paper was Support Vector Machine,
I lists the parameters used to configure the USRP device for
K Nearest Neighbours and Random Forest. Results showed
communication.
that on average Support Vector Machine was the fastest at
classifying the data. Our paper provides a similar analysis in
the field of activity detection using RF sensing.
III. M ETHODOLOGY
This section details the hardware setup, software config- B. Software Configuration
uration, data pre-processing and dashboard design. Figure 1 This paper makes use of GNU radio open-source software
displays the process of real-time activity monitoring with running in an Ubuntu virtual environment. GNU radio is used
the human movement taking place between the transmitter to create a python script using the parameters listed in Table
and receiver. The amplitude CSI of a selected subcarrier I, to define the USRP operation. The output of this script
4 IEEE SENSORS JOURNAL, VOL. XX, NO. XX, XXXX 2017
TABLE I: Parameters set for USRP OFDM Communication of each of the pilot subcarriers. standard deviation is selected
Parameter Value as the metric to show this as it provides a measure of variation
Number of Subcarriers 64 amongst the group of pilot subcarriers. subcarrier 40 was
Pilot carriers 12,26,40, 54
Sample Rate 400 kHz
shown to have the highest standard deviation value in all
Centre Frequency 2.4 GHz collected movement samples.
Bandwidth 20 MHz 2) Pre-processing of Selected Sub Carrier: The pre-
processing stage consists of two steps to finalise the data for
Machine Learning. The first step is to filter the data and then
contains the CSI information. The Raw output is piped to a once a filter is taken, the features can be extracted. A butter-
buffer. A second python script is used to read the buffer and worth low-pass filter with an order of 1 and a cut-off frequency
extract the CSI information. of 0.5 is applied to the data to remove high-frequency noise
from the signal. high-frequency noise is not the result of
C. Data Collection human activity as human activity is detected at low frequencies
[25]. Figure 4 shows a comparison of a movement sample
The training data is collected from a real-time stream. Three
before and after the pre-processing.
phases of data collection are completed for each of the three
classifications. The three classifications used in this model
are an ”empty room”, ”no activity” and movement. Figure
3 is used for the initial demonstration of a real-time sensing
system using RF signals. The concept of using RF signals to
sense movements in real-time has been mentioned in works
such as [22] but our work seeks to contribute to the real-
time application of RF sensing monitoring. Our work looks
to implement multiple classifications in real-time, which are Fig. 4: Non-Filtered vs Filtered CSI data
missing from works [18] and [20]. For real-life scenarios,
multiple devices can be used as a transmitter and receiver and
The last step is to take the features of the filtered 1000
placed 2 metres apart. This can allow for a larger monitoring
CSI data points. Six features are taken to summarise all the
area. The CSI data is collected and saved in CSV format.
CSI amplitude data of subcarrier 40. The six features used
300 CSV files for each classification are collected. The data
are mean, max, min, kurtosis, skew, and standard deviation
collected for movement consisted of a person performing
[26]. These features describe the wave of the signal including
sitting and standing simultaneously while the USRP transmits
the peaks and size of peaks. Mean is used to show the mean
and receives RF signals until 300 CSVs of the movement
of each value. CSI samples with peaks caused by movements
action are collected. For ”no activity” the person remains still
will show a higher mean than samples with fewer peaks in
sitting for 150 samples and a further 150 samples of standing
amplitude. Similarly to the min and max values, where lower
still are collected. The reason for this is to allow the real-
movement samples will have a lower max value and may
time model to account for any differences between sitting still
display lower min values. Kurtosis and skew will also vary on
and standing still to remain as ”no activity”. For the ”empty
the shape of the peaks observed in the CSI samples. Standard
room” samples, participants leave the room while CSI data is
deviation will take note of the variance of the samples.
collected. Steps 1, 2 and 3 in Figure 2 show the steps taken
These features provide an effective way of highlighting the
in the data collection stage. The extracted data is then saved
differences between the 3 classifications. During the training
into CSV format for the training stage.
phase, this process is carried out for each sample and the
appropriate label is added. In the real-time process, the CSI
D. Proposed Real-Time Activity Monitoring System data is processed and then fed via the AI model to give the
1) Sub carrier selection: During the training phase of the prediction of the model.
real-time system, the data must be processed to improve the 3) Machine Learning: A selection of machine learning algo-
machine learning classification accuracy. This was performed rithms is tested against the data. The selected algorithms are
in two stages, the first is to select one subcarrier from the selected based on the related work using these algorithms with
64 captured ones. Subcarrier selection is used to remove CSI data. These algorithms are Random Forest [11], Bagged
redundant CSI subcarriers which can result in overfitting dur- Trees [27], K Nearest Neighbours (KNN) and SVM [28].
ing machine learning processes [23]. Overfitting is when too 4) Real-time Classification: This paper aims to provide users
much data is passed through the algorithm, and it causes the with the detection of movement as it happens. A real-time
algorithm to memorise the training data rather than recognise system is defined as a system in which input data is processed
patterns of the data. The selection of the subcarrier is decided within milliseconds so that it is available virtually immedi-
in two steps; firstly by looking at the pilot subcarriers which ately.
show the variations caused by the human movement as they In this experiment, the sensing data is the CSI and it is
are used to carry the pilot symbols in the Channel Estimation constantly being collected from received RF signals. The
process [24]. Subcarrier number 40 shows to have the most system works by using a window size of 1000 CSI data points.
variation. This is calculated by taking the standard deviation During the time of classifying the CSI window, another 1000
AUTHOR et al.: PREPARATION OF PAPERS FOR IEEE TRANSACTIONS AND JOURNALS (FEBRUARY 2017) 5
CSI data points are collected. 1000 CSI data points are equal
to approximately 2.5 seconds due to the 400 kHz sample rate
used in the RF communication. The real-time system must
classify the data before the new data is received. Therefore
the system must perform classification within the 2.5 seconds
time frame as the next CSI window will be complete by the
end of this time frame. Within this time frame, raw samples
are passed to code which will carry out all pre-processing of
the data and then pass the pre-processed data to the machine
learning model. The entire process is recorded by taking
timestamps before and after classification is provided. The
time of classification is taken on a virtual machine with an
i7 9700 3.00GHz CPU and 12GB of RAM.
To evaluate the timing performance 20 samples from each
classification are removed from the training data. The re- Fig. 5: Accuracy comparison of selected algorithms
maining 240 samples are used to train a model and that
model is used on the new testing samples and the time and TABLE II: Results from Comparison of Machine Learning
accuracy are recorded. Additionally, a real-time comparison is Algorithms
made between the different algorithms using full CSI data and
features to justify a choice in algorithm and between complete Algorithm Accuracy Classification Precision Recall F1
(%) Score
CSI data and the extracted features of the CSI data. Random 99.33 Empty 0.99 0.99 0.99
5) Dashboard Design: The Dashboard is created using the Forest Movement 1 1 1
Flask python package. Flask is a micro framework that uses No Activity 0.99 0.99 0.99
Bagged 93.11 Empty 0.86 0.95 0.90
python to create web pages. The web page is designed with Trees Movement 1 1 1
simplicity in mind. The reason for a simplistic design is to No Activity 0.94 0.85 0.89
refrain from distractions as this system is designed with alerts KNN 99.11 Empty 0.99 0.99 0.99
Movement 1 1 1
in mind. The web page is comprised of a title banner, a heading No Activity 0.99 0.99 0.99
declaring the classification of the data and a visual image of the SVM 98.22 Empty 0.99 0.96 0.97
current CSI to which the classification relates. Additionally, Movement 0.97 1 0.99
No Activity 0.98 0.99 0.99
for this paper, the web page displays the total time of the
classification at the bottom of the web page. This is used to
calculate the total time between stages 3, 4, 5 and 6 shown
in Figure 2. Python is used to take a timestamp at the start 9 groups are used as training data. Once all 10 groups have
of stage 3 and JavaScript is used on the web page in stage 6 been used as testing data once, then the classification results
to take the end timestamp and duration can be calculated by for all samples can be presented.
using these two timestamps. The Random Forest confusion matrix shows all the move-
ment samples were correctly classified. 299 out of 300 samples
were correctly classified as ”no activity” samples. The ”empty
IV. R ESULTS
room” correctly classified 295 samples. As expected, most of
A. Comparison of Machine Learning Algorithms the algorithm results have been able to establish the movement.
Figure 5 displays the accuracy of each of the algorithms The difficulty lies with differentiating between ”empty room”
when using 10-fold cross-validation. Table II shows the results and ”no activity” as this is where most of the errors in the
achieved after all data has been tested during each of the classification occur. The algorithms have classified most of
10 folds of the k-fold cross validation method including the the samples correctly for all three classifications. hence the
Accuracy, Precision, Recall and F1 score which are common high accuracy score achieved. For the implementation of a real
indicators of how machine learning algorithms perform. system, some inference can be implemented which will work
The results show that Random Forest performed the best on the assumption that a movement must take place before a
with an accuracy of 99.33 %. KNN had an accuracy result of room can be empty. This can eliminate the small error rate in
99.11 % and SVM had an accuracy score of 98.22 %. Bagged classifying between ”empty room” and ”no activity”.
trees showed the lowest accuracy score of 93.11 %. All the 1) CSI data vs Feature selection: The above results are
algorithms however were able to achieve an accuracy score of achieved by using features from the CSI data described in
over 90 %. This, therefore, indicates that the data of the three Section III-D.2. To justify the use of features in this paper
classifications are distinctive. Figure 6 shows the confusion a comparison is made between the processed CSI data and
matrix of the best-performing Random Forest algorithm. The extracted features. Figure 7 shows the comparison of accuracy
figure shows how all 300 samples were classified in each of between using the CSI data vs the Features.
the 10-folds during cross-validation testing. The 10-fold cross- The algorithms perform better when using the features of
validation testing method splits the data into 10 groups, with the CSI data except for SVM. Random Forest had the highest
each group taking a turn as the testing data and the remaining accuracy out of all results of 99.11 % when using features.
6 IEEE SENSORS JOURNAL, VOL. XX, NO. XX, XXXX 2017
V. C ONCLUSION
In this paper, a novel real-time sensing system is presented.
The system makes use of AI to identify the moving or presence
of a person in a room using Wi-Fi. The use of Wi-Fi ensures a
non-contact method of movement detection where a wearable
device is not required. A USRP device is set up to use Wi-
Fi between a transmitting antenna and a receiving antenna.
The Wi-Fi signals are then transmitted through the atmosphere
and when a person is present or moves the Wi-Fi signal is
disrupted. These disruptions are present in the CSI. Every 1000
CSI data points received at the receiver side is then processed
and passed to AI models to make a classification on whether
Fig. 9: Dashboard Output for all Three Classifications the disruption indicates an ”empty room”, the presence of a
person or movement. 300 training samples are collected for
each classification and the accuracy and computational time
The Dashboard is used to provide visualisation of the of 4 machine learning algorithms are compared. The results
classification produced from the AI model. The result of the AI are presented, and the justification is given for the choice
must be presented, and it is not expected that family members of algorithm to create an AI model. The AI model which
or care staff to read from terminal outputs. The result of the AI achieved 100 % accuracy on new unseen testing data is used
model is passed to the dashboard python code as a variable. in a dashboard to create a real-time monitoring system. Future
Then the dashboard loads the web page based on the contents work will seek to create an alert system where irregularities in
of the variable. Rules can also be set to create alerts if any movements can be flagged up as an alert. Which will create
activity is detected which is not considered regular and then a more effective monitoring system which will not require
raise an alert. These alerts can be used to assist. The dashboard constant monitoring of a dashboard.
displays the detected classification and a visual representation
of the CSI. Figure 9 shows all of the classes outputted on the
R EFERENCES
dashboard with images and time for classification. It can be
seen how the amplitude is affected in the movement sample. [1] Maria Vittoria Conti et al. “Bioactive compounds in
The difference between ”empty room” and ”no activity” is legumes: Implications for sustainable nutrition and
less clear to visualise but it can be noted that the ”empty health in the elderly population”. In: Trends in Food
room” sample has a slightly higher amplitude than the ”no Science & Technology (2021).
activity”. This is the reason why most errors in classification
8 IEEE SENSORS JOURNAL, VOL. XX, NO. XX, XXXX 2017
[2] Soe Ye Yint Tun, Samaneh Madanian, and Farhaan [17] Syed Aziz Shah et al. “Privacy-preserving non-wearable
Mirza. “Internet of things (IoT) applications for elderly occupancy monitoring system exploiting Wi-Fi imaging
care: a reflective review”. In: Aging clinical and exper- for next-generation body centric communication”. In:
imental research 33.4 (2021), pp. 855–867. Micromachines 11.4 (2020), p. 379.
[3] Francisco Luna-Perejón et al. “IoT garment for remote [18] Daqing Zhang et al. “Anti-fall: A non-intrusive and real-
elderly care network”. In: Biomedical Signal Processing time fall detector leveraging CSI from commodity WiFi
and Control 69 (2021), p. 102848. devices”. In: International Conference on Smart Homes
[4] Eleni Boumpa and Athanasios Kakarountas. “Home and Health Telematics. Springer. 2015, pp. 181–193.
supporting smart systems for elderly people”. In: Con- [19] Yao Ge et al. “Real-Time Human Activity Recogni-
vergence of ICT and Smart Devices for Emerging Ap- tion System Exploiting Ubiquitous Wi-Fi Signals”. In:
plications. Springer, 2020, pp. 81–98. 2021 IEEE International Symposium on Antennas and
[5] Gokmen Ascioglu and Yavuz Senol. “Design of a Propagation and USNC-URSI Radio Science Meeting
wearable wireless multi-sensor monitoring system and (APS/URSI). IEEE. 2021, pp. 1033–1034.
application for activity recognition using deep learn- [20] Aryan Sharma et al. “Passive WiFi CSI sensing based
ing”. In: IEEE Access 8 (2020), pp. 169183–169195. machine learning framework for COVID-Safe occu-
[6] Nahed Jalloul. “Wearable sensors for the monitoring pancy monitoring”. In: 2021 IEEE International Con-
of movement disorders”. In: Biomedical journal 41.4 ference on Communications Workshops (ICC Work-
(2018), pp. 249–253. shops). IEEE. 2021, pp. 1–6.
[7] Ahmad Jalal, Majid Ali Khan Quaid, Kibum Kim, et al. [21] Munder Abdulatef Al-Hashem, Ali Mohammad
“A study of accelerometer and gyroscope measurements Alqudah, and Qasem Qananwah. “Performance
in physical life-log activities detection systems”. In: Evaluation of Different Machine Learning Classification
Sensors 20.22 (2020), p. 6670. Algorithms for Disease Diagnosis”. In: International
[8] David Caicedo, Ashish Pandharipande, and Frans MJ Journal of E-Health and Medical Communications
Willems. “Detection performance analysis of an ultra- (IJEHMC) 12.6 (2021), pp. 1–28.
sonic presence sensor”. In: 2013 IEEE International [22] Syed Aziz Shah and Francesco Fioranelli. “Human
Conference on Acoustics, Speech and Signal Process- activity recognition: Preliminary results for dataset
ing. IEEE. 2013, pp. 2780–2784. portability using FMCW radar”. In: 2019 International
[9] Francy Shu and Jeff Shu. “An eight-camera fall de- Radar Conference (RADAR). IEEE. 2019, pp. 1–4.
tection system using human fall pattern recognition [23] Yu Taso et al. “Subcarrier selection for efficient CSI-
via machine learning by a low-cost android box”. In: based indoor localization”. In: IOP Conference Series:
Scientific reports 11.1 (2021), pp. 1–17. Materials Science and Engineering. Vol. 383. 1. IOP
[10] William Taylor et al. “Radar sensing for activity clas- Publishing. 2018, p. 012017.
sification in elderly people exploiting micro-doppler [24] Mehran Soltani et al. “Deep learning-based channel
signatures using machine learning”. In: Sensors 21.11 estimation”. In: IEEE Communications Letters 23.4
(2021), p. 3881. (2019), pp. 652–655.
[11] William Taylor et al. “An intelligent non-invasive [25] Yan Wang et al. “E-eyes: device-free location-oriented
real-time human activity recognition system for next- activity identification using fine-grained wifi signa-
generation healthcare”. In: Sensors 20.9 (2020), p. 2653. tures”. In: Proceedings of the 20th annual international
[12] Shigeyuki Tateno et al. “Privacy-preserved fall detection conference on Mobile computing and networking. 2014,
method with three-dimensional convolutional neural pp. 617–628.
network using low-resolution infrared array sensor”. In: [26] Itaf Omar Joudeh et al. “Location Independence in
Sensors 20.20 (2020), p. 5957. Machine Learning Classification of Sitting-Down and
[13] Yu Gu et al. “MoSense: An RF-based motion detec- Standing-Up Actions using Wi-Fi Sensors”. In: 2021
tion system via off-the-shelf WiFi devices”. In: IEEE IEEE International Symposium on Medical Measure-
Internet of Things Journal 4.6 (2017), pp. 2326–2341. ments and Applications (MeMeA). IEEE. 2021, pp. 1–6.
[14] Božidara Cvetković et al. “Real-time activity monitor- [27] Robert M Keenan and Le-Nam Tran. “Fall detection
ing with a wristband and a smartphone”. In: Information using Wi-Fi signals and threshold-based activity seg-
Fusion 43 (2018), pp. 77–93. mentation”. In: 2020 IEEE 31st Annual International
[15] Aboajeila Milad Ashleibta et al. “5g-enabled contactless Symposium on Personal, Indoor and Mobile Radio
multi-user presence and activity detection for indepen- Communications. IEEE. 2020, pp. 1–6.
dent assisted living”. In: Scientific Reports 11.1 (2021), [28] Sheheryar Arshad et al. “Wi-chase: A WiFi based
pp. 1–15. human activity recognition system for sensorless envi-
[16] Ghassem Mokhtari, Qing Zhang, and Amir Fazlollahi. ronments”. In: 2017 IEEE 18th International Sympo-
“Non-wearable UWB sensor to detect falls in smart sium on A World of Wireless, Mobile and Multimedia
home environment”. In: 2017 IEEE International Con- Networks (WoWMoM). IEEE. 2017, pp. 1–6.
ference on Pervasive Computing and Communications
Workshops (PerCom Workshops). IEEE. 2017, pp. 274–
278.