2019 29th International Telecommunication Networks and Applications Conference (ITNAC)

Ambient Intelligence for Smart Home using The

Internet of Things
Hakilo Sabit Peter Han Joo Chong Jeff Kilby
Department of Electrical and Electronic Department of Electrical and Electronic Department of Electrical and Electronic
Engineering Engineering Engineering
Auckland University of Technology Auckland University of Technology Auckland University of Technology
Auckland, New Zealand Auckland, New Zealand Auckland, New Zealand
hakilo.sabit@aut.ac.nz peter.chong@aut.ac.nz jeffrey.kilby@aut.ac.nz

Abstract—This article presents an ambience intelligence interfaces such as gesture, voice, non-interruptive means. The
application for smart home systems for efficient use of true ambient environments react to the presence of people and
electricity, enhance comfort zone, independence of living and the sensors and actuators have become part of the environment
security. The system integrates smart home occupant’s [2]. Smart home offers various benefits such as energy saving,
identification, sensors-actuators deployment, a gateway Hub, control home appliances, security, data processing so that host
machine learning, and cloud computing components to realize can supervise the whole condition of house and control it even
the objectives of smart living. A Mobile phone MAC-address outside of the home [3-5]. A smart home system often consists
based occupant identification and machine learning algorithms of sensors (e.g. light, temperature, etc.) to measure the current
are proposed to address the multiple smart home occupant
home ambience conditions, computational units
problems and an instant user control versus rule-based control
(microcontrollers, GPU , FPGA, Asics) to store data, process
conflicts. Results show that machine learning algorithm could
learn the ambience preferences of multiple occupants when control algorithms, generate events, and actuators to control
trained on a large enough dataset. The proposed system can the various appliances to set ambient conditions based on the
implement ambient intelligence applications in smart home. control algorithm/decision outputs. Smart home systems
provide two types of control: user control (an aggregation of
Keywords— Ambient Intelligence, Smart home, Machine lower-level controls into more zone-specific groups) and rule-
learning, Cloud Computing, Internet of Things based control, where the system makes decisions―usually
based on a schedule [6]. A smart home should adjust the
I. INTRODUCTION ambience conditions to the liking of a user present at home
Ambient Intelligence (AmI) is the vision that technology autonomously based on the behaviour of the user that has been
will become invisible, embedded in our natural surroundings, learned until present. It should also accept instant user control.
present whenever we need it, enabled by simple and effortless However, there are often conflicting needs and challenges to
interactions, attuned to all our senses, adaptive to users and manage the system when the control moves from user control
context-sensitive, and autonomous [1]. AmI is an electronic to some form of rule-based behaviour, because the rules might
environment consisting of unobtrusive embedded devices insufficiently reflect what the resident actually want [7]. This
capable of ubiquitous computing, communication, and natural problem is further complicated when multiple occupants with
user interfaces that interacts with and respond appropriately to conflicting ambience preferences are present simultaneously.
the humans in that environment. The availability of smart and Nurgaliyev et al. [8] present a preference-based method for
interactive everyday objects opens up a whole range of new resolving conflict and evaluated a multi-user context
applications. Applications such as object tracking, navigation, awareness by a Virtual Assistant in recognizing multi-user
environmental monitoring and protection, home automation conflicts dynamically. The main goal of an ambient
can be implemented in a whole new way. Combined with the intelligence is to adjust the smart home ambience to the liking
Internet of Things, this will create a digital intelligence of the occupants. There are two broad approaches to this;
network where technology gains the ability to sense, predict smart home occupant preference scheduling and learning
and respond to our needs and is being integrated into our occupant activities and behaviour.
natural behaviours. A significant development in the Wireless
III. SYSTEM OVERVIEW
Sensor Network (WSN) based smart home is the introduction
of the concept of IoT. IoT, driven by the need for remote The proposed system for the smart home contains the
monitoring and control, ubiquitous connectivity and pervasive components: Smart home occupants identification, sensors
computing, is the next evolution of computing along with and actuators deployment, a gateway Hub, machine learning,
Artificial intelligence (AI), Virtual reality (VR) and cloud computing. Smart home Systems require major
augmented reality (AR). The motivation behind this research objectives such as sensor and controller unit, transmission of
paper is to develop an AmI application for smart homes for data (Cloud) and application for user, which are described as
various purposes such as efficient use of electricity, enhance in follow.
comfort zone, independence of living and security.
A. Smart Home Occupants Identification
II. RELATED WORK The smart home’s ability to adapt and react to residents’
The basic idea of AmI is the interconnection of presence can be limited by the lack of mechanism to identify
environment to technologies such as sensing, computing and which resident is triggering events at the current instance of
networking provided as a service to a user via natural time. With a single resident in the smart home this is a trivial

XXX-X-XXXX-XXXX-X/XX/$XX.00 ©20XX IEEE

978-1-7281-3673-8/19/$31.00 ©2019 IEEE
 2019 29th International Telecommunication Networks and Applications Conference (ITNAC)

problem, but multiple residents transform it into a serious D. Machine Learning

issue [9]. Design of a smart home should include mechanism Machine learning tracks its early roots in classical math
for multi-residents or multiple source of data identification. In and statistics. At present, machine learning uses analytic
this project we implement a mechanism of identifying models and algorithms that learn from data, finding hidden
multiple residents as well as visitors by means of mobile insights without being explicitly programmed where to look.
phone Media Access Control (MAC) addresses. As the use of Using learning algorithms based on thousands of data
and always carrying of mobile phones has become a norm samples, computers can make predictions based on these
nowadays, MAC address based smart home occupant learned experiences to a high level of accuracy. In this project
identification is a simple and elegant solution compared to a machine learning algorithm is used to learn the identified
other mechanism such as behaviometrics [9] and bio-feature smart home occupant preference of ambience setting during
[10]. Here a Linux utility known as nmap is used to scan and different times of the day. Further, the identification of
detect a list of wireless devices (MAC) associated with the multiple occupants will trigger another process whereby
home gateway router Wi Fi access point. Besides identifying multiple ambience preference are averaged to generate
multiple occupants and guests, this is used to average the appropriate.
ambience condition preferences when multiple occupants are
present at the same time. E. Cloud Computing
B. Sensors/Actuators deployment Making sense of the voluminous IoT sensor data is
computationally challenging. Hence Cloud Computing is
In the smart home system under study for this project used to store data, provide machine learning, and
multiple sensor and actuator units are deployed at various visualization and anytime, anywhere access. Cloud
locations in the smart home. The sensor units consist of a Wi- computing is a model for enabling ubiquitous, convenient,
Fi module, microcontroller, power supply unit, and on-demand network access to a shared pool of configurable
Temperature sensor, relative Humidity sensor, ambient light computing resources (e.g. networks, servers, storage,
sensor, atmospheric pressure sensor, PIR sensor, and 3-axis applications, and services) that can be rapidly provisioned
accelerometer sensors. The actuator units consist of a Wi-Fi and released with minimum management effort or service
module, microcontroller, power supply unit, and optical provider interaction. Here an IBM cloud is used as it provides
relays for triggering, light, air-conditioner, fan, piezo easy to use API to access all the analytic tools. IBM Bluemix
buzzers, and dehumidifier. These units are custom designed Cloud application will receive the sensors data from the
based on the ever popular ESP8266 Wi-Fi modules during gateway through MQTT. One can visualize the sensors data
this project (Fig. 1). The sensor units are programmed in such on boards and cards sections as well as in Node-red. In the
a way that they measure and collect the home ambience data Node-red, initial set profile of different occupants (in this
and publish to the gateway hub through the MQTT [11]. project three different profiles are made for three different
users according to their preferred temperature, humidity and
light) are stored. The Cloud application will compare to the
current sensor data, generate an action command and send
back to the gateway to adjust the appliances via the actuator
unit.
IV. NETWORK ARCHITECTURE
There exists various choice of connectivity options for the
Internet of Things products and systems including Wi-Fi,
Fig 1. Smart home sensor unit (left) and actuator unit (right).
Bluetooth, ZigBee, Z-Wave, 6LowPAN, NFC, Thread,
SigFox, LoRa, Weightless, NB-IoT, LTE-M, and 2G/3G/4G
C. Gateway hub cellular. Depending on the set of features and requirements
The sensor and actuator units generate thousands of data commonly associated with M2M and IoT such as low cost,
points reliability and receive and execute commands based on low power, compact form factors, rapid connection setup
the trigger signals received. However, they provide a limited times, availability, and massively scalable deployments, Wi-
processing capability to implement a smart system on their Fi preferred is preferred for this project. Regarding the
own. A Raspberry Pi 3B is used here to provide a place to pre- application layer protocol for data communication, a
process data locally at the edge before sending it on to the lightweight protocol Message Queuing Telemetry Transport
cloud. This will minimize the volume of data that needs to be (MQTT) is preferred. MQTT is simple, open source,
forwarded on to the cloud and improves the response time and bandwidth-efficient, uses little battery power and designed so
network transmission costs. In addition, the gateway provides that it is easy to implement it for any applications. The
additional security for the network and the data it transports. protocol uses a publish/subscribe architecture in contrast to
It protects the data leaks and devices compromise by HTTP with its request/response paradigm. Publish/Subscribe
malicious attacks with features such as tamper detection, is event-driven and enables messages to be pushed to clients.
encryption, hardware random number generation and crypto MQTT work as Transmission Control Protocol (TCP), So if
engines. The gateway hub receives the sensors data that it has client send the message it requires send CONNECT message
subscribed for via MQTT protocol. Besides this, it also to the broker and broker responds with CONNECT message.
identifies user through the MAC address of the user’s mobile Then, this connect is kept open so client can publish the data
phone. Finally, it gathers data of sensors and identified user, and subscriber receive it. Broker also request to client for
Publishes to the Cloud (IBM- Blue-mix). perform authorization and client authentication. The Network
architecture is for the smart home system is show in Fig. 2.
 2019 29th International Telecommunication Networks and Applications Conference (ITNAC)

Fig 3. Ambient light levels during autonomous run of the smart home.

VI. CONCLUSION
Fig 2. Network architecture for the smart home system.
This paper presented the challenges of multi-occupant smart
home ambience management. It also analyzed the effects of
V. RESULTS transitioning between rule-based control and manual user
This section presents three stages. Stage one is when data is ambience control. Identification of multiple residents using
being collected based on the settings for different smart home MAC addresses and machine learning algorithm for instant
occupants. In stage two, data collected in previous stage is user control versus rule-based control solutions are evaluated.
used in Training machine learning smart home system. Stage The results suggest the capability of the experimental
three and final stage is testing stage. implementation of ambient intelligence in smart homes for
Stage I. In this first stage, the ambient sensors data and the the benefits such as efficient energy usage, confort, security,
user settings are recorded and saved for training in the next and independence of living.
stage. The Ambient sensors data is also being published to
the IBM Cloud Node-red application simultaneously via the REFERENCES
MQTT at a second interval. [1] B. de Ruyter, E. Aarts, P. Markopoulos, and W. Ijsselsteijn, “Ambient
Intelligence Research in HomeLab: Engineering the User Experience,”
Stage II. Here we train and deploy a custom IBM Machine in Ambient Intelligence, Springer, Berlin, Heidelberg, 2005, pp. 49–61.
Learning (ML) model utilizing the IBM SPSS predictive
[2] A. Komarek, J. Pavlik, L. Mercl, and V. Sobeslav, “Hardware Layer of
analytics algorithm trained on data collected in the previous Ambient Intelligence Environment Implementation,” in Computational
stage. The machine learning algorithm takes, room Collective Intelligence, 2017, pp. 325–334.
temperature, ambient light, and current time as inputs and [3] G. Chong, L. Zhihao, and Y. Yifeng, “The research and implement of
generate triggers (light on/off, fan on/off, heater on/off) as smart home system based on Internet of Things,” in 2011 International
outputs. Once a custom ML model is trained, it is deployed Conference on Electronics, Communications and Control (ICECC),
2011, pp. 2944–2947.
to Watson Services and can be called at run time to extract
[4] M. M. Moazzami, G. Xing, D. Mashima, W. P. Chen, and U. Herberg,
relevant triggers. Few training iterations are done to improve “SPOT: A smartphone-based platform to tackle heterogeneity in smart-
the precision, Accuracy, and recall as performance evaluation home IoT systems,” in 2016 IEEE 3rd World Forum on Internet of
metrics. The machine learning performance measure for a Things (WF-IoT), 2016, pp. 514–519.
limited training and testing data sets are given in Table 1. [5] R. Khan, S. U. Khan, R. Zaheer, and S. Khan, “Future Internet: The
Internet of Things Architecture, Possible Applications and Key
TABLE I. MACHINE LEARNING PERFORMANCE SUMMARY Challenges,” in 2012 10th International Conference on Frontiers of
Information Technology, 2012, pp. 257–260.
‘Partitions’ Training_set Testing_set [6] S. Makonin, L. Bartram, and F. Popowich, “A Smarter Smart Home:
Correct 304000 (70.4%) 14423 (68.9%) Case Studies of Ambient Intelligence,” IEEE Pervasive Computing,
vol. 12, no. 1, pp. 58–66, Jan. 2013.
Wrong 128000 2240 [7] A. J. B. Brush, B. Lee, R. Mahajan, S. Agarwal, S. Saroiu, and C.
Total 432000 20934 Dixon, “Home Automation in the Wild: Challenges and
Opportunities,” in Proceedings of the SIGCHI Conference on Human
Factors in Computing Systems, New York, NY, USA, 2011, pp. 2115–
The fully trained smart home system is let to run for a period 2124.
of Nov 26, 2017 to Dec 10, 2017. During this period the [8] K. Nurgaliyev, D. D. Mauro, N. Khan, and J. C. Augusto, “Improved
system autonomously switched on and off the fan and light Multi-user Interaction in a Smart Environment Through a Preference-
appropriately with occasional fails to keep the ambience Based Conflict Resolution Virtual Assistant,” in 2017 International
Conference on Intelligent Environments (IE), 2017, pp. 100–107).
conditions within bounds. It can be observed that the room
[9] A. S. Crandall and D. J. Cook, “Behaviometrics for Identifying Smart
temperature only occasionally went out of the average range Home Residents,” in Human Aspects in Ambient Intelligence, Atlantis
of three different occupants, i.e. 24.5 – 26.5 degree Celsius. Press, Paris, 2013, pp. 55–71.
The three different occupants’ working hours fairly overlap [10] G. Mokhtari, Q. Zhang, G. Nourbakhsh, S. Ball, and M. Karunanithi,
except for occasional periods. The running stage shows a “BLUESOUND: A New Resident Identification Sensor #x2014;Using
Ultrasound Array and BLE Technology for Smart Home Platform,”
moderate flickering of the lights indicative of which is the IEEE Sensors Journal, vol. 17, no. 5, pp. 1503–1512, Mar. 2017.
machine learning algorithm needing more training data sets. [11] “MQ Telemetry Transport (MQTT) V3.1 Protocol Specification,” 19-
The occasional flickering points are indicated on Fig. 3. Aug2010.[Online].Available:http://www.ibm.com/developerworks/lib
rary/ws-mqtt/index.html. [Accessed: 14- Dec-2017].