This article has been accepted for inclusion in a future issue of this journal.

Content is final as presented, with the exception of pagination.

IEEE TRANSACTIONS ON AUTOMATION SCIENCE AND ENGINEERING

Point-n-Press: An Intelligent Universal Remote

Control System for Home Appliances
Kuen-Min Lee, Wei-Guang Teng, Member, IEEE, and Ting-Wei Hou, Member, IEEE

AbstractWith numerous connected devices and appliances,

the smart home is one of the representative elds of Internet of
Things (IoT). As the complexity of devices/appliances increase,
numerous buttons (sometimes dozens) are designed on the remote
controller in home spaces even if several of them are seldom used.
A user may be confused with the controller even if he or she only
intends to perform a simple operation. This confusion also leads
to a higher probability of mal-operations. In addition, conventional methods of communication between remote controllers and
connected devices, such as eXtensible Markup Language (XML)
messages, are usually bandwidth-consumptive. To address these
problems, an intelligent universal remote control system for home
appliances named Point-n-Press is proposed. Point-n-Press addresses the directionality feature, which enables easy and intuitive
control by pointing to the target device to display the target's
control interface on the screen of the remote controller. By leveraging the state dependencies of home device/appliance operations,
only functional buttons that are relevant to the current context
are utilized. Two real prototypes are implemented to verify the
feasibility of the proposed scheme. The evaluation results show
that Point-n-Press is a useful and suitable control scheme for
IoT-based smart homes.
Note to PractitionersThis work presents an intelligent universal remote control system for home appliances. A smart phone
equipped with infrared (IR) capabilities is easier to realize functions of the Point-n-Press remote controller (PPRC). Otherwise,
some IR universal serial bus (USB) dongles that can provide
support for Android operation system are considered to be used.
In addition, several open source universal plug and play (UPnP)
libraries are useful to help us to greatly reduce the development
time. Finally, for implementation of the Point-n-Press control box
(PPCB), the design of the small and narrow hole is a trick for the
IR mechanism. According to the design of this mechanism, only
one device can be simultaneously pointed and controlled by the
proposed control system. Therefore, the proposed control system
always displays one control user interface of a single device. These
details help us to achieve the purposes of the intelligent universal
remote control system for home appliances.
Index TermsIntuitive control, IoT, less bandwidth consumption, smart home, user-friendly UI.
Manuscript received November 15, 2015; revised January 29, 2016; accepted
February 18, 2016. This paper was recommended for publication by Associate
Editor W. Shen and Editor M. Zhou upon evaluation of the reviewers comments.
K.-M. Lee is with the Department of Engineering Science, National Cheng
Kung University, Tainan 701, Taiwan, and the Industrial Technology Research
Institute, Tainan 709, Taiwan (e-mail: allen_lee@itri.org.tw).
W.-G. Teng and T.-W. Hou are with the Department of Engineering Science,
National Cheng Kung University, Tainan 701, Taiwan (e-mail: wgteng@mail.
ncku.edu.tw; hou@nc.es.ncku.edu.tw).
Color versions of one or more of the gures in this paper are available online
at http://ieeexplore.ieee.org.
Digital Object Identier 10.1109/TASE.2016.2539381

I. INTRODUCTION

NTERNET of Things (IoT) [1] is a technology that connects all things and the Internet in smart spaces. By implementations of intelligence with sensing devices, IoT has been
widely applied to different elds, such as smart homes [2], [3].
The application elds in smart homes [4] incorporate smartness into home areas for comfort, safety, security, healthcare,
and energy conservation [5], [6]. The need for comfort and a
convenient life are especially important in smart homes. Thus,
home automation is one of the most essential and critical components for the IoT-based smart home technology. Home automation systems are used to control home devices or appliances
in smart homes and provide automatic remote control inside or
outside homes [7]. Nevertheless, although remote control provides convenience and ease of use, some major problems require
consideration and improvement, such as how to provide an intuitive and user-friendly remote control scheme in IoT-based
smart homes [8].
The goal of this paper is to develop an intelligent universal remote control system for home appliances called Point-n-Press.
Point-n-Press automatically detects the device (or appliance)
when a user points the controller at it. Also, a user interface
(UI) for controlling this device is immediately displayed on the
screen of the controller. Only the functional buttons that are
relevant to the current control context appear on the UI. The
UI provides intuitive operations and user-friendly interfaces,
which enable users to simply enable and control the target device among the increasingly complex functionalities of home
devices in a shared space for IoT-based smart homes. Note that
a nite state machine (FSM) is used to model all operational
states of a device and dependencies among these states. Multiple bit-string formatted control codes (modeled as bit-vector
forms), which represent the control operations, are also applied
in the proposed scheme to decrease the bandwidth consumption.
Two real prototypes are implemented in smart homes to demonstrate the feasibility of the proposed scheme. Fig. 1 shows one
of the prototypes for controlling a fan. In this control prototype,
Point-n-Press is implemented in a mobile phone; a fan can be directly controlled by pointing to an external control box near the
fan. Note that two state dependencies are included in the control
process of the fan. First, the fan can only be started by pressing
the Power button when it is powered off, whereas pressing
other buttons is useless. Second, the wind speed button has
no effect on the fan when the fan is in sleep or natural mode,
because the wind speed is automatically adjusted. Thus, by considering the state dependencies, only functional buttons that are

1545-5955 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.
2

IEEE TRANSACTIONS ON AUTOMATION SCIENCE AND ENGINEERING

Fig. 1. Prototype of the proposed control system for control of a fan.

relevant to the current context are displayed on the screen of the

controller.
The remainder of this paper is organized as follows. Related
studies are reviewed in Section II. In Section III, the detailed
design of the proposed technique, i.e., Point-n-Press, are described. Two real prototypes and the experimental results are
implemented and discussed in Section IV, and the conclusions
are presented in Section V.
II. RELATED STUDIES
Currently, the majority of devices and appliances in smart
homes are equipped with a remote controller, which includes
a number of buttons and wireless transceivers [9][11]. This
setup provides higher operational complexity around the space
with numerous devices or appliances. Thus, the idea of the universal remote controller (URC) is introduced to integrate multiple functions of home devices or appliances into one single remote controller [8], [12][14]. Nevertheless, various functions
and buttons of a URC results in more complicated operations,
the problems of intuition and user-friendliness remain.
Numerous solutions are proposed to develop URCs with a
liquid crystal display (LCD) screen, networking capability, and
several techniques [15][17]. Typical techniques for these types
of URCs include universal plug and play (UPnP)1 and universal
remote console specification of the alternate interface access
protocol (AIAP-URC).2 Built in with these techniques, the device or appliance can be automatically detected via a network
[18][21], and a UI is dynamically generated from descriptions
and properties of the device or appliance [22][24].
Because these techniques facilitate control setting and increase the ease of control, intuition and user-friendliness can
be improved. Although the UPnP technique discovers devices
1[Online].

Available: http://www.upnp.org/

2[Online].

Available: http://zing.ncsl.nist.gov/incits/v2/

around a specic space [18][21], multiple instances of the same

type of devices or appliances are frequently located in this specic area, e.g., lights/lamps. Therefore, these lights/lamps are
displayed on the screen of the controller; the mapping and correspondence between the device's UI and the actual device may
confuse users. The kind of control systems is not sufciently intuitive and user-friendly to users.
Several efforts thus have been made to achieve intuitive control. For instance, an intuitive and user-friendly telehealth service for the elderly and long-term patients has been discussed in
previous studies [25]. Although a user-friendly and zero-conguration design is utilized in this service, both the directionality
of infrared (IR) and the state dependencies of the device operations have not been considered. In addition, a brain machine
interface system is proposed to achieve automation for solving
the problem of mental fatigue [26]. Although user preferences
and error perception feedback are considered, the machine is too
expensive for implementation in smart homes, and the state dependencies of the device operations is not monitored or considered. Furthermore, the directionality of IR has been considered
in previous studies [27]. The control system requires a specic
and customized remote controller with a special striped pattern.
Nevertheless, the usage of this system is limited to televisions.
Recently, some interaction approaches are proposed to control appliances in smart homes, including the use of voice commands [28] and gure gestures [29], [30]. Nevertheless, the
voice control may not work well when the environment is noisy.
Additionally, with gesture sensing techniques, users may have
to guess or learn the control gestures of the appliances that are
probably not natural and intuitive. Besides, an interactive television with a beyond-screen interface is introduced to provide intuitive interactions for controlling TVs [31]. The system focuses
on interacting with TVs and does not consider the directionality
of IR and the state dependencies of the device operations. All
icons for the control features, which are displayed on the screen
of the controller, may confuse users.
Many problems regarding intelligent and user-friendly UIs
have been proposed in numerous elds. For instance, a three-dimensional (3-D) view interface is introduced in a previous study
[32]. With the 3-D user-friendly interface, users can easily control the target device. However, the features of the directionality
of the IR characteristic and the state dependencies of the device
operations are not considered. Similar to previous studies, an
easy-to-control interface is presented with a haptic interface for
customer electronic devices [33], in which a predened menu
needs to be prepared and the UIs need to be arranged prior to
set up of the control system. Thus, the control system lacks
exibility.
On the other hand, with unnecessary, nonfunctional, and even
useless functions or buttons appear on the screen of the controller, a higher probability of incorrect operations may be generated when controlling appliances. Hence, the state dependencies of the device operations are introduced so that inactive buttons (that are unrelated to the current state) are not displayed.
For example, when an air conditioner is powered off, the only
necessary (or functional) button is the Power-on button. When
the air conditioner operates in dehumidication mode, the buttons Temperature Set and Air Volume are inactive.

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.
LEE et al.: POINT-N-PRESS: AN INTELLIGENT UNIVERSAL REMOTE CONTROL SYSTEM FOR HOME APPLIANCES

Fig. 2. System architecture of the proposed control system.

It is also noticed that complicated formats and functions used

for controlling home appliances may produce large transmission
messages, e.g., eXtensible Markup Language (XML) messages
[18], [34][36]. For these kinds of XML-based control systems,
the integration with LonWorks and UPnP devices by an open
service gateway initiative (OSGi) gateway and a universal
remote console are constructed to increase the convenience of
the smart home control system [18]. This integration provides
a user-friendly UI but does not provide an intuitive control
scheme. From the experimental results, this scheme consumes
more bandwidth than the proposed Point-n-Press because it
works on XML-based messages. Furthermore, a user-friendly
and interactive UI that employs a radio frequency for consumer
electronics (RF4CE) approach is presented to decrease the
bandwidth consumption [14]. Although the RF4CE approach
automatically establishes the communication protocol between
a device and a remote controller, this approach may consume
more bandwidth than the proposed Point-n-Press based on
the experimental results. Moreover, the intuitive control with
IR directionality and the state dependencies of the device
operations are not utilized.
III. PROPOSED SCHEME OF POINT-N-PRESS
Here, the comprehensive system architecture and its primary
components are discussed in Section III-A. Section III-B describes the detailed workows of the proposed Point-n-Press.
The key features of the device control prole (DCP) are presented in Section III-C.
A. System Architecture
The architecture of Point-n-Press consists of two parts: 1)
the Point-n-Press remote controller (PPRC) and 2) a number of
target devices, which embed in the Point-n-Press control box
(PPCB) for interacting with the PPRC, as shown in Fig. 2. The
detailed architectures of the PPRC and target devices are illustrated in Fig. 3.
The functions of the components in the PPRC are described
as follows.
Interface Generator creates a UI according to the properties and descriptions of the target device and its current
state.
Device Prole Registry stores the information of the current target device, such as its current state and dependency
between each state.

Fig. 3. Software stacks of (a) the PPRC and (b) the target device.

URC Control is the main component of the PPRC, which

is responsible for receiving DCPs from target devices. The
URC Control subsequently analyzes the DCP and the current state of the target device to perform additional control
operations.
UPnP Control Point is a set of network protocols that enables networked devices to seamlessly discover each other
in the network and establishes functional network services
for communication. The UPnP Control Point is applied to
transmit the control commands to target devices.
Android platform is the operating platform of the PPRC,
which is Linux-based.
Underlying communication interfaces are used between
the PPRC and the target devices, including IR, ZigBee,
Wi-Fi, and Bluetooth. A gravity sensor (G sensor) is utilized to detect the movement of the PPRC. An IR transceiver is used to detect and identify a certain target device
as the PPRC is pointed to a specic target device.
Each target device, which is controlled by the PPRC, is composed of two parts: (1) the PPCB and (2) the home appliance.
Note that even a non-UPnP conventional and legacy appliance
with IR capability can be adapted and controlled with the proposed control system via the PPCB mechanism. The functions
of the components in the PPCB are detailed as follows.

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.
4

IEEE TRANSACTIONS ON AUTOMATION SCIENCE AND ENGINEERING

Device Control Prole is a le that species the properties

and descriptions of a target device. The DCP is described
in Section III-C.
Target Control generates DCPs and transfers the corresponding DCPs to the URC Control of the PPRC.
UPnP Device communicates with the UPnP Control Point
of the PPRC and executes the received UPnP control commands.
Linux platform is applied as the operating platform of the
PPCB.
Underlying information communication interfaces use
an IR transceiver to receive a detection signal from the
PPRC when the PPRC is pointing to the IR transceiver of
the PPCB. ZigBee, Wi-Fi, and Bluetooth can also be utilized as the communication interfaces between the PPRC
and the target devices.

Fig. 4. Conventional XML example of a DCP message for a fan.

B. Primary Workflows
With the G sensor in the PPRC, the system can detect that the
PPRC has been shaken (i.e., preparing to perform some control operations) and may be used for controlling appliances.
The PPRC then sends a Be Ready signal to the PPCB in
the vicinity via a Bluetooth or Wi-Fi wireless network. When
a PPCB receives the Be Ready signal, the PPCB initializes
and enables its internal IR receiver. Once the PPRC is xed on
and pointed to a specic PPCB, the IR receiver of the PPCB
subsequently receives the signal that was transmitted from the
IR transmitter of the PPRC using directionality of the IR characteristic. During this time, the Target Control component of
the PPCB simultaneously transmits the DCP to the URC Control component of the PPRC via a Bluetooth or Wi-Fi wireless
network. After registering the received DCP to the Device Prole Registry component, the URC Control component generates
a control UI via the Interface Generator component according
to the DCP and state dependencies of the FSM. At this point,
the PPRC can issue control commands using the corresponding
control UI via the UPnP Control Point component. The PPRC
then transmits control commands to the UPnP Device component of the PPCB to control the specic target device. Last, the
PPCB transmits the received control commands from the PPRC
to the appliance to generate the corresponding control operations via an IR transmitter. The home appliance performs the
corresponding operations issued by the PPRC.
C. Device Control Profile
A remote controller controls appliances by the transmission
of messages. Fig. 4 shows an example of an XML message [18]
(i.e., the DCP) for a fan with three buttons. Although the XML
message increases the convenience of discovering and controlling appliances, the complicated control formats and sizes consume signicant bandwidth, produce heavy loads and create
bottlenecks in the network transmission.
To solve this problem, the proposed DCP incorporates the
typical feature that each home appliance is sequentially operated (e.g., the power is changed from off to on and then off
again after pressing the power button). Besides, the bit-string
encoding scheme, i.e., modeled as a bit-vector form, is used to

Fig. 5. Example of a DCP message for a fan.

Fig. 6. Finite states of a fan's control functions.

reduce the bandwidth consumption. Thus, the design of the DCP

includes: 1) a finite state machine that represents the dependencies of every operational state of the appliance and 2) multiple bit-string formatted control codes that represent the control operations. Fig. 5 shows a DCP message for our previous
fan example.
Fig. 6 illustrates an example of using a nite state diagram
to represent the state transitions for controlling a fan with three
buttons. By transforming the control ow of an appliance into
a FSM, the UI only needs to show the states that are relevant to
the current state (i.e., candidates of subsequent states). Note that
two state dependencies are included in the control process of this
fan. First, when the fan is powered off, it can only be started by
pressing the Power button, whereas pressing other buttons is
useless. Second, when the fan is in sleep or natural mode, the

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.
LEE et al.: POINT-N-PRESS: AN INTELLIGENT UNIVERSAL REMOTE CONTROL SYSTEM FOR HOME APPLIANCES

TABLE I
REPRESENTATION OF FUNCTIONS AND STATES IN BIT-STRING FORMATS FOR CONTROLLING A FAN

Fig. 7. Function buttons and status parameters of a fan as represented in bitvector forms.

wind speed is automatically adjusted, which indicates that the

wind speed button exerts no effect on the fan.
The steps of constructing an FSM and thus the DCP for the
fan example are then illustrated. First, all buttons and operational states of the fan are listed as shown in Fig. 6. Second, all
current states, control buttons, and the corresponding next states
are summarized as shown in Table I. Then, all of the operational
states are using the design of bit-vector forms to model the function buttons and status parameters, as shown in Fig. 7. Specically, a vector with 3 b is used to model the function buttons of
a fan, and the status parameters are encoded into another vector
with 8 b. Note that status parameters may be independently set
to determine an operational state, e.g., the operational state 0
46 is obtained by setting the status parameters of power on,
normal mode, and gentle wind speed to 1 and the others to 0
simultaneously. This bit-string encoding scheme is thus a more
exible design to include all possible operational states. In addition, only two rows (representing function buttons and status
parameters) have to be declared in the DCP header that saves
the le size. After the encoding process, the transition state of
the fan's operational states can be transferred to a FSM and represented in bit-string formats, as shown in Table I and Fig. 8.
In this FSM, each trigger condition between two states (i.e.,
a transition in the FSM) is encoded in the bit-string format
and treated as a control operation. As shown in Fig. 8, once a
user presses the Power button on the screen of the PPRC,
the 0 01 control signal is transmitted and the current state
becomes the 0 46 state (power on, normal mode, moderate
wind speed). If the user presses the Mode button, the 0
02 control signal is transmitted and the current state becomes the 0 0A state (power on, sleep mode, no display
for wind speed operations). Moreover, Table I lists the functions and states of the fan in bit-string formats. For example,
the hexadecimal string 0 01 denotes the power off state,
and the only available operation is power on (i.e., no display
is available for the buttons of Mode selection and Wind

Fig. 8. Transition state diagram of a fan's operational states as represented in

bit-string formats.

speed adjustment); the hexadecimal string 0 0A indicates

that the power is on and that the fan is in sleep mode with
uncontrollable speed. Therefore, with the design of the FSM
and bit-vector models, i.e., represented in multiple bit-string
formatted control codes, the representation of DCP messages
and thus the bandwidth consumption is signicantly improved.
IV. SYSTEM IMPLEMENTATION
To verify the feasibility of the proposed Point-n-Press control system, two real prototypes are implemented, including a
fan and a lighting control system. Implementation details are
described in Section IV-A. Section IV-B shows the operational
UIs of two real prototypes. Furthermore, the bandwidth consumption comparisons among the proposed Point-n-Press, the
RF4CE-based [14], and the conventional XML [18] approaches
are discussed in Section IV-C.
A. Demonstration With Two Real Prototypes
Two control prototypes have been launched in IoT-based
smart homes. The rst prototype is a fan, as shown in Fig. 1.
The PPRC is implemented on the platform of a commercial
smart phone, which is equipped with 1.2 GHz, 1 Gbytes random
access memory (RAM), 16 Gbytes read only memory (ROM),
a Bluetooth communication, a Wi-Fi communication, a G
sensor, and a micro-universal serial bus (micro-USB) interface.

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.
6

IEEE TRANSACTIONS ON AUTOMATION SCIENCE AND ENGINEERING

Fig. 10. System architecture of the proposed lighting control system.

Fig. 9. Smart-home demo room with the proposed lighting control system.

In addition, all software components (e.g., UPnP Control Point

and URC Control) are developed based on the Android 4.1
version. The IR capability of the PPRC is provided by a USB
IR transceiver dongle through a physical connection cable that
is connected to the micro-USB interface.
The PPCB is employed with an ARM9-based development
board, which is equipped with 200 MHz, 1 Mbytes Flash,
64 Mbytes Not AND (NAND) Flash, 64 Mbytes (synchronous
dynamic random access memory (SDRAM), and 22 general
purpose input/output (GPIO) interfaces. The internal software
components are developed based on the Linux 2.4.18 version.
The IR mechanism of the PPCB is designed with a narrow
coverage range to prevent unexpected control operations of
receiving multiple IR signals. The small and narrow hole is
constructed and treated as the IR mechanism in the PPCB.
Based on the design of this mechanism, the proposed control
system can guarantee that only one device can be simultaneously pointed and controlled. Consequently, the proposed
system always displays one control UI of a single appliance on
the screen of the PPRC.
The second example of the demonstrated prototype is a
lighting control system. As shown in Fig. 9, the lighting control
system is located in a smart home demo room, in which eight
controllable lights and four control scenarios can be set. The
designed control scenarios in the proposed lighting system
include night, reading, cinema, and sleeping scenarios.
The system architecture of the proposed lighting control
system is shown in Fig. 10. The communications between the
PPRC and the PPCB are IR, Bluetooth, and Wi-Fi. In addition,
IR transmission is applied between the PPCB and the control
gateway. Note that the control gateway is equipped with 2.3
GHz, 2 Gbytes SDRAM, two USB connectors, and one USB IR
receiver. The internal data transformation between the control
gateway and the power line controller uses an USB cable.
The plugs from no. 1 to no. 8 are connected to a power line
controller via a power line network. The representation of the
functions and states in the bit-string formatted control codes of
Point-n-Press can be described for 36 types of control states.

Fig. 11. System operations and UI demonstration for controlling a fan. (a)
Search for appliances (not yet pointing to the PPCB of the fan and nothing is
displayed on the screen). (b) Pointing to the PPCB of the fan (at this point, the
state of the fan is powered off). (c) Pointing to the PPCB of the fan and pressing
the Power button (at this point, the state of the fan is powered on in normal
mode and a moderate wind speed). (d) Pointing to the PPCB of the fan and
pressing the Mode button (at this point, the state of the fan is powered on in
natural mode with no wind speed control available).

B. System Operations and UIs Demonstration

The operational UIs for controlling a fan are shown in Fig. 11.
A smart phone, which is treated as the PPRC, is picked up,
and the Be Ready signal is sent to the PPCB, as shown in
Fig. 11(a). No other control icons are displayed on the screen
of the PPRC at this time. Next, the PPRC receives the DCP
from the PPCB via a Bluetooth or Wi-Fi wireless network after
the PPRC is pointed to the PPCB of the fan. The icon and the
current state of the fan are displayed on the control screen (at
this point, only the Power icon is displayed, which indicates

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.
LEE et al.: POINT-N-PRESS: AN INTELLIGENT UNIVERSAL REMOTE CONTROL SYSTEM FOR HOME APPLIANCES

TABLE II
COMPARISON OF BANDWIDTH CONSUMPTION FOR CONTROLLING A FAN

TABLE III
COMPARISON OF BANDWIDTH CONSUMPTION FOR CONTROLLING A LIGHTING
CONTROL SYSTEM

Fig. 12. System operations and UI demonstration for controlling a lighting

control system. (a) Search for appliances (not yet pointing to the PPCB of the
lighting control system and nothing is displayed on the screen). (b) Pointing to
the PPCB of the lighting control system (at this point, the states of all lights
are powered off). (c) Pointing to the PPCB of the lighting control system and
pressing the Mode button (at this point, the state of the lighting control system
is powered on in scenario control mode with no single lighting control available). (d) Pointing to the PPCB of the lighting control system and pressing the
Mode button again (at this point, the state of the lighting control system is
changed to single lighting control mode and the scenario control mode is not
available).

that the fan is powered off, and other operations of the fan are
not allowed), as shown in Fig. 11(b). As the Power button
is pressed, the state displayed on the screen of the PPRC is
switched to power on. The fan is operated in normal mode with a
moderate wind speed, as shown in Fig. 11(c). After the Mode
button is pressed, the fan is operated in natural mode without
wind speed control functions (i.e., the Speed icon and related
options are hidden), as shown in Fig. 11(d).
Fig. 12 shows the demonstrated operations and control UIs of
the proposed lighting control system. As shown in Fig. 12(a), a
smart phone, which is also treated as the PPRC, is picked up and
the Be Ready signal is issued to the PPCB. After the PPRC is
pointed to the PPCB of the lighting control system, the PPRC
receives the DCP from the PPCB via a wireless network. The
icon and its current state of the lighting control system are subsequently displayed on the control screen (at this point, only
the Mode icon and the power off information are displayed,
which indicates that the lighting control system is powered off
and only the Mode icon is available to enable the lighting control system), as shown in Fig. 12(b). After the Mode button
is pressed by a user, the state displayed on the screen of the
PPRC is changed to the power on state, as shown in Fig. 12(c).
The lighting control system is operated in scenario mode (i.e.,

its default mode after power on) without single lighting control
functions (i.e., the Single icon and related options are hidden).
After the Mode button is pressed again, the state is operated in
single lighting control mode without scenario control functions
(i.e., the Scenario icon and related options cannot be operated), as shown in Fig. 12(d).
C. Bandwidth Consumption Comparisons
Here, two previous remote control approachesa RF4CEbased approach [14] and an XML-based approach [18]are applied for the comparison with the proposed Point-n-Press. Controlling the same fan with IR capability and three buttons, as
shown in Fig. 1, the comparisons of bandwidth consumption
are summarized in Table II. The comparison of the bandwidth
consumption for controlling a lighting control system among
the proposed Point-n-Press, the RF4CE-based, and the conventional XML approaches is also shown in Table III.
As shown in Fig. 4, the DCP le size for a fan in the previous
XML approach is 885 bytes. For the RF4CE-based approach,
which is based on the denition of the consumer electronics
remote control (CERC) frame, the consumer electronics control
(CEC) frame, and the state transmission functions, the size of
the DCP is 288 bytes for a fan with six state machines. Note
that each state machine uses 48 bytes, which refers to the design
of a previous study [14]. Conversely, the le size of the DCP
of the proposed Point-n-Press is only 242 bytes, as illustrated
in Fig. 5. Thus, the bandwidth consumption of the proposed
Point-n-Press while transmitting the DCP for the rst time is
more efcient compared with the bandwidth consumption in the
previous two studies. When a user begins to control the fan, the
command set of the previous XML-based study uses numeric
values as transmission messages, such as 01. Therefore, the
size of the command set of the conventional XML approach is

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.
8

IEEE TRANSACTIONS ON AUTOMATION SCIENCE AND ENGINEERING

Fig. 13. Examples of conventional XML state return messages for a fan. (a)
Minimum state return message. (b) Maximum state return message.

three bytes. However, the size of the state return message ranges
from a minimum of 213 bytes to a maximum of 378 bytes, as
shown in Fig. 13.
For the RF4CE-based approach, the size of the command set
is always two bytes using an efcient transmit protocol and a
FSM control scheme. The size of the state return message ranges
from a minimum of 182 bytes to a maximum of 332 bytes because the CEC frame format contains a maximum of 16 data
blocks, in which the size of each data block is ten bytes. Note
that only 1 data block is included in the minimum state return
message and the maximum state return message consists of 15
data blocks.
For Point-n-Press, with the designs of the FSM and bit-string
formatted control codes, both the sizes of the command set and
the state return message are always two bytes for all operations
of the fan. The bandwidth consumption of Point-n-Press is signicantly less than the sizes of the data transmission in the previous two studies in Table II. The bandwidth consumption of
Point-n-Press is superior to the bandwidth consumption in previous studies to obtain the return state of the appliances due to
the use of the hexadecimal string (e.g., 0 01) is used.
For the lighting control system, the comparisons between
Point-n-Press and the conventional approaches are shown in
Table III. For the proposed Point-n-Press, the le size of the
DCP for controlling a lighting control system is only 688 bytes
with 36 state machines. Conversely, both the sizes of the command set and the state return message are always two bytes
for all operations of the fan with the designs of the FSM and
bit-string formatted control codes. The le size of the DCP is
2,930 bytes with 36 types of control states for the conventional
XML approach. In addition, the size of the state return message
for the conventional XML approach ranges from a minimum
of 598 bytes to a maximum of 1,560 bytes, and the size of the
command set is three bytes due to the numeric values. Both the
sizes of the DCP and the state return message are larger than the
proposed Point-n-Press.
For the RF4CE-based approach, the le size of the DCP is
816 bytes for a lighting control system with 17 state machines
based on the denition of the CERC, the CEC frame, and the
state transmission functions. The size of the command set is always two bytes when an efcient transmission protocol and a
FSM control scheme are employed. The size of the state return

Fig. 14. Accumulated bandwidth consumption of the proposed Point-n-Press

compared with the accumulated bandwidth consumption for conventional approaches for controlling a fan.

message ranges from a minimum of 182 bytes to a maximum of

332 bytes because the CEC frame format contains a maximum
of 16 data blocks. Compared with the proposed Point-n-Press,
the size of the command set is equivalent to Point-n-Press because the FSM control scheme is utilized. Nevertheless, both the
sizes of the DCP and the state return message exceed the sizes
of the proposed Point-n-Press.
Another simulation is constructed in this study to assess the
impacts of bandwidth consumption in smart home environments
when many users simultaneously control devices/appliances.
This simulation assumes that three users simultaneously control and operate (e.g., set or get state) the home appliances (i.e.,
the previously mentioned experimental fan with three buttons)
100 times with the same network bandwidth. For example, the
bandwidth is provided by the same wireless access point (WAP).
In addition, each operation has been simulated more than 1,000
times. Note that users' behaviors can be modeled as follows: The
rst 10% of users' behaviors (i.e., the rst 10 times) comprises
the command set with a probability of 80%, and the subsequent
operations of the command set exhibit a 50% probability. This
nding indicates that the following 90 times of state return and
command set operations are randomly implemented.
As shown in Fig. 14, the conventional XML approach consumes excessive bandwidth as the operations increase (i.e.,
multiple users simultaneously control numerous devices/appliances) due to the complex le structure. The RF4CE-based
approach consumes less bandwidth than the conventional
XML approach because a FSM mechanism is adopted in the
transmission strategy. Although an efcient transmission protocol is utilized in the RF4CE-based approach, the bandwidth

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.
LEE et al.: POINT-N-PRESS: AN INTELLIGENT UNIVERSAL REMOTE CONTROL SYSTEM FOR HOME APPLIANCES

conventional XML approach and the RF4CE-based approach

are 94.31% and 93.70%, respectively.
V. CONCLUSION

Fig. 15. Accumulated bandwidth consumption of the proposed Point-n-Press

compared with the accumulated bandwidth consumption for conventional approaches for controlling a lighting control system.

consumption remains signicantly larger than the bandwidth

consumption of the Point-n-Press due to the bit-string formatted control codes. As a result, the proposed Point-n-Press
is superior to the previous two studies in terms of bandwidth
consumption due to the application of the FSM mechanism
and bit-string formatted control codes. Because the proposed
remote control scheme employs the hexadecimal string as the
transmission message, the bandwidth consumption is always
substantially less than the previous two approaches. In addition,
the accumulated bandwidth consumption of Point-n-Press is
also substantially more stable and slow compared with the
conventional two approaches. The nal results indicate that
the accumulated bandwidth consumption of the proposed
Point-n-Press, the conventional XML, and the RF4CE-based
approaches are 1.32, 40.674, and 34.38 Kbytes, respectively.
Therefore, the reduced percentage of bandwidth consumption of Point-n-Press compared with the conventional XML
approach and the RF4CE-based approach are 96.75% and
96.16%, respectively.
Fig. 15 shows the accumulated bandwidth consumption of
the proposed Point-n-Press compared with the accumulated
bandwidth consumption for the conventional two approaches
for controlling the lighting control system under the same
assumptions and conditions as described in Fig. 14. The accumulation of bandwidth consumption is much more stable
and slow compared with the previous two studies. After
100 operations, the accumulated bandwidth consumptions
of the proposed Point-n-Press, the conventional XML, and
the RF4CE-based approaches are 2.658, 46.782, and 42.243
Kbytes, respectively. As a result, the reduce percentage of
bandwidth consumption of Point-n-Press compared with the

An intuitive control system with a set of user-friendly

operations, called Point-n-Press, is proposed for controlling
connected devices/appliances in IoT-based smart homes. The
proposed scheme leverages the directionality characteristic
of IR to enable easy and intuitive control of devices (i.e.,
controlling an appliance in smart homes by pointing to it). A
user-friendly UI is designed by considering the state dependencies between each control operation. This design disables
buttons that are irrelevant to the current context to prevent
users from performing mal-operations. With the demonstration
of two real prototypes with controlling appliances in smart
homes, the feasibility of an intelligent universal remote control
system for home appliances with intuitive and user-friendly
features is veried. In the designs of the FSM and bit-string
formatted control codes, the communication between remote
controllers and appliances requires less bandwidth consumption. Compared with previous studies that employ conventional
XML and the RF4CE-based approaches, the proposed control
system signicantly reduces the bandwidth consumption while
multiple users simultaneously control numerous appliances.
Consequently, the proposed Point-n-Press control system not
only enhances the features of intuition and user-friendliness
but also establishes a less bandwidth-consumptive control
system. Nevertheless, the implementation of the proposed
control system is currently limited to IR sensors. Moreover,
state dependencies of devices/appliances must be manually
identied. Therefore, to control devices/appliances with a more
precise pointing mechanism, and to support an autodiscovery
mechanism of state dependencies are two possible directions
for future research.
ACKNOWLEDGMENT
The authors would like to thank P.-Y. Chen and Y.-T. Lin of
the Industrial Technology Research Institute (ITRI) for implementing related prototypes.
REFERENCES
[1] F. Wang, L. Hu, J. Zhou, and K. Zhao, A survey from the perspective
of evolutionary process in the Internet of Things, Int. J. Distrib. Sens.
Netw., vol. 2015, no. 9, pp. 19, Jan. 2015.
[2] M. Javier et al., From the internet of things to the internet of people,
IEEE Int. Comput., vol. 19, no. 2, pp. 4047, Mar. 2015.
[3] L. Zhang, L. Zheng, H. Yang, and T. J. Pan, Research of the key technologies of the smart home based on IOT, Appl. Mech. Mater., vol.
713715, pp. 23042307, Jan. 2015.
[4] M. R. Alam, M. B. I. Reaz, and M. A. M. Ali, A review of smart
homespast, present, and future, IEEE Trans. Syst., Man, Cyber., vol.
42, no. 6, pp. 11901203, Nov. 2012.
[5] K. Ben, P. Sachin, A. Pieter, and G. Ken, A survey of research on
cloud robotics and automation, IEEE Trans. Autom. Sci. Eng., vol.
12, no. 2, pp. 398409, Apr. 2015.
[6] C.-H. Lu et al., Energy-responsive aggregate context for energy
saving in a multi-resident environment, IEEE Trans. Autom. Sci.
Eng., vol. 11, no. 3, pp. 715729, Jul. 2014.
[7] H. C. Tadimeti and M. Pulipati, Overview of automation systems and
home appliances control using PC and microcontroller, Int. J. Sci.
Res., vol. 2, no. 4, pp. 127131, Apr. 2013.
[8] T. Kim, H. Lee, and Y. Chung, Advanced universal remote controller
for home automation and security, IEEE Trans. Consumer Electron.,
vol. 56, no. 4, pp. 25372542, Nov. 2010.

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.
10

IEEE TRANSACTIONS ON AUTOMATION SCIENCE AND ENGINEERING

[9] J. Han, H. Lee, and K.-R. Park, Remote-controllable and energy-saving room architecture based on ZigBee communication,
IEEE Trans. Consumer Electron., vol. 55, no. 1, pp. 264268, Feb.
2009.
[10] K. Gill, S.-H. Yang, F. Yao, and X. Lu, A ZigBee-based home automation system, IEEE Trans. Consumer Electron., vol. 55, no. 2, pp.
422430, May 2009.
[11] H. Erdem and A. Uner, A multi-channel remote controller for home
and ofce appliances, IEEE Trans. Consumer Electron., vol. 55, no.
4, pp. 21842189, Nov. 2009.
[12] M. Tezuka, Y. Honda, and M. Kato, Development of bi-directional
remote controller protocol and systems for domestic appliances, IEEE
Trans. Consumer Electron., vol. 46, no. 3, pp. 802811, Aug. 2000.
[13] P. Palanisamy, A universal device controller (UDC), Int. J. Comput.
Appl., vol. 82, no. 10, pp. 2429, Nov. 2013.
[14] C.-F. Lai, M. Chen, M. Qiu, A. V. Vasilakos, and J. H. Park, A
RF4CE-based remote controller with interactive graphical user interface applied to home automation system, ACM Trans. Embed.
Comput. Syst., vol. 12, no. 2, pp. 119, Feb. 2013.
[15] W.-P. Sung, C.-M. Kao, and R. Chen, Research and design of touchscreen domestic universal remote controller with self-learning function, Appl. Mech. Mater., vol. 599611, pp. 10281031, Aug. 2014.
[16] Y.-Y. FanJiang and T.-F. Chueh, Adaptive user interface for universal remote controller, Int. J. Comput. Sci. Eng., vol. 10, no. 12,
pp. 9097, Feb. 2015.
[17] M. S. Alamgir, S. M. A. Z. Chowdhury, and K. T. Ahmmed, Development of a PoE switch based remote control system for controlling
home and ofce appliances with data communication facility, Int. J.
Control Autom., vol. 8, no. 1, pp. 103110, Jan. 2015.
[18] Y.-H. Sheng, W.-W. Lin, C.-S. Tu, and K.-M. Lee, Integrate LonWorks and UPnP devices by OSGi gateway and universal remote console, in Proc. 8th Int. Conf. Adv. Commun. Technol., Phoenix Park,
Korea, Feb. 2006, pp. 18481850.
[19] G.-J. Horng, C.-K. Lin, C.-H. Tseng, C.-H. Wang, and J.-F. Yang, An
agent-based smart home system and its service-scheduling mechanism:
Design and implementation, Wireless Pers. Commun., vol. 78, no. 1,
pp. 521542, Sep. 2014.
[20] S. Pradeep and V. Padmajothi, Brain controlled smart home network
based on cognitive state, Int. J. Emerg. Tech. Comput. Sci. Electron.
(IJETCSE), vol. 13, no. 2, pp. 5962, Mar. 2015.
[21] P. Kabilan, H. N. Askar, P. N. Anand, S. Manimaran, and G.
Venkatesh, Facebook based home appliances security control and
monitoring using microcontroller, Asia Pac. J. Res., vol. 1, no. XXV,
pp. 1925, Mar. 2015.
[22] M. Miyakawa, Y. Maeda, and N. Koyama, Relationships between
usability and number of buttons of a universal remote console -preliminary experimental result-, in Proc. IEEE Int. Conf. Syst. , Man
Cybner., Singapore, Oct. 2008, pp. 424428.
[23] J. Haines and J. M. Winters, Evaluating automatically generated personalized interfaces on an ergometer for people with diverse abilities,
in Proc. IEEE Int. Conf. Eng. Med. Biol. Soc., Vancouver, BC, Canada,
Aug. 2008, pp. 56325635.
[24] J. H. Andrews and F. Hussain, Johar: A framework for developing
accessible applications, in Proc. 11th Internat. ACM SIGACCESS
Conf. on Computers and Accessibility, Pittsburgh, USA, Oct. 2009,
pp. 243244.
[25] H. Y. Tung, K. F. Tsang, H. C. Tung, K. T. Chui, and H. R. Chi, The
design of dual radio ZigBee homecare gateway for remote patient monitoring, IEEE Trans. Consumer Electron., vol. 59, no. 4, pp. 756764,
Nov. 2013.
[26] C. I. Penaloza, Y. Mae, F. F. Cuellar, M. Kojima, and T. Arai, Brain
machine interface system automation considering user preferences and
error perception feedback, IEEE Trans. Autom. Sci. Eng., vol. 11, no.
4, pp. 12751281, Oct. 2014.
[27] K. Lee, Y. S. Kim, Y. Wang, S. Kim, and K.-H. Baek, A remote
pointing device by using modulated IR signals with a 2-D striped pattern, IEEE Trans. Consumer Electron., vol. 59, no. 3, pp. 699704,
Aug. 2013.
[28] F. Portet, M. Vacher, C. Golanski, C. Roux, and B. Meillon, Design
and evaluation of a smart home voice interface for the elderly: Acceptability and objection aspects, Pers. Ubiquit. Comput., vol. 17, no. 1,
pp. 127144, Jan. 2013.

[29] R. Aoki, A. Maeda, T. Watanabe, M. Kobayashi, and M. Abe,

Twist&tap: Text entry for TV remotes using easy-to-learn wrist
motion and key operation, IEEE Trans. Consumer Electron., vol. 56,
no. 1, pp. 161168, Feb. 2010.
[30] L. Jing, Y. Zhou, Z. Cheng, and T. Huang, Magic ring: A nger-worn
device for multiple appliances control using static nger gestures,
Sensors, vol. 12, no. 5, pp. 57755790, May 2012.
[31] M. Shin and W. Lee, Beyond-screen interface for interactive television, Archives of Design Res., vol. 27, no. 1, pp. 143157, Feb. 2014.
[32] H. Jinsoo, Y. Jaekwan, J. Jonghyun, and P. Kwang-Roh, User-friendly
home automation based on 3D virtual world, IEEE Trans. Consumer
Electron., vol. 56, no. 3, pp. 18431847, Aug. 2010.
[33] K. Taewan, L. Hakjoon, and C. Yunmo, A universal remote control
with haptic interface for customer electronic devices, IEEE Trans.
Consumer Electron., vol. 56, no. 2, pp. 913918, May 2010.
[34] L. Wang, D. Peng, and T. Zhang, Design of smart home system based
on WiFi smart plug, Int. J. Smart Home, vol. 9, no. 6, pp. 173182,
Jul. 2015.
[35] F. Paganelli, F. Paradiso, S. Turchi, A. Luchetta, P. Castrogiovanni,
and D. Giuli, Appliance recognition in an OSGi-based home energy
management gateway, Int. J. Distrib. Sens. N., vol. 2015, pp. 112,
Jan. 2015.
[36] V. Vujovi and M. Maksimovi, Raspberry Pi as a sensor web node for
home automation, Comput. Electr. Eng., vol. 44, pp. 153171, May
2015.

Kuen-Min Lee received the M.S. degree in electrical

engineering from National Chung Cheng University,
Chiayi, Taiwan, in 1999. He is currently working toward the Ph.D. degree in engineering science at National Cheng Kung University, Tainan, Taiwan.
Additionally, he is a Senior Engineer with the
Information and Communications Research Laboratories, Industrial Technology Research Institute
(ITRI), Tainan, Taiwan. His research interests
include embedded systems, home networking, and
cloud distributed systems.

Wei-Guang Teng (M'06) received the B.S. and

Ph.D. degrees in electrical engineering from National Taiwan University, Taipei, Taiwan, in 1998
and 2004, respectively.
He is currently an Associate Professor with the
Department of Engineering Science, National Cheng
Kung University, Tainan, Taiwan. He was a Senior
Researcher with Groundhog Technologies from 2004
to 2005. His research interests include data mining,
multimedia networking, and mobile computing.

Ting-Wei Hou (M'07) received B.S., M.S., and

Ph.D. degrees in electrical engineering from National Cheng Kung University, Tainan, Taiwan, in
1983, 1985, and 1990, respectively.
He is currently a Professor and the Chairman of the
Department of Engineering Science, National Cheng
Kung University. He also served as the director in the
Department of Medical Informatics, National Cheng
Kung University Hospital, from 2007 to 2014. His
major researches include smart home computing systems, medical information systems, and smart hospital computing systems.