Assistive Technology

©istockphoto.com/
StockImages_AT

Assistive intelligent
robotic wheelchairs
Iosif Papadakis Ktistakis and Nikolaos G. Bourbakis

I
n a world where technological
achievements and breakthroughs
a re astonishing a nd where
computers a nd a ll kinds of
electronic devices have become
one with their users, there are still
people that need assistance. This
t y pe of a id often has to come
through technology and the imple-
mentation of ideas. People with dis-
abilities and the elderly usually
require diligent monitoring and help
to complete their daily tasks.

Around the world, there are mil-
lions of disabled and elderly people
who rely on wheelchairs in their
daily lives. Wheelchairs provide the
elderly with easier mobility and the
disabled with the ability to move.
What if these wheelchairs could do
more for people? What would the
benefits be, and what would people
gain from such a development?
The continuous health monitor-
ing of people in need can be achieved
with ease with wheelchairs that have
intelligent software installed. The
common solution through human
health-care professionals does not
scale economically. Moreover, due
to the high cost of medical care for
these groups, and, at the same time,
the lack of large number of quali-
Digital Object Identifier 10.1109/MPOT.2016.2614755
Date of publication: 10 January 2017
fied health-care providers, robotic
nurses [intelligent robotic wheel-
c h a i r s ( I R W s ) ] i n c o m b i n a tion
with health-monitoring devices
(wearable health-monitoring devic-
es) represent the most effective and
desirable solution.
The continuous monitoring of
people at risk and the elderly offers
additional safety. Often these people
are vulnerable to crimes like theft
and homicide. As a result, techno-
logical assistants may offer them a
sense of protection that can drive sus-
picious people away. Robotic nurses
©istockPhoto.com/Wavebreakmedia
can provide more help in assisting
these people in their daily tasks and
needs. Several activities such as eat-
ing and drinking, washing, and mov-
ing are some of the daily tasks with
which an elderly person or a person
with disabilities may need help. One
significant challenge is when a per-
son is not able to get up from a sitting
position on his/her own, which will
be further analyzed in this article.
In the United States, the elder
population (people 65 years and old-
er) numbered 46.2 million in 2014,
as reported by the Administration

10 ■ January/February 2017 IEEE Potentials 0278-6648/17©2017IEEE

on Aging. This represents one in ev-
ery seven Americans, 14.5% of the
U.S. population. The number of el-
derly are increasing and, by 2060,
there will be approximately 98 mil-
lion, more than twice the 2014 num-
ber, as detailed by the Administra-
tion for Community Living. This
data increases when checking the
World Bank of Data for population
ages 65 and above, which shows the
yearly increase in this group of peo-
ple. Expectations show that by the
year 2050, the amount of people in
this group will be 2.5 times larger.
Moreover, in the United States,
approximately 56.7 million people
(19% of the population) have a dis-
ability, according to U.S. Census
Bureau reports. In addition, approx-
imately 33% of people over 65 ex-
perience a fall each year. Falls also
ocurred to 50% of people over 85
years old. The injuries that are as-
sociated with these falls range from
hip fractures to death.
More than 700 million people
around the world have some kind of
disability, according to the United
Nations and the World Health Organi-
zation. Considering these statistics,
one deduction can be made: more
people will need technical assistance
in the near future than they need
now. This increasing rate of people
in need of assistance has gained the
attention of the scientific commu-
nity over the last few years. Several
solutions have been proposed to al-
low people from these groups to live
a more independent life. One of these
solutions is an IRW or a robotic nurse.
There are special wheelchairs that
offer enhanced capabilities and are
made for assisting people in need.
Some have robotic arms mounted
on them and perform various daily
tasks, others include cameras and
sensors and can walk through cor-
ridors by themselves, and some can
even twist knobs to open doors. There
are several wheelchairs that perform
a single task but very few can com-
bine more than one task.
Research and development
The first illustrations of a wheel-
chair go back to the ancient Greek
IEEE Potentials January/February 2017 ■ 11
culture, but the first wheelchair is
considered to be the one made for
Philip II of Spain in 1595. Since
that time, wheelchairs have ad-
vanced toward so-called “smart” or
“intelligent” wheelchairs. The idea
of using robotic wheelchairs as
caregivers has been explored in the
past. A major breakthrough hap-
pened between 1995 a nd 1999
when t he Robot ics W heelcha ir
Competition took place at the Inter-
national Joint Conference on Artifi-
cial Intelligence (IJCAI), where the
first attempts to create new wheel-
chairs w ith robotic capabilities
were made. The first prototypes
were developed over the next few
There are different types of assistive systems,
and their applications range in mobility assistance
from an intelligent wheelchair to assistive
manipulation and physical rehabilitation.
years, but eventually, this develop-
ment ceased. In the last few years,
several prototypes have been devel-
oped, and many scientific works
have been published in the area.
As mentioned previously, there
are different types of assistive sys-
tems, and their applications range
in mobility assistance from an intel-
ligent wheelchair to assistive manip-
ulation and physical rehabilitation.
Among the commercial systems, the
most successful are the Manus ma-
nipulator by Exact Dynamics, the
Rapter manipulator by Applied Re-
sources Corporation, and the JACO
manipulator by Kinova Robotics.
These systems offer the ability to in-
teract with the world by controlling
the movement of a robotic arm. Al-
though mounting a robotic manipu-
lator on a wheelchair designed for
transportation and maneuverability
has limitations, there are ways to
overcome them.
In Simpson (2005), a compre-
hensive review is provided of IRWs
that have been implemented up to
the year 2004. More recently, work
has been done by Abolghasemi and
his team, where they implemented a
wheelchair with a robotic arm JACO
with the goal of managing to position
itself to pick up a glass of water from
a table. Jiang and his team imple-
mented a 3-D vision-based system to
control an IRW that allows the user
to perform daily tasks such as get-
ting dressed and eating. It uses ges-
tures, speech, and a combination of
the two as inputs to control the ma-
nipulator (the robitc arm JACO).
Wang has implemented a wheel-
chair with two robotic arms, called
Manus, which is a personal mo-
bility and manipulation appliance
(PerMMA). Wang and his team have de-
veloped an IRW with the goal of provid-
ing users mobility and ­manipulation
­ ptions as well as serving as a clinical
o
tool. Two wheelchair versions have
been implemented: Gen I PerMMA
and Gen II PerMMA. The first focus-
es on enhanced manipulation, while
the second highlights advanced mo-
bility control.
Pathirage and his group created a
P300 brain computer interface for an
IRW to grasp objects using a robotic
arm. An adaptive visual stimuli grid
was used to select an object from the
scene image. Graser and his team
created FRIEND to assist people with
severe disabilities in daily tasks. The
latest version is designed to assist li-
brarians with cataloging books and
collecting them.
Kim and his group developed UCF-
MANUS and UCF-MANUS II, six-
degrees-of-freedom IRWs, that use
vision information, object detection,
recognition, and grasping. It is used
for daily tasks for people with severe
physical impairments.
Cunningham and his colleagues
produced Jamster, a mobile, dual-arm
assistive robot that performs a vari-
ety of tasks. The dual arm is a Bax-
ter robot by Rethink Robotics and is
mounted on a wheelchair. It can be
 (a) (b) (c)

Fig1 (a) The initial wheelchair, (b) the wheelchair with arms, and (c) the final view.

Fig2 The get-up and sit-down states of the intelligent robotic wheelchair Nosokoma.

commanded with a sip puff device with two robotic arms (Fig. 1) and a ligent software algorithms that will
called the Jamboxx. wearable health-monitoring device to be applied to it. Nosokoma will use
Faria and her team developed the service the needs of a user. This piece in-house intelligent algorithms that
Intellwheels project. Their aim was of technology is being produced are related to path planning/naviga-
to create an intelligent wheelchair Wright State University at the Center tion, object recognition, tracking and
platform that will adapt to any com- for Assistive Research Technologies. grasping, and learning patterns. In
mercial wheelchair. The commands The wheelchair is Nosokoma, which addition, robust human-user and
were given w ith a joystick, voice is a Greek word meaning “the nurse.” IRW interaction is needed.
commands, head movements, ges- The approach is to design and im- As mentioned previously, some
tures, and facial expressions. plement a mobile, human-friendly, people require assistance with their
daily tasks. One of the frequent
tasks that Nosokoma will perform is
The IRW, Nosokoma, will move in front of the user to help a person to get up from a sit-
in a predefined personalized position, ting position and rest on the IRW. A

after a command is given by a user, and will aid person would normally perform the
action of getting up without think-
the user in getting up from the sitting position ing or stressing about it. The act is
to help him/her relax on the wheelchair. written in the nerves and muscles of
the human system since it is a often-
executed task undertaken since a
These systems share two basic dual-arm robotic assistant wheel- very young age.
requirements. First, the robotic sys- chair that will be operated by inter- However, there are people who
tem must be safe for the user to move face and commands. The autono- struggle when they try to get up from
and to perform daily tasks. Any fail- mous wheelchair will carry a laptop, a chair because of their weakened
ures must be predicted in a way to two cameras, a microphone, speak- legs or the inability of their body to
prevent any harm from coming to ers, range sensors, a Global Posi- follow the commands of their nervous
the user. Second, for such a system tioning System (GPS), a screen, and system. These people might need ex-
to be useful, it needs to interact ef- the two robotic arms. The goal is to tra help or a small amount of force to
fectively with the user. advance wheelchairs into the future pull them upward. This is where the
with a robot assistant for people Nosokoma can assist. There are sev-
Here comes the robotic nurse at need. eral scenarios that have been stud-
The IRW presented here consists of In particular, this transformation ied and are currently being imple-
an autonomous wheelchair mounted will be achieved by developing intel- mented. More specifically, the IRW,

12 ■ January/February 2017 IEEE Potentials

Nosokoma, will move in front of the
user in a predefined personalized po-
sition, after a command is given by a
user, and will aid the user in getting
up from the sitting position to help
him/her relax on the wheelchair.
The robotic arms extend and provide
a personalized amount of force that
will help a user to get up, as seen in
Fig. 2, which is one of the scenarios
that have been studied.
In this scenario, the IRW will use
the cameras to create stereo images,
detect the user’s position (although
most of its time the IRW will track the
user), and determine the distance to
travel to reach the predefined posi-
tion from which it will extend its ro-
botic arms to safely reach the user.
After verbal interaction, the user will
grab the robotic arms and synchro-
nize his/her effort with the move-
ment of the robotic arms.
Nosokoma uses a neural network
to learn the user’s responses and
range of forces so that it can make
the appropriate adjustments. The as-
sumption has been made that users
of extreme and high-risk cases will
not be tested for the project. Nosoko-
ma will also have subtasks including
the delivery of objects. The delivery
subtask will be a combination of rec-
ognition, grasping, path planning,
and navigation. Also, Nosokoma can
bring an object to the user by being
aware of its position from the GPS
coordinates and from the location of
the user in the house floor map.
Conclusions
This article offered a review on the
state of the art in research develop-
ment of IRWs for assisting people in
need. IRWs have the potential to
revolutionize the health-care system
and provide assistance to millions
of people around the world. With
new technological achievements
that are to come, new designs and
new ideas will be implemented.
Read more about it
••R. C. Simpson, “Smart wheel-
chairs: A literature review,” J. Reha-
IEEE Potentials January/February 2017 ■ 13
bil. Res. Develop., vol. 42, no. 4, pp.
423–436, 2005.
••P. Abolghasemi, R. Rahmatiza-
deh, A Behal, and L. Bölöni, “A real-
time technique for positioning a
wheelchair-mounted robotic arm for
household manipulation tasks,” in
Proc. Workshop on Artificial Intelli-
gence Applied to Assistive Technolo-
gies and Smart Environments, 2016.
••H. Jiang, T. Zhang, J. P. Wachs,
and B. S. Duerstock, “Enhanced
control of a wheelchair-mounted ro-
botic manipulator using 3-D vision
and multimodal interaction,” Comp.
Vision Image Understand. J., vol.
149, pp. 21–31, 2016.
••H. Wang, G. G. Grindle, J. Can-
diotti, C. Chung, M. Shino, E. Hous-
ton, and R. A. Cooper, “The personal
mobility and manipulation appliance
(PerMMA): A robotic wheelchair with
advanced mobility and manipula-
tion,” in Proc. Annu. Int. Conf. IEEE
Engineering in Medicine and Biology
Society, 2012, pp. 3324–3327.
••I. Pathirage, K. Khokar, E. Klay,
R. Alqasemi, and R. Dubey, “A vision
based P300 brain computer inter-
face for grasping using a wheelchair-
mounted robotic ar m,” i n Proc.
IEEE/ASME Int. Conf. Advanced In-
telligent Mechatronics, 2013, pp.
188–193.
••A. Graser, T. Heyer, L. Fotoohi,
U. Lange, H. Kampe, B. Enjarini,
and D. Ristic-Durrant, “A Supportive
FRIEND at work: Robotic workplace
assistance for the disabled,” IEEE
Robot. Automat. Mag., vol. 20, no. 4,
pp. 148–159, 2013.
••D.-J. Kim, R. Hazlett-Knudsen,
H. Culver-Godfrey, G. Rucks, T. Cun-
ningham, D. Portee, and A. Behal,
“How autonomy impacts performance
and satisfaction: Results from a study
with spinal cord injured subjects us-
ing an assistive robot,” IEEE Trans.
Syst. Man Cybern. A, vol. 42, no. 1,
pp. 2–14, 2012.
••A. Cunningham, W. Keddy-Hec-
tor, U. Sinha, D. Whalen, D. Kruse, J.
Braasch, and J. T. Wen, “Jamster: A
mobile dual-arm assistive robot with
jamboxx control,” in Proc. IEEE Int.
Conf. Automation Science and Engi-
neering, 2014, pp. 509–514.
••B. M. Faria, L. P. Reis, N. Lau,
J. C. Soares, and S. Vasconcelos,
“Patient classification and automatic
configuration of an intelligent wheel-
chair,” in Proc. Int. Conf. Agents and
Artificial Intelligence, Berlin, 2012,
pp. 268–282.
••I. P. Ktistakis and N. Bourba-
kis, “A survey on robotic wheelchairs
mounted with robotic arms,” in Proc.
IEEE Nat. Aerospace and Electronics
Conf., 2015, pp. 258–262.
••N. Bourbakis, I. P. Ktistakis, L.
Tsoukalas, and M. Alamaniotis, “An
autonomous intelligent wheelchair
for assisting people at need in smart
homes: A case study,” in Proc. IEEE
6th Int. Conf. Information, Intelli-
gence, Systems and Applications,
2015, pp. 1–7.
••A. Pantelopoulos and N. Bour-
bakis, “A survey on wearable based
systems for monitoring and progno-
sis,” IEEE Trans. Syst. Man Cybern.,
vol. 40, no. 1, pp. 1–12, 2010.
About the authors
Iosif Papadakis Ktistakis (iosif
.papadakis@wright.edu) earned his
diploma degree from the ­Production
and Management Engineering Depart-
ment, Technical ­University of Crete,
Greece. He is currently a Ph.D. stu-
dent at Wright State University in
Dayton, Ohio, and a graduate research
assistant at the Center of Assistive Re-
search Technologies.
Nikolaos G. Bourbakis (nikolaos
.bourbakis@wright.edu) earned his
Ph.D. degree in computer science
and engineering from the University
of Patras, Greece. He is currently
the OBR distinguished professor of
information technology in the Com-
puter Science a nd Engineer ing
Department, a professor in the Ge-
riatrics Department, and the direc-
tor of t he Center of A ssist ive
Research Technologies at Wright
State University in Dayton, Ohio. He
is an IEEE Fellow.