Está en la página 1de 21

Expert Systems with Applications 42 (2015) 12021222

Contents lists available at ScienceDirect

Expert Systems with Applications


journal homepage: www.elsevier.com/locate/eswa

RecomMetz: A context-aware knowledge-based mobile recommender


system for movie showtimes
Luis Omar Colombo-Mendoza a,, Rafael Valencia-Garca a, Alejandro Rodrguez-Gonzlez b,
Giner Alor-Hernndez c, Jos Javier Samper-Zapater d
a

Facultad de Informtica, Universidad de Murcia, Campus de Espinardo, Spain


Department of Engineering, School of Engineering, Universidad Internacional de la Rioja, Spain
Division of Research and Postgraduate Studies, Instituto Tecnolgico de Orizaba, Mexico
d
Departament dInformtica, Escola Tcnica Superior dEnginyeria, Universitat de Valncia, Av. de la Universitat, 46100 Burjassot, Valncia, Spain
b
c

a r t i c l e

i n f o

Article history:
Available online 22 September 2014
Keywords:
Knowledge-based recommender systems
Context-aware systems
Semantic Web
Ontology reasoning

a b s t r a c t
Recommender systems are used to provide ltered information from a large amount of elements. They
provide personalized recommendations on products or services to users. The recommendations are
intended to provide interesting elements to users. Recommender systems can be developed using different techniques and algorithms where the selection of these techniques depends on the area in which they
will be applied. This paper proposes a recommender system in the leisure domain, specically in the
movie showtimes domain. The system proposed is called RecomMetz, and it is a context-aware mobile
recommender system based on Semantic Web technologies. In detail, a domain ontology primarily
serving a semantic similarity metric adjusted to the concept of packages of single items was developed
in this research. In addition, location, crowd and time were considered as three different kinds of contextual information in RecomMetz. In a nutshell, RecomMetz has unique features: (1) the items to be recommended have a composite structure (movie theater + movie + showtime), (2) the integration of the time
and crowd factors into a context-aware model, (3) the implementation of an ontology-based context
modeling approach and (4) the development of a multi-platform native mobile user interface intended
to leverage the hardware capabilities (sensors) of mobile devices. The evaluation results show the efciency and effectiveness of the recommendation mechanism implemented by RecomMetz in both a
cold-start scenario and a no cold-start scenario.
2014 Elsevier Ltd. All rights reserved.

1. Introduction
Recommender systems are software tools and techniques providing suggestions for items to be of use to a user (Ricci, Rokach,
& Shapira, 2011). Recommender systems began to appear in the literature in the early nineteens, specically, the Tapestry mail system (Goldberg, Nichols, Oki, & Terry, 1992) is considered the rst
recommender system since the collaborative ltering concept
was coined (Resnick & Varian, 1997).
Recommender systems can be classied into ve different categories depending on the technique employed to predict the utility
of the items for the user, i.e., according to the recommendation
technique:
Corresponding author.
E-mail
addresses:
luisomar.colombo@um.es
(L.O.
Colombo-Mendoza),
valencia@um.es
(R.
Valencia-Garca),
alejandro.rodriguez@unir.net
(A. Rodrguez-Gonzlez), galor@itorizaba.edu.mx (G. Alor-Hernndez), jsamper@
irtic.uv.es (J.J. Samper-Zapater).
http://dx.doi.org/10.1016/j.eswa.2014.09.016
0957-4174/ 2014 Elsevier Ltd. All rights reserved.

1. Content-based recommender systems: these systems recommend items that are similar to the ones that the user liked
in the past.
2. Collaborative ltering recommender systems: these systems
recommend to the active user the items that other users
with similar tastes liked in the past.
3. Demographic recommender systems: these systems recommend items based on the identication of the demographic
niche the user ts better according to a personal demographic prole.
4. Knowledge-based recommender systems: these systems recommend items based on either inferences about user preferences or specic domain knowledge about how items
meet user preferences.
5. Hybrid recommender systems: these systems are based on the
combination of the above mentioned techniques.
Nowadays, recommender systems have taken advantage of
Semantic Web technologies to efciently overcome the challenges

L.O. Colombo-Mendoza et al. / Expert Systems with Applications 42 (2015) 12021222

related to the incredible growth of the Web: information overload


and data sources heterogeneity. In fact, Semantic Web technologies
enable computer systems to integrate, share, process and interpret
the information formerly readable by humans (Web of documents)
and infer new knowledge (Berners-Lee, Hendler, & Lassila, 2001).
Ontologies are one of the basic components of the Semantic Web
stack because they represent the standard mechanism for enriching the data with additional meaning irrespective of the area of
concern. Articial intelligence researchers have borrowed the term
ontology from philosophy and they have formalized it by means
of ve kinds of components: classes, relations, functions, axioms
and instances (Gruber, 1993).
More recently, with the aim of providing recommender systems
with the ability of recommend items to users in certain circumstances, the incorporation of contextual information into the traditional recommendation process has been leveraged. Context-aware
recommender systems are a special kind of recommender system
that take into consideration information about the context in
which recommendations occurs in addition to the information that
traditional recommender systems deal with: information about the
items to be recommended and information about the users asking
for recommendations (Adomavicius & Tuzhilin, 2008).
The boom of the mobile market, namely smartphones and tablet PCs, in addition to the advances in wireless networks, has
brought new opportunities in context-aware recommender systems development: the integration of real-time contextual information (Liu, Xu, Liao, & Chen, 2014; Noguera, Barranco, Segura, &
Martnez, 2012). For instance, the availability of location data from
positioning systems becomes more and more common among
mobile devices. In fact, context-aware computing generally refers
to ubiquitous or pervasive computing in which mobile systems
that can sense their physical environment and accordingly adapt
their behavior are considered to be context-aware systems.
In this paper, as salient contribution a context-aware knowledge-based mobile recommender system for the movies domain
called RecomMetz is proposed. The context-aware approach implemented by RecomMetz is based on a three-part model comprising
location, time and crowd information. This model is partially modeled by using a domain ontology primarily serving to the recommendation process. Unlike others contributions related to the
movies domain our proposal is intended to recommend movies
as scheduled activities rather than as content, i.e., as movie showtimes at movie theaters. In fact, the items to be recommended are
viewed as composed items: movie theater + movie + showtime.
The remaining of this paper is structured as follows. Section 2
presents a review of the state-of-the-art on recommender systems
development. In Section 3 the context-aware model implemented
by the recommender system as well as the overall recommendation process are explained. The architecture and internal functionality of the recommender system are described in Section 4.
Section 5 presents the evaluation method aimed at assessing the
performance of the recommender system. Finally, conclusions
and future directions are discussed in Section 6.
2. Related works
In recent years, several researches have been proposed in the
tourism and leisure domains because of the importance of the corresponding industries for the world economy. According to the
World Travel & Tourism Council (WTTC), the direct contribution
of the Travel & Tourism industry to GDP in 2012 was USD2,056.6bn
(2.9% of GDP). This contribution includes the economic activity
generated by the restaurant and leisure industries directly
supported by tourists, along with the economic activity generated
by industries such as hotels, travel agents, airlines and other passenger transportation services (Turner, 2013).

1203

In particular, different recommendation approaches have


emerged due to the requirements inherent to the aforementioned
domains: (1) personalization: the users need to nd the most suitable items (e.g. points of interest, restaurants and movies) for them
starting from their personal preferences, (2) information ltering:
the users need to identify the most relevant information (e.g. visitors ow, customer opinions, critics reviews and ratings) about
items to be chosen. Most of these proposals are focused on the
tourism domain whereas the few proposals on the leisure domain
are actually related to activities supported by tourists.
In this work, the implementation proposed is related to an
activity not necessarily supported by tourists. In order to identify
the opportunity niches of our approach, prior approaches on
knowledge-based recommendation, context-aware recommendation and mobile recommendation are reviewed and discussed in
following subsections. In addition, a comparative analysis is presented for each research eld. It is important to notice that one
proposal may be related to more than one research eld at a time
so that the same proposal would appear in more than one comparative analysis. Proposals not completely related for the purposes of
the comparative analysis are only discussed.
2.1. Context-aware recommender systems
The concept of context has signicantly evolved since it was
rst introduced in Schilit, Adams, and Want (1994). Nowadays, it
is widely accepted that there may be diverse kinds of contextual
information classied into three main categories: user context
(e.g. location and mood), device context (e.g. network bandwidth
and display resolution) and physical context (e.g. time and
weather) depending on the perspective adopted: user-side or
application-side (Jung, 2009; Ylmaz & Erdur, 2012). The interpretation of context may vary between one application to another
according to the domain and architecture of the application so that
it is quite suitable to give a denitive denition of this concept. For
instance, Adomavicius and Tuzhilin (2008) described possible
interpretations of context across different elds concerning to recommender systems such as e-commerce and ubiquitous systems.
Although location seems to be a recurrent kind of contextual information among context-aware recommender systems, it does not
always refer to the geographical location of the user either relative
or absolute as in the case of the social tag-based collaborative ltering approach proposed by Lee, Kaoli, and Huang (2014) as part
of a smart TV system. This approach is context-aware in the sense
that it considers information about both user context and device
context. Here, the recommendations are solely computed from
the user preferences; then, the resulting recommendations are
re-ranked accordingly. For instance, location is dened as the
appropriateness of the multimedia content for the place in which
the user is accessing the system, and it is classied either as public
or private. Similarly, in Tsai and Chung (2012) location was interpreted as an static parameter representing the geographical location of the public information booths in a theme park. In this
case, the location of a user in the park is tracked each time the user
accesses a booth by using an RFID wristband and it is determined
by the location of the booth the user is accessing. To a lesser extent,
time and crowd are also recurrent kinds of contextual information.
For instance, Dao, Jeong, and Ahn (2012) presented a Genetic
Algorithm (GA)-based collaborative ltering approach for mobile
location-based advertising. This approach addressed contextawareness by calculating the similarity between contexts along
with the similarity between users (neighbors). Here, context is
the combination of the time and motivation factors whereas location is considered individually. In detail, time is dened as the day
and time a user is going to visit a place. On the assumption that the
places the users visit vary depending on the period of the day, the

1204

L.O. Colombo-Mendoza et al. / Expert Systems with Applications 42 (2015) 12021222

Table 1
Comparison of proposals for context-aware recommendation (KB = knowledge-based, CFB = collaborative ltering-based, CB = content-based; PrF = pre-ltering, PoF = postltering, CM = contextual modeling, PB = programming-based, DMB = data model-based, OB = ontology-based).
Work

Kind of recommender
system (RS)
KB

CFB

SMARTMUSEUM (Ruotsalo et al.,


2013)

SR-REJA-3D (Noguera et al.,


2012)
iTravel (Yang & Hwang, 2013)
Route recommendation system
(Tsai & Chung, 2012)

(Lee et al., 2014)

Group corrections-driven
ltering (Blanco-Fernndez
et al., 2011)
Context-aware collaborative
ltering using genetic
algorithm (CACF-GA) (Dao
et al., 2012)
Wang and Wu (2011)
Yang, Cheng, and Dia (2008)
SigTur/E-destination (Moreno,
Valls, Isern, Marin, & Borrs,
2013)
Fang et al. (2012)
iConAwa (Ylmaz and Erdur,
2012)
Turist@ (Batet, Moreno, Snchez,
Isern, & Valls, 2012)

Filtering technique

Context-aware
approach

Ontology-based
context model

PrF

PB

X (item-based)

Vector space model (VSM)- based


cosine similarity

X (user-based)

Unknown

X (user-based)
X (item-based)

Pearson correlation coefcient


Change detection method for
sequential patterns and k-medoids
clustering algorithm
Pearson correlation coefcient and
Tanimoto coefcient

CB

X (user-based)

X (user-based)

X (user-based)

X (user-based)

X (user-based)

X
X

X
N/A (expert system)
X (user-based)

Knowledge-based similarity (distance


in hierarchy, common attributes and
common siblings) and Pearsoncorrelation coefcient
Adjusted Pearson correlation
coefcient and Genetic Algorithmbased similarity metric (contextsimilarity)
Pearson correlation coefcient and
association rules mining
Cosine similarity
Aggregation operator (OWA and LSP)based similarity measure, Pearson
correlation coefcient and k-means
clustering algorithm
Association rules mining
Rule-based reasoning
Weighted Euclidean distance, ClusDM
clustering algorithm Manhatan
distance, Minkowsky distance among
others

visiting time is not computed as the time in hours and minutes but
as a class representing the period to which the time corresponds:
morning, lunch time, afternoon and so on. Lee et al. (2014) did
not interpret crowd as the amount of people around the smart
TV system but as the kind or audience or companion so that the
appropriateness of the multimedia content for the audience
around the user accessing the system is determined by using the
Motion Picture Association of America (MPAA) rating system.
Table 1 presents a comparative analysis of context-aware recommender systems reviewed in the literature. This comparative
analysis considered the following factors: (1) kind of recommender
system, (2) ltering technique, (3) context-aware approach, (4)
context modeling approach; and (5) context dimensions.
2.1.1. Geo-recommender systems
According to Carretero, Isaila, Kermarrec, Taiani, and Tirado
(2012) geo-recommendation is the ability to recommend places
of possible interest to a user. We have adjusted this denition to
cover location-awareness as follows: the ability to recommend
places of possible interest to a user taking into account both the
current geographical location of the user and the geographical
location of the places. As it can be inferred, this new denition
of geo-recommendation takes into consideration the dynamic nature of users locations allowing for mobility, a concept from
mobile computing that is further described in Section 2.3 of this
paper. This is how recommender systems and mobile computing
converge in geo-recommender systems research. Some already
existing proposals supporting this asseveration are discussed
below. Fang et al. (2012) proposed a recommender system for

PoF

CM

Out-door scenario:
location, in-door
scenario: companion
and motivation
Location

X
X

Location
Location, time and
crowd

Location type, mobile


device, crowd and
network conditions
Time

Location and time

X
X
X

Location
X

Location
Location

X
X

OB
X

DMB

Context dimensions

X
X

Location
Location
Location

assisting in-door shopping. In addition, a Received Signal Strength


(RSS) pattern mining positioning method was proposed due to the
constraints of current positioning systems for in-door usage. In
detail, the users locations are inferred by a location server from
the real-time information transferred by their mobile devices as
a result of measuring the strength of the radio signals received
from nearby base stations. The proximity of users to stores is determined by computing the similarities between the real-time RSS
information and pre-dened RSS patterns in a database in which
these RSS patterns represent the location of the stores. In a nutshell, the location of each user is calculated as the location of his
nearest store. Yang, Cheng, and Dia (2008) proposed a locationaware mobile recommender system for m-commerce or rather,
for location-based advertising. This system relies on an agentbased architecture so that a catalog of vendors information such
as physical addresses and webpages links needs to be maintained
by the vendors themselves. The webpages of these vendors are recommended to users by matching the physical addresses in the vendor catalog with geographical locations tracked by the built-in GPS
receivers of the users mobile devices. In this context, the distance
between a users location and each vendors location is calculated
by using the Euclidean distance metric. In addition, user proles
learn from webpage visiting histories (implicit preferences) are
considered in the recommendation process. Batet, Moreno,
Snchez, Isern, and Valls (2012) proposed a mobile recommender
system for tourism activities called Turist@; this system also
makes use of the built-in GPS receivers of mobile devices to track
the position of users. Turist@ relies on an agent-based architecture
with the aim of providing pro-active recommendations along with

L.O. Colombo-Mendoza et al. / Expert Systems with Applications 42 (2015) 12021222

1205

Table 2
Comparison of proposals for geo-recommendation.
Work

Geo-localization technology

Geo-recommendation technique

SMARTMUSEUM (Ruotsalo et al., 2013)


SR-REJA-3D (Noguera et al., 2012)
Route recommendation system (Tsai and Chung, 2012)
Wang and Wu (2011)
Yang et al. (2008)
SigTur/E-destination (Moreno et al., 2013)
Fang et al. (2012)
iConAwa (Ylmaz and Erdur, 2012)

GPS receivers of mobile devices


GPS receivers of mobile devices
RFID readers
RFID reader-equipped mobile devices
GPS receivers of mobile devices
Static location information
Radio signal receiver of mobile phones
GPS receivers of mobile devices

Fixed-distance bounding box


Euclidean distance
N/A
N/A
Euclidean distance
JST topology suite API-based
Received Signal Strength (RSS)-based positioning method
Haversine formula

the typical on-demand recommendations. In detail, when a user is


near a potentially interesting activity, an agent running in his
mobile device can automatically suggest the activity to him. The
distance is calculated taking into account the users position and
the location of each one of the places where the activities are
carried.
Table 2 presents a comparative analysis of geo-recommender
systems reviewed in the literature. This comparative analysis considered the following factors: (1) geo-localization technology, (2)
geo-recommendation technique.
2.2. Knowledge-based recommender systems
The use of a semantic description of the domain in which the
recommendations are provided has become an increasingly common approach towards the improvement of the quality of the recommendations. To this aim, ontologies have been used as the
preferred mechanism due to their capacity for semantically
describing concepts without the constraints typically imposed by
other models such as databases (open-world assumption). In fact,
ontologies are formal and explicit descriptions of shared conceptualizations, and they can be exploited in several ways. Recent proposals in recommender systems literature take advantage of
ontologies to model the features of the users besides the features
of the items in the domain of the systems, i.e. the items handled
by the systems (Moreno, Valls, Isern, Marin, & Borrs, 2013;
Ruotsalo et al., 2013). This approach has resulted in the so-called
ontology-based user prole modeling. Furthermore, starting from
ontology-based user proles and domain ontologies, recommendation processes can be in part addressed as implicit knowledge
inference processes. For instance, an ontology-based recommender
system for tourism and leisure activities called SigTur/E-Destination was proposed in Moreno et al. (2013). This system takes
advantage of a domain ontology; this ontology also guides the recommendation process to some extent. A particular version of the
domain ontology is built for each user with the aim of storing
the degrees of interest in the items in the domain of the system
for a given user. Thereby, inferences based on the propagation of
the explicit preferences over the hierarchies of concepts in the
ontology are easily carried out. Other proposals related to the eld
of context-awareness have been used ontologies to model the
information about the context of the users along with his preferences/behaviors (Blanco-Fernndez, Lpez-Nores, Pazos-Arias, &
Garca-Duque, 2011; Ylmaz & Erdur, 2012). For instance, BlancoFernndez et al. (2011) proposed an ontology-supported timeaware recommendation approach for e-commerce; this approach
was intended to model the variation of the potential interest of
each type of product or any of its features with regard to absolute
dates or purchase times. To this aim, time functions representing
the variations in users interests were linked to the items in the
ontology (classes and properties). In addition, user group proles
were used to characterize consumption stereotypes existing
among users with similar preferences so that the shapes of the
time functions were dynamically corrected by temporal curves

built from the consumption stereotypes. In Ylmaz and Erdur


(2012), a context-aware expert system called iConAwa was proposed, providing users with information and services about nearby
points of interest (POI). Like the Turist@ recommender system, iConAwa relies on an agent-based architecture with the aim of providing location-aware pro-active functioning. In that work, the concept
of context ontology was introduced, so that the context was modeled by using an ontology primarily composed of information about
location, time, mobile device and network factors. The ontology was
used to model user proles as well, so that user preferences were
considered to be part of the context, and not vice versa. From this
perspective, each time the context of a new user need to be characterized, new instances of the classes of the context ontology are created. It is important to notice that in this case the domain of the
system was modeled by means of a separate ontology.
Table 3 presents a comparative analysis of knowledge-based
recommender systems reviewed in the literature. This comparative
analysis considered the following factors: (1) domain, (2) ontology
population technique to be used, (3) user proling technique, (4)
ontology-based prole; and 5) rating scale.
2.3. Mobile recommenders
Mobile computing is a paradigm that enables the use of computing capability without a predened location and network connection. It relies on the concept of mobility; mobility is the
ability to change the location of mobile elements while connected
to a network (Forman & Zahorjan, 1994). This approach has proven
to be useful in personalization research. In fact, beyond the possibility of access recommender systems at anytime and anywhere
(ubiquity) as mobile applications, the proliferation of mobile
devices such as smartphones and tablet PCs has led to the evolution of context-aware recommender systems due to the increasing
availability of GPS and GLONASS built-in sensors along with the
extended availability of other kinds of embedded sensors such as
motion and environmental sensors (Yu, Kim, Shin, & Jo, 2009;
Zheng, Zheng, Xie, & Yang, 2012). Under this context, a ubiquitous
mobile recommender system called SMARTMUSEUM was proposed in Ruotsalo et al. (2013) with the aim of recommending cultural heritage items in both out-door and in-door scenarios. To this
aim, different kinds of contextual information are leveraged by
SMARTMUSEUM depending on the scenario in which a user is
moving at a given time. The out-door scenario takes advantage of
the geographical location of the users determined by means of
GPS-enabled mobile devices in order to recommend near sites of
potential interest. In contrast, the location of the objects within a
place (a museum) is relevant for the in-door scenario in the sense
that the objects recommended by SMARTMUSEUM can be physically addressed by the users to retrieve additional information contained in RFID tags attached to them. Similarly, with the aim of
leveraging the advantages of mobile computing and, at the same
time, avoiding the drawbacks related to the well-known hardware
constraints of mobile devices, namely storage capacity and available RAM memory, Noguera et al. (2012) presented a Geographic

1206

L.O. Colombo-Mendoza et al. / Expert Systems with Applications 42 (2015) 12021222

Table 3
Comparison of proposals for knowledge-based recommendation.
Work

SMARTMUSEUM (Ruotsalo
et al., 2013)
SR-REJA-3D (Noguera et al.,
2012)
iTravel (Yang and Hwang,
2013)
Lee et al. (2014)

Group corrections-driven
ltering (BlancoFernndez et al., 2011)
SigTur/E-destination
(Moreno et al., 2013)
Turist@ (Batet et al., 2012)

Ontology
population
technique

User proling technique

Tourism (in-door and


out-door cultural
heritage)
Eating

Web crawling

N/A

Tourism (points of
interest (POIs))
Leisure (movies)

N/A

N/A

E-commerce

Web crawling

Tourism and leisure


(activities)
Tourism and leisure
(activities)

Manual population
by domain experts
N/A

Domain

Explicit
user
feedback

User behavior
(implicit user
feedback)

Ontology-based
user prole

Rating scale

Qualitative
(binary)

Group
proles
(stereotypes)

Quantitative
(ve-stars)
Quantitative
(ten-stars)
Quantitative
(tag-based
ve- stars)
Quantitative
(ten-stars)
Quantitative
(ve-stars)
Qualitative
(ve-stars)

Table 4
Comparison of proposals for mobile recommendation.
Work

Telecommunications network
technology

Mobile operating system

SMARTMUSEUM (Ruotsalo et al., 2013)


SR-REJA-3D (Noguera et al., 2012)
iTravel (Yang and Hwang, 2013)

Windows Mobile 6 and Symbian OS


iOS, Windows Mobile 6 and Symbian OS
Java 2 Micro Edition platform

Route recommendation system (Tsai and Chung, 2012)


Lee et al. (2014)

Wireless LAN
Cellular networks
Wireless LAN (peer-to-peer
networks)
Wireless LAN
Wireless LAN and cellular networks

Group corrections-driven ltering (Blanco-Fernndez et al., 2011)


Context-aware collaborative ltering using Genetic Algorithm (CACF-GA) (Dao
et al., 2012)
Wang and Wu (2011)
Yang et al. (2008)
SigTur/E-destination (Moreno et al., 2013)
iConAwa (Ylmaz and Erdur, 2012)
Turist@ (Batet et al., 2012)

Dial-up DBV-T and GSM


Cellular networks (via SMS and
MMS)
Wireless LAN and cellular networks
Unknown
Unknown
Wireless LAN and cellular networks
Wireless LAN and cellular networks

Information System (GIS)-based mobile recommender system for


restaurants. The system takes advantage of the advent of mobile
Graphic Processing Units (GPUS) to provide 3D-based geo-visualization capabilities. In detail, the system relies on a three-tier client/
server architecture composed of: (1) a recommendation server, (2)
a remote GIS server and (3) a mobile client application which is in
charge of on-demand downloading small subsets of the 3D maps
from the GIS server using cellular networks, and then storing them
into the devices storage. Finally, a mobile recommender system for
on-tour travel attractions called iTravel was proposed in Yang and
Hwang (2013). iTravel brings a collaborative ltering technique to
a decentralized architecture based on a mobile peer-to-peer network (Bluetooth or Wi-Fi Direct) as a means for improving the
usage of the evaluations given by other users with similar interests
(neighbors) in computing the recommendations for the active user.
In detail, two mobile devices within their proximity distance identied by means of Radio Frequency (RF) identication can
exchange evaluations data by using three different data exchange
methods: (1) unconditional exchange, (2) similarity-based
exchange and (3) hybrid method.
Table 4 presents a comparative analysis of mobile recommender systems reviewed in the literature. This comparative analysis considered the following factors: (1) telecommunications
network technology, (2) mobile operating system.

N/A (park booths)


Platform-independent (unknown Web
technology)
Java 2 Micro Edition platform
N/A
Platform-independent (JavaScript-based)
Platform-independent (JSP-based)
Platform-independent (JSF-based)
Java 2 Micro Edition platform
Java 2 Micro Edition platform

Based on the results obtained from the comparative analysis


summarized in Tables 14, we can conclude that our initiative
integrates several features that are not presented in other related
proposals or features that are considered by very few works,
namely: (1) an approach for recommending packages of items
applied to the movie showtimes domain (movie + movie theater + showtime), (2) the integration of the time and crowd factors
(in its literal sense) into a context-aware model, (3) the implementation of an ontology-based context modeling approach and (4) the
development of a multi-platform native mobile user interface
intended to leverage the hardware capabilities (sensors) of mobile
devices.

3. Context-aware recommendation approach


Context-aware recommender systems can deal with many different kinds of contextual information, each of them representing
a part or component of the context as a whole. In this paper, three
different kinds of contextual information are considered: (1) location, (2) crowd and (3) time. Location and crowd are formally considered to be factors of the user context, whereas time is
considered to be part of the physical context of the application as
is explained in Section 2.1 of this paper. However, we will hereafter

L.O. Colombo-Mendoza et al. / Expert Systems with Applications 42 (2015) 12021222

refer to the three kinds of contextual information as a whole as


user context. The components of the resulting three-part context-aware recommendation approach are described below.
3.1. Location
Users location with respect to the products/services to be recommended may be one of the most exploited factors in contextaware recommender systems. Nevertheless, this rule does not necessarily apply to tourism and leisure domains. For instance, when a
user is closer to a movie theater he has not visited yet than one he
has visited many times in the past, he is more likely to visit the farthest one despite its location.
In fact, we have proposed a distance decay-based function representing that the likelihood that a user will visit a movie theater
falls exponentially as the distance between them increases and
the frequency of visits to the movie theater decreases. This function is based on the function proposed in Yang et al. (2008) and
it is detailed in the following section of this paper.
According to the above, a location-aware recommendation
approach is proposed in this work. The approach is based on both
the current locations of users collected from GPS-enabled mobile
devices and the visiting histories of users collected from user
proles.
3.2. Crowd
The amount of people crowded around products/services may
be a relevant factor to take into account in context-aware recommendations, especially in tourism and leisure domains. For
instance, avoiding a movie premiere crowd at a movie theater
may be desirable for the people that are not going to attend the
premiere, i.e., for the people that are going to attend another showtime at the same movie theater. This may be desirable even for the
people that are going to attend the movie premiere.
Taking this into account, we have incorporated the crowdaware feature into RecomMetz based on the above outlined two
cases so that a user can be aware of the crowds at movie theaters
in real-time. A Foursquare-based manual user check-in method is
proposed for this purpose (see Section 5.3 of this paper). This
method supports the crowd-related post-ltering, enabling more
accurate recommendations. Otherwise, the crowd-aware recommendations would be based only on the assumption that the users
attend the movie showtimes recommended by RecomMetz. In
addition, the use of Foursquare as an underlying geolocation platform allows RecomMetz to be aware of the crowds at movie theaters in a more realistic way in the sense that, the users beyond
the scope of RecomMetz (Foursquare users) are also considered
as part of the crowds.
Because of the context-awareness capability, there are two different kinds of actors involved in RecomMetz: (1) active user, who
is currently making a recommendation request to the recommendation engine and (2) already served user, who has already been
served by the recommendation engine and he is currently giving
the corresponding feedback (through the user check-in mechanism). Therefore, there are two main use cases or scenarios
describing the user-system interaction: (1) feedback gathering in
the form of entries in movie theater visiting histories and (2) feedback gathering in the form of user preferences.
3.3. Time
In many cases, time may impose several limitations on recommending products/services. For instance, in tourism and leisure
domains, the access to scheduled activities (services) such as
guided tours, movie showtimes and stage plays may be limited

1207

by two different factors from a user perspective: (1) the availability


of time for accessing the products/services and (2) the amount of
time required to get to the products/services. The former factor is
related to user preferences (explicit user feedback) whereas the
latter factor is related to information about user contexts, including location and crowd.
In this paper, a time-aware recommendation approach taking
into account the latter factor is proposed. It is important to notice
that this approach is related to location information only. In detail,
three different transport modalities are considered into the user
proles in order to compute the time required to get to the products/services from the users current locations: (1) driving modality, (2) transit modality and (3) walking modality. This feature
makes our proposal markedly different from other time-aware recommendation approaches that only consider the available time of
a user for watching a movie by matching the current date and time
to the movie schedules (Yu et al., 2009).
At this point, it is important to remark that, context-aware recommender systems compute a recommendation starting from
both user contexts and user preferences as is explained below.
3.4. Integrating the context components into a recommendation
process
Traditional recommender systems process information about
users and items only. On the other hand, context-aware recommender systems additionally process information about contexts.
In fact, according to Adomavicius and Tuzhilin (2008) contextaware recommender systems implement a three-dimensional
(users, items and contexts) process comprising four different components: (1) the information about the users preferences (input),
(2) the information about the current context (input), (3) the recommendation engine and (4) the resulting set of recommendations
(output). Therefore, the recommendation engine can be described
as a function as follows: R: User  Item  Context ? Rating.
Based on the stage of the recommendation process in which the
contextual information is used, Adomavicius and Tuzhilin (2008)
identied three different paradigms in context-aware recommender systems: (1) contextual pre-ltering, (2) contextual postltering and (3) contextual modeling. In contextual pre-ltering,
the recommendation input is selected taking into consideration
the information about the current context; then, the similarities
are computed as usual. In contextual post-ltering, the recommendation input is selected ignoring the contextual information and
the similarities are computed as usual; then, the recommendation
output, i.e., the resulting set of recommendations is contextualized.
In contextual modeling, the similarities are directly computed
using the information about the current context along with the
information about items and users.
In addition, context-aware applications can deal with contextual information in three different ways known as context modeling approaches (Gu, Wang, Pung, & Zhang, 2004): (1)
programming-based approach (2) data model-based approach
and (3) ontology-based approach. Programming-based context
models are dependent on the applications, and they lack formality
and expressiveness. The data model-based approach uses conceptual modeling languages which are formal and domain-independent, e.g., the E-R model. Finally, ontology-based context models
take advantage of Semantic Web Technologies, namely ontologies,
to add the capability of reasoning about the context.
In this work, a three-dimensional context-aware recommendation process is implemented by means of a software architecture
integrating, to some extent, the above outlined typical components
of context-aware recommender systems. It is important to notice
that, in this work, the context component of the recommendation
process further comprises three more components: (1) location

1208

L.O. Colombo-Mendoza et al. / Expert Systems with Applications 42 (2015) 12021222

information, (2) time information and (3) crowd information. In


summary, the context dimension of the process is a threedimensional dimension. In addition, these different kinds of
contextual information are processed at different stages of the
recommendation process affecting either its input or its output.
Therefore, a hybrid pre-ltering/post-ltering context-aware
recommendation paradigm is adopted by the above outlined
recommendation process. In detail, the location information is
processed by using both the pre-ltering and post-ltering
techniques; the time information is processed by using the preltering technique only; similarly, the crowd information is
processed by using the post-ltering technique only. In addition,
the context modeling approach adopted in this work is the
ontology-based modeling approach as is widely explained in
Section 4.1.1 of this paper. The overall recommendation process is
described below.
(1) User prole updating: Use the location and time information
from the current context of the active user to update the corresponding user prole (dynamic information). As explained
in Section 3.3 of this paper, the time information refers to the
time at which the user made the recommendation request.
(2) Time-related pre-ltering (phase 1): Discard the movies are
currently not in movie theaters.
(3) Location-related pre-ltering: Discard the movies are currently in movie theaters if the movie theaters are far away
from the active user (above a distance threshold dened in
the user prole) and they have not been frequently visited
by the user. Therefore, not only the movie times at the nearest movie theaters but also the movie times at the most popular movie theaters irrespective of their position are
obtained by using the following formula (1):

Distance Decayu; mTi

1
eki

x f ix Distanceu;mTi

where u is a value representing the active users position, mT


is an array that contains values representing the locations of
the intended movie theaters, f is an array that contains values
indicating if the distance between the active users position
and the location of the corresponding movie theater is below
the user-dened distance threshold or not. k is an array containing the values representing the quartiles for the frequencies of visits to the corresponding movie theater.
In detail, the frequency of visits for each movie theater needs
to be rst calculated. For this purpose, the entries in the
users visiting history are analyzed so that the resulting frequency values can be classied into four groups of same size
(quartiles) starting from the lowest value; then, a percentage
representing the corresponding quartile can be assigned to
each frequency value.
(4) Time-related pre-ltering (phase 2): Discard the movie times
that the active user is not likely to attend based on the distance between the active users current position (origin) and
the theaters location (destination), and taking into account
the preferred transport modality and the most optimal route
between the origin and the destination. This operation is
repeatedly performed for the same movie because a movie
is usually scheduled at different showtimes in the same
movie theater. In fact, the time-related pre-ltering is performed starting from the nearest movie time available for
a movie by using the following formula (2):

Leeway sTi  rT gTi:

where sT is an array containing the times at which the


intended movies are scheduled (showtimes), rT is the time
at which the active user made the recommendation request,

(5)

(6)

(7)

(8)

gT is an array that contains the times required to get to the


movie theaters from the active users current location.
Similarities computing: Compute semantic similarities
between movie showtimes (similarities matrix) starting
from the knowledge inferred from the underlying domain
ontology.
Knowledge gathering: Identify the preferences of the active
user (about movies, movie theaters and showtimes) through
analyzing the corresponding user prole (static information).
Recommendations computing: Compute the recommendations to be made to the active user by using the similarities
matrix and the active users preferences. It is important to
notice that, each resulting recommendation is composed
of: (1) the name of a movie theater, (2) the title of a movie
and (3) a showtime.
Location-related post-ltering and crowd-related post-ltering:
Retrieve the crowd information (i.e., the amount of people
crowded at the intended movie theaters) from the domain
ontology; then, rank the recommendations based on both
the crowd information and the values resulting from applying the formula (1) (location-related pre-ltering). In detail,
the set of recommendation values resulting from recommendations computing is used as input for calculating quartiles as in the case of the location-related pre-ltering.
However, in this case, this operation is separately performed
for the location and crowd variables so that the recommendations are ranked based on both a value representing a
location-related quartile and a value representing a crowdrelated quartile. As it can be inferred, the location variable
refers to the values resulting from applying the formula (1)
whereas the crowd variable refers to the values representing
the amount of people crowded at the intended movie theaters and the amount of served users by RecomMetz who
are attending the intended showtimes. Finally, it is important to notice that the ranking process can be performed
using a different weight for the location and crowd variables
according to the weights provided by users through user
proles.

The functionalities and associations of the components of the


architecture intended to implement this process are clearly
described in the following section of this paper.
4. Architecture and implementation
The RecomMetzs architecture is depicted in Fig. 1. It is composed of six main components, namely user interface, Data Repository, Information Collector, recommendation engine, ContextAware Subsystem and User Check-in Subsystem.
In Fig. 1 the components involved in the RecomMetzs architecture are organized in a three-tier client/server design in which each
tier plays a specic role based on the individual roles of its components. In this section, these roles are widely explained.
Fig. 2 depicts the functioning of RecomMetz by means of a dataow-type diagram emphasizing the associations among components of the architecture. This diagram covers the rst ve phases
of the context-aware recommendation process outlined in Section 3 of this work as follows.
1. The active user requests a recommendation.
2. The user interface sends the users position and time information to the user proler.
3. The user proler retrieves the users preferences from the
User Database.
4. It builds a user prole starting from both the contextual
information and the users preferences.

1209

L.O. Colombo-Mendoza et al. / Expert Systems with Applications 42 (2015) 12021222

Google
Geocoding API

IMDB API

Time-aware
Service
Foursquare API

Domain
Database

Locaon-aware
Service

Informaon Collector

User
Database

Crowd-aware
Service

Context-aware Subsystem

Knowledgebased
Recommender

Domain
Ontology

User proler

User Interface

Google Movies
Showmes

User Check-in
Subsystem

Recommendaon Engine

Data Repository

Client
(Mobile device)

Server
Fig. 1. RecomMetzs architecture.

Theaters
List

Showmes
List

(ltered)

14.

12.
9.

Locaonaware Service
8.

User
prole

5.

15.
10.

Theaters
List
Movies

Simmil.
Matrix

(ltered)

List

13.
4.

User
Interface

1.

2.

User
Proler

16.

7.

Knowledge-based
Recommender

Time-aware
Service

3.

6.

User
Database

Domain
Database

Informaon
Collector
11.

Acve User

Google Distance
Matrix API
Fig. 2. Partial view of the functioning of RecomMetz.

5. The time-aware service retrieves the user prole.


6. It also retrieves all the movies and movie theaters from the
Domain Database.
7. It generates a list of movies currently in movie theaters (and
the corresponding list of movie theaters).
8. The Location-aware Service retrieves the previously generated lists.

9. It also retrieves the previously built user prole.


10. It calls the Information Collector to calculate the distances
between the users current position (origin) and each one
of the movie theaters (destinations) as well as the times
required to get to each one of the destinations.
11. The Information Collector sends a series of HTTP requests to
the Google Distance Matrix API.

1210

L.O. Colombo-Mendoza et al. / Expert Systems with Applications 42 (2015) 12021222

12. The Location-aware Service lters the list of movie theaters


previously generated by the time-aware service by applying
the formula (1).
13. It retrieves all the movie showtimes from the Domain
Database.
14. It generates a list of the movie showtimes that the user is
likely to attend by applying the formula (2).
15. The knowledge-based recommender retrieves the lists of
movies, movie theaters and showtimes.
16. It computes the semantic similarities between all the moviemovie theater-showtime triples resulting from the previous
ltering operations.
4.1. Data Repository
The Data Repository is responsible for enabling user data persistence. Among other information, user data include the preferences
(ratings) explicitly stated by users and the visiting histories
recorded by the User Check-in Subsystem (see Section 5.3 of this
paper). The Data Repository also acts as a back-end system for
the recommendation engine because it is responsible for replicating the in-memory computed similarities matrix. Because of the
nature of our work, the Data Repository is built on top of the
MySQL Relational DataBase Management System (RDBMS), the
worlds most popular free and open source RDBMS.1
Persisting and querying data is a key issue in ontology-based
systems because of the inefciency related to the manipulation
of le systems comprising Resource Description Framework
(RDF) and Web Ontology Language (OWL) les. RDF (Cyganiak,
Wood, & Lanthaler, 2014) is a framework for expressing information about resources in the Web. OWL (Bechofer et al., 2004) is a
semantic markup language for publishing and sharing ontologies
on the World Wide Web. As a solution to the aforementioned issue,
ontology repositories built on top of well-known RDBMSs (e.g.,
OWLDB, Sesame and KAON) have recently emerged because
RDBMSs represent a mature and efcient technology to deal with
large and persistent amounts of information.
Nevertheless, due to the lack of a standardized ontology repository providing support for different query and inferring languages
as well as a broad set of APIs for popular programming languages,
we have implemented an ontology management mechanism as
part of the Data Repository. This mechanism takes advantage of
the underlying RDBMS in the sense that it allows storing the ontologys structure and its instances as table records.
In detail, the core of the ontology management mechanism is
composed of: (1) a table for storing all the individuals of the ontology, (2) a table for representing associations between individuals
(object properties) as RDF triples (subject, predicate and object),
(3) a table for describing properties either object properties or
datatype properties and (4) one table for each datatype property
(for storing the values of all the individuals).
4.1.1. Domain ontology
An existing ontology maintained by the Department of Informatics at the University of Zurich was used as the basis for developing the domain ontology of this work.2 This ontology called
Movie Ontology (MO) is based on the Internet Movie Database
(IMDb) movie descriptions and domain knowledge so that it contains concept hierarchies for semantically describing movies. The
MO ontology is composed of different classes, including Movie,
Genre and Person; also, it comprises several inheritance hierarchies, e.g., the class Person is further divided into eight different
1
2

http://www.mysql.com/.
https://babbage.inf.unibz.it/trac/obdapublic/wiki/Example_MovieOntology#no1.

classes, including Actor, Actress and Film_Director. It was


dened using OWL. Therefore, the domain ontology in this work is
based on OWL.
Because the MO ontology does not include the vocabulary necessary to describe the concepts related to movie theaters and
showtimes, it was extended by means of the following classes,
object properties and datatypes. Similarly, the resources intended
to infer the crowd information of each active user in RecomMetz
were carefully added to the MO ontology.
 MovieTheater: it is a class that represents the information
about movie theaters, including not only the information
about movie theaters themselves (such as the name and
motionPictureTechnology datatypes) but also the information about the movie theaters locations. The latter includes a
textual description based on geographical data (the number, street, city, state and country datatypes) as
well as a description representing a point in geographical
coordinates, i.e., latitude and longitude (the lat and lng
datatypes). In detail, the motionPictureTechnology datatype allows classifying the movie theaters as 2D, 3D, 4DX
and IMAX-enhanced movie theaters. This classication can
be leveraged by the knowledge-based recommender subsystem to infer the likelihood that a user will like a movie theater by contrasting the motion picture systems available in
the movie theater with the movie formats (2D, 3D, 4DX
and IMAX) a user usually sees or likes to see at cinemas. In
addition, crowd information, i.e., the information about the
amount of people currently crowded at movie theaters (the
movieTheaterUserCounter datatype) is represented by this
class. As it can be inferred, the crowd information is treated
as real-time information rather than as historical data.
 Showtime: it is a class representing information about showtimes. On the one hand, this information includes the
hour, minute and period datatypes. On the other
hand, this information includes the aforementioned format datatype. As it can be inferred, the showtimes are represented as 12-h clock times in the domain ontology
because this is the convention used by the Googles Showtimes search service. Thereby, unnecessary time conversions
during ontology population are avoided. Additionally, the
Showtime class is the root class of an inheritance hierarchy
that comprises the equivalent classes: MorningShowtime,
LunchTimeShowtime, AfternoonShowtime, EveningShowtime and NightShowtime. These classes are leveraged by the knowledge-based recommender subsystem to
infer the likelihood that a user will like a movie by analyzing
the period of the day at which the movie is scheduled and
the period of the day the user usually attends or likes to
attend movie theaters. As in the case of the Showtime
class, another kind of crowd information, namely, the information about the amount of already served users, who are
currently attending the showtimes (the showtimeUserCounter datatype), is represented by this class.
 isShownAt: it is an object property that lists the times a
movie is shown in a movie theater.
 isShownIn: it is an object property that lists the movie theaters in which a movie is currently being shown. Actually,
this object property lists the showtimes currently at a movie
theater.
At some point, the above outlined changes (the isShownIn
and isShownAt object properties and the movieTheaterUserCounter datatype) support the real-time side of RecomMetz. On
the one hand, the isShownIn and isShownAt object properties
support the identication of movies that are currently in movie

L.O. Colombo-Mendoza et al. / Expert Systems with Applications 42 (2015) 12021222

theaters. On the other hand, the movieTheaterUserCounter datatype is as an indicator of the location of the already served users in
movie theaters; thereby, it is interpreted as the crowd information
of the active users in RecomMetz. Unlike the crowd information,
the location and time information of each active user in RecomMetz is modeled by means of his user prole as is pointed out in
Section 4.4.3 of this paper.

1211

Finally, as explained in Section 4.4.1.1 of this work, the Information Collector is also leveraged for other purposes not concerning
ontology population. In fact, the Information Collector acts as a
broker for the Google Distance Matrix API, which is used by the
Location-aware Service to calculate distances between origins
and destinations. Similarly, the Information Collector acts as a broker for the Foursquare Venues API, which is used by the User Checkin Subsystem to know the number of people at movie theaters in
real-time.

4.2. Information Collector


4.3. User Check-in Subsystem
The Information Collector is responsible for extracting not only
information about movies but also information about movie theaters and showtimes. This information is used to populate the
domain ontology, i.e., to initialize the ontology and keep it up-todate. Ontology population (OP) is a subprocess of Ontology-Based
Information Extraction (OBIE) and OBIE is a subeld of Information
Extraction (IE). Unlike OP, OBIE also groups several tasks related to
ontology construction (Ontology Learning). In fact, OP is the process
of extracting instances and property values with respect to classes
and properties for a given ontology (Wimalasuriya & Dou, 2010).
According to the above, two different data sources are used by
the information collector component in order to populate the
domain ontology as follows. The Internet Movie Database (IMDb),
which is the worlds most popular and authoritative source for
movie, TV and celebrity content,3 is used for retrieving information
about movies. Similarly, the Googles Movies Showtimes search service is used for retrieving information about both movie theaters
and movie showtimes. It is important to notice that this service does
not provide geocoding information about the location of movie theaters. Therefore, the Google Geocoding API, which is part of the Google
Maps Web Services suite, is used in this work for converting the geographic data provided by the Googles Movies Showtimes service into
geographic coordinates.
Unlike IMDb, the Googles Movies Showtimes search service does
not expose an Application Programming Interface (API). Thus, a different mechanism for extracting information from this service had
to be implemented. In fact, the information collector component
implements a semi-automatic mechanism for populating the
domain ontology from semi-structured documents (XML documents in the case of both IMDB and the Google Geocoding API,
and HTML documents in the case of the Googles Movies Showtimes
service).
In addition to the mechanism for adding new individuals to the
domain ontology, a complementary mechanism for deleting individuals from the ontology, or rather, for keeping the ontology upto-date was needed. In detail, the outdated individuals of the
Showtime class that are linked to individuals of the Movie class
via the isShownAt object property must be unlinked; similarly,
the corresponding individuals of the Movie class that are linked
to individuals of the MovieTheater class via the isShownIn
object property must be unlinked to avoid recommending movies
no longer in movie theaters (the real-time side of the ontology).
The Information Collector implements the mechanism outlined
above like a database trigger that executes each time a new
individual of the Showtime class is added to the domain ontology. In this approach, the individuals are not deleted but unlinked;
this is due to, movies no longer in movie theaters and movies still
in movie theaters may have properties in common, e.g., actors,
genres and lm locations. In fact, the properties about movies no
longer in movie theaters can be leveraged to provide recommendations related to movies still in movie theaters. As future work, this
capability will be integrated into the recommendation approach
proposed in this paper (see Section 6 of this paper).
3

http://www.imdb.com/.

The User Check-in Subsystem is in charge of keeping a record of


the movie theaters actually visited by each already served user of
RecomMetz. In detail, when a user arrives to a movie theater, the
movie theater, being or not part of the outcome of the recommendation process, is identied by the User Check-in Subsystem. This
task is performed by taking advantage of the Foursquare Venues
and Checkins APIs so that not only the positions of the RecomMetz
users but also the positions of other people being part of the crowds
at movie theaters are taken into account by the crowd-aware service in calculating crowds. The Venues API allows searching for
Foursquare venues near given geographical locations and to know
the number of people who are currently at those places. The Checkins API allows Foursquare users to check into given venues. In
detail, when a user arrives to a movie theater, the corresponding
Foursquare venue is searched for; then, once the venue is identied,
a new check-in is created at that venue. Finally, for crowd-awareness purposes, the value of the counter in the domain ontology that
represents the amount of people crowded at the movie theater
searched is updated according to the data retrieved by the Venues
API as part of the response for the Foursquare check-in request.
Similarly, the showtime (which is linked to a movie) to be
attended by the user is requested when the arrival at the movie
theater occurs so that a movie showtime, i.e., a triple composed
of a movie theater, a movie and a showtime is stored in the Data
Repository (User Data) as a movie showtime pending to be rated
as follows. The User Check-in Subsystem retrieves the list of movie
showtimes most likely to be attended by the then-active user,
which in the context of the internal functioning of RecomMetz, is
formerly computed by the time-aware service. This service lters
the list by movie theater in order to obtain the set of movie showtimes currently at the movie theater the already server user is
arriving. Subsequently, the list is ltered by the time (hour, minute
and period) at which the movies are scheduled in order to obtain
the set of showtimes still available at the movie theater, considering possible travel delays. The ltered list is presented to the user
so that he is able to indicate the specic showtime he is going to
attend when the arrival at the movie theater occurs. At this point
in the user check-in process, the User Check-in Subsystem is able
to identify the resultant movie showtime and mark it as a movie
showtime attended by the user but not yet rated. Finally, for
crowd-awareness purposes, the value of the counter that represents the amount of people that are going to attend the corresponding showtime is properly updated in the domain ontology.
At the same time, the User Check-in Subsystem might be used
as a feedback mechanism for the overall recommendation process
in the sense that it would allow contrasting the movie showtimes
recommended by RecomMetz with the movie showtimes actually
attended (selected) by the users. This kind of feedback does not
be confused with the feedback represented by user preferences
because the fact that a user attends one of the top-n movie showtimes recommended by RecomMetz does not ensure that he will
actually like it. From this perspective, in the best-case scenario, a
user rates a showtime once he is able to do it, i.e. a user can rate
a movie showtime only if he has attended it before. In order to

1212

L.O. Colombo-Mendoza et al. / Expert Systems with Applications 42 (2015) 12021222

ensure the above-outlined best-case scenario, RecomMetz enables


the user to rate only the movie showtimes previously registered by
means of the User Check-In Subsystem. See Section 4.4.3 of this
paper for details about how users can rate items into RecomMetz.
4.4. Recommendation engine
The RecomMetzs core is the recommendation engine, which is
an abstract layer that comprises all the phases of the contextaware recommendation process proposed in this research: prole
updating, context-related ltering operations, similarities computing, knowledge gathering, and recommendations computing.
4.4.1. Context-aware Subsystem
The RecomMetzs context-awareness capability is to some
extend supported by the Context-Aware Subsystem. This component is actually an abstract layer comprising the components that
implement the context-related ltering operations: locationrelated pre-ltering, location-related post-ltering, time-related
pre-ltering (phase 1 and phase 2) and crowd-related postltering.
4.4.1.1. Location-aware service. The calculation of the distance
between the active users current location and a movie theaters
location is a complex task because it must be based on determining
the optimal route for a given transport modality. Therefore, this
task cannot be directly addressed as the distance between two
coordinates but as a graph-theory problem. Beyond the low-level
approach related to the implementation of graph-theory algorithms, there are other high-level solutions. For instance, the Google Distance Matrix API of the Google Maps Web Services suite allows
calculating travel distance and time between locations based on
recommended routes by taking advantage of the Google Maps
engine. It supports several modes of transportation, including transit, driving, walking and cycling.
Because of the purposes of our work, the Google Distance Matrix
API is used by the Location-aware Service to compute the distances
between users current locations and movie theaters locations. As
it can be inferred, not only the location-related ltering (pre-ltering and post-ltering) but also the time-related pre-ltering
(phase 2) can be performed by using the Google Distance Matrix
API, as explained below. As a prerequisite, these tasks require
retrieving the locations of all the movie theaters by means of the
individuals of the MovieTheater class.
In the case of the location-related pre-ltering, the set of distances retrieved from the Google Distance Matrix API by the Information Collector is ltered by applying the formula (1). As a
prerequisite, this formula requires the computation of the frequencies of visits to the intended movie theaters. For this purpose, the
visiting history in the corresponding user prole must be analyzed.
The resulting set of frequencies is used as input for calculating quartiles so that four equal groups of data are obtained for the variable
frequency of visits. Finally, a decimal value representing a percentage
is assigned to each one of the values of the data sets according to the
group into which it is categorized. The values 1.0, 0.75, 0.5 and 0.25
are used for the quartiles 1 (q1), 2 (q2), 3 (q3) and 4 (q4), respectively. The values above a threshold experimentally set to 0.75 are
considered to be part of the outcome of the location-related preltering. In the case of the location-related post-ltering, the set
of distances resulting from the previous pre-ltering operation is
directly used as input for calculating quartiles so that four equal
groups of data are obtained for the variable location.
4.4.1.2. Time-aware service. The time-aware service represents the
entry point of the recommendation process in the sense that it is
responsible for identifying all the movies are currently in movie

theaters, i.e., it is responsible for discarding all the movies are currently not in movie theaters (time-related pre-ltering, phase 1). In
detail, the time-aware service operates at the domain ontology
level, and it is in charge of identifying all the individuals of the
Movie class that are actually related to at least one individual
of the MovieTheater class via the isShownIn object property.
Furthermore, at a higher level of abstraction, the time-aware
service is responsible for estimating the likelihood of movie showtimes attendance according to formula (2) (leeway). For this purpose, the time required to get to all the movie theaters (instances
of the MovieTheater class resulting from the previous pre-ltering operations) from the active users current location is rst calculated by sending a series of HTTP requests to the Google Distance
Matrix API. In fact, this task is actually performed by the Location-aware Service because both the distances and times between
locations can be retrieved in the same response from the Google
Distance Matrix API.
4.4.1.3. Crowd-aware Service. The crowd-related post-ltering
relies on the estimation of not only the amount of people already
crowded at the movie theater that is likely to be visited by the
active user but also the amount of people (already server users)
that are attending the same showtime that the active user is likely
to attend. This estimation is done by the Crowd-aware Service by
analyzing the people counters maintained by the User Check-in
Subsystem in the domain ontology. In detail, for each recommendation to be presented to the active user, i.e., for each result in
the output of the previous pre-ltering operation, a value representing the amount of people at the intended movie theater is
retrieved by analyzing the corresponding individual of the MovieTheater class whereas a value representing the amount of already
served users attending the intended showtime is retrieved by analyzing the corresponding individual of the Showtime class. Similarly, the amount of already served users attending each one of the
simultaneous showtimes at the same movie theater is retrieved by
analyzing all the individuals of the Showtime class that are
linked to the corresponding individual of the Movie Theater class
via the isShownAt object property.
For each intended showtime, a more realistic value representing
the amount of attendants is calculated as follows: (1) summarizing
the values representing the amount of already served users attending each one of the simultaneous showtimes, (2) subtracting the
resulting value to the value representing the amount of people at
the intended movie theater, (3) dividing the resulting value by
the amount of concurrent showtimes and (4) adding the resulting
value to the original value.
The resulting data set is used as input for calculating quartiles
as in the case of the location-related pre and post-ltering so that
four equal groups of data are obtained also for the variable crowd.
The values assigned at this stage along with the values assigned at
the location-related post-ltering are then used by the recommendation engine to compute the nal recommendations as explained
below.
4.4.2. Knowledge-based recommender
Unlike other recommendation approaches related to the movies
domain, our proposal is intended to recommend showtimes for the
movies currently in movie theaters instead of simply recommending movies as content. Therefore, there are not one but three
different reference classes in the domain ontology: Movie,
MovieTheater and Showtime so that the items to be recommended are triples composed of an individual of the Movie class,
an individual of the MovieTheater class and an individual of the
Showtime class at the same time.
In detail, the triples are recommended to users starting from
both the information available about user contexts (time, location

L.O. Colombo-Mendoza et al. / Expert Systems with Applications 42 (2015) 12021222

and crowd) and the preferences explicitly provided by users. In


fact, the knowledge-based recommender subsystem is responsible
for suggesting items semantically related to the ones liked by users
in the past. This denition relates to the semantic similarity measure introduced in Blanco-Fernndez et al. (2008), which is
opposed to the syntactic similarity measures used by purely
content-based recommendation approaches.
Furthermore, as it is clearly explained in the following
subsection of this paper, the preferences are represented by triples
of single ratings and each triple is composed according to one of
the following statements: (1) by integrating ratings for individuals
of the reference classes (one individual for each one of the three
reference classes), (2) by integrating ratings for individuals or data
values from other sources directly linked to individuals of the reference classes through object or datatype properties (one individual/data value for each one of the three reference classes) and (3)
by combining the aforementioned kinds of ratings. It is important
to notice that, in the second case, the object and datatype properties represent features of individuals of the reference classes. Here,
the corresponding reference classes are clearly identied. For
instance, the inverse object property of the belongsToGenre
object property of the Genre class as well as the isActorIn
object property of the Actor class (the domain of the properties)
represent features of the Movie reference class (the range of the
properties). This approach represents a primary step towards the
property-based recommendation approach outlined in LpezNores, Blanco-Fernndez, Pazos-Arias, and Gil-Solla (2012).
The core of the recommendation process is the computation of
the semantic similarities between all the triples existing in the recommendation space. As it can be inferred, a preliminary step that
consists in building these triples is required to this aim. Due to
all the possible combinations (n * n0 * n00 ), this step would seem a
resource-consuming task; however, it is not because of the features
inherent to the domain of this research only the movies currently
in movie theaters (showtimes) are considered in the recommendation space. In addition, the space is signicantly reduced by performing the context-related pre-ltering operations because only
the movie theaters near the user (location-related pre-ltering)
and the movie times the user is likely to attend (time-related
pre-ltering, phase 2) are considered.
The similarities are calculated by using the following formula
(3) and they are represented as a symmetric matrix of order n. In
this context, n is the number of resulting triples so that each element Xij of the matrix is a oat number representing the similarity
between the triples i and j.
This similarity metric have already been proposed in the recommender systems literature (Carrer-Neto, Hernndez-Alcaraz,
Valencia-Garca, & Garca-Snchez, 2012). However, the interpretation given to the metric in this work is markedly different because
of the kind of items to be recommended: packages of items as
stated by Xie, Lakshmanan, and Wood (2011).

Sab Similarity a; b

#P 
X
commona; b; Pi
 weightPi:

maxdega; Pi; degb; Pi


i1

composing the set (P[i]). deg(b, p) represents the number of triples


associated to the triple b through the three properties (at the same
time) composing the set (P[i]). common(a, b, p) represents the number
of triples associated to both the triples a and b through the set (P[i]).
weight(p) is the importance of the set (P[i]) in the recommendation.
Once the similarities are calculated, the recommendations can
be computed as follows. It is important to notice that this process
is repeated each time an already served user rates a triple. As a
result of this process, an array of order 10 is built. This array represents the recommendations to be made to an already served user
who has become an active user.
(1) If the triple is being analyzed is composed of individuals of
the reference classes of the domain ontology; then, the
knowledge-based
recommender
Subsystem
directly
searches for the top-10 most similar triples in the similarities matrix. In contrast, if the triple is being analyzed is composed of individuals/data values from any other source;
then, the knowledge-based recommender Subsystem
searches for all the triples composed of individuals of the
reference classes that are linked to the individuals/data values of the triple is being analyzed through the three underlying properties at the same time. As it can be inferred, in
this case the search is performed in the ontology. Finally, if
the triple is being analyzed is composed of both individuals
of the reference classes and individuals/ data values from
any other source; then, a hybrid method combining the
two methods outlined above is used as follows. First, for
the individuals/data values coming from sources different
from the reference classes, its related individuals of the reference classes are searched in the ontology so that new
equivalent triples are built by replacing the individuals/data
values rated by users by related individuals of the reference
classes. Second, for each equivalent triple, the top 10 most
similar triples in the similarities matrix are searched. As it
was explained above, calculating recommendations in the
case of hybrid triples requires a preliminary step intended
to make these triples compatible with the triples manipulated by the similarities matrix in order to make the system
able to provide accurate recommendations.
(2) In the rst case as well as in the last case, each value of the
array is calculated by accumulating the current value to the
value resulting from multiplying the corresponding similarity value by the value given by the user (rating) in the past to
the triple is being analyzed as depicted in formula (4). In the
second case, each value of the array is calculated by accumulating the current value to the value given by the user in the
past to the triple is being analyzed. As it can be inferred, the
rating for the triple is calculated as the average of the ratings
given to the single items composing the triple. Fig. 3 depicts
the process of calculating recommendations in the case of
hybrid triples.

recomi recomi Sab  rate:


3

where a and b are the arrays representing the triples (packages) of


individuals to be compared. P is a three-dimensional matrix containing sets of three property-reference class pairs. Here, a set can
contain both object and datatype properties and each property
must belong to one different reference class. The properties specied by means of the P parameter are the properties relevant for
the recommendation process, i.e., the properties to be considered
by the recommendation engine in computing the similarities
between triples. deg(a, p) represents the number of triples associated to the triple a through the three properties (at the same time)

1213

where a is a triple from the similarities matrix, b is the triplet


is being analyzed, Sab is the value representing the degree of
similarly between triples a and b, rate is the rating given by
the user to the triple b individually.
(3) The nal recommendation values are calculated by using the
following formula (5). At this stage, recommendations were
calculated solely based on similarities (formula (5)), i.e., by
ignoring any kind of contextual information; therefore, they
need to be contextualized according to the post-ltering
context-aware paradigm. For this purpose, the values
assigned, to the triples in the recommendation space as a
result of the calculation of quartiles for both, the location

1214

L.O. Colombo-Mendoza et al. / Expert Systems with Applications 42 (2015) 12021222

I like Thor,
I like CINESA,
I do not
20:00hrs
showme

I like Thor (0.5),


I like CINESA
(0.5),
I do not like night
showmes (-0.5)
Ontologyreasoning
(knowledgegathering)

CaptainAmerica, NEOCINE,
21:05hrs

Userprole

0.75

0.70

0.90

I like Thor,
I like CINESA,
I do not
20:05hrs
showme

Preferencesa
rray

*
Thor, CINESA, 20:00hrs

[0]

(0.5+0.5+(-0.5))/3
=0.166

=
0.1245

Recommendaons
compung

Similariesm
atrix
Fig. 3. Recommendations computing process.

and crowd variables (location-related post-ltering and


crowd-related post-ltering, respectively), are used as input
parameters in this formula.

final Recom recomi lQuai  lW mTCQuai


 mTCW:

where recom is the array containing the recommendation


values for the top-10 most similar triples (formula (4)). lQua
is an array containing the values that represent the quartiles
corresponding to the variable location for the top-10 most
similar triples. lW is the value that represents the weight
(importance) of the variable location compared with the variable crowd. mTCQua is an array containing the values that
represent the quartiles corresponding to the variable crowd
for the top-10 most similar triples. mTCW is the value that
represents the importance of the variable crowd compared
with the variable location.
Finally, towards the implementation, for further research, of a
feedback mechanism for the overall recommendation process,
the nal recommendations are stored by the knowledge-based recommender subsystem in the Data Repository as user data (see Section 6 of this paper).
4.4.3. User proler
The user proler is responsible for building a personal prole
for each RecomMetz user starting from both all the ratings given
by him to the items of his interest throughout the time as a user
and a minimum of information set up from the beginning. It is

important to notice that in this work, ratings represent the liking


of users for the movie showtimes recommended by RecomMetz.
Under this context, a user rates a movie showtime by giving one
rating for each single item composing it: a movie, a movie theater
and the showtime.
As it can be inferred, the user proling technique employed in
this work is the explicit user feedback. In general, there are two different techniques for collecting user interests and they are focused
on the interaction of the user with RecomMetz: (1) explicit feedback by means of ratings, comments or any other measure of interest directly given by the users to the items and/or their features
and (2) implicit feedback through observation of the user behavior
like, for example, the time spent on viewing items information or
the navigation histories, i.e., the order of the links visited during
users sessions (Resnick & Varian, 1997). It is important to notice
that this classication does not include the demographic-based
user modeling technique because it is closely related to another
kind of recommender system which has not been considered in
this research: demographic recommender system (Montaner,
Lpez, & De La Rosa, 2003) (see Section 2.1 of this paper).
In this work, a provisional user prole is built by the user proler each time a user account is created. The provisional preferences are replaced by real preferences as the user interacts with
the system to explicitly state his opinions about the items of his
interest; this process is known as user prole updating, and it is
triggered each time a user rates a triple of the system.
It is important to remark that user proles also store contextual
information, namely location and time information. This information is collected by the user interface when a user made a recommendation request by giving a rating for a triple of RecomMetz. In

L.O. Colombo-Mendoza et al. / Expert Systems with Applications 42 (2015) 12021222

fact, the user prole updating is the rst phase of the context-aware
recommendation process proposed in this work. From this perspective, along with the active users preferences, the current location of
the user as well as the time-related information, namely the hour,
minute and period at which the user made the recommendation
request are tracked by the rich user interface running in his mobile
device. They are sent to the user proler as input parameters for the
user prole updating process and, in general, for the recommendation process. Similarly, user proles also store visiting histories;
nevertheless, they do not need to be updated when new entries
in visiting histories are generated because they are updated when
the ratings for the triples related to the entries are given.
Provisional proles are built according to the responses given by
potential users to a series of questions presented during user
authentication process. These questions are intended to reveal the
factors affecting the decisions about what movie to watch, where
to watch it and when (what time) to watch it. In addition, specic
interests for each one of the factors addressed in the questionnaire
are required to users in order to provide recommendations as accurate as possible from the beginning. For instance, if a question about
decisive factors is: do you choose a movie to watch by the genre
that it belongs to?; then, the complementary question about specic preferences might be: do you like drama movies?. Obviously,
the complementary question would be selected by the user proler
only if an afrmative reply is given to the rst question. As it can be
inferred, these sample questions are related only to the Movie reference class of the domain ontology; however, there are also questions related to the remaining of the reference classes, i.e., the
MovieTheater and Showtime classes.
It is important to notice that the above outlined technique is
intended to alleviate the cold-start problem by propagating the
preferences explicitly stated by users either over a hierarchy of
classes in the ontology e.g. over all the instances of the NightShowtime subclass of the Showtime class or over a set of
instances fullling a SPARQUL query, e.g. over all the instances of
the Movie class where the related Genre is drama. The former
case is depicted in Fig. 3. At the same time, it allows the recommendation engine to determine the weights for the properties
(object and datatype properties in the domain ontology) to be
taken into account in computing the matrix of similarities. In fact,
the questions about decisive factors are closely related to properties of the domain ontology so that the weights of the properties
are practically determined by users through answering the questionnaire presented during user account creation. For instance,
the above outlined question about decisive factors (do you choose
a movie to watch by the genre that it belongs to?) is closely related
to the property belongsToGenre of the domain ontology.
Both, the responses to the questionnaire presented during user
account creation and the preferences explicitly stated by users as
them interact with the recommender system, are internally represented as values from the following 5-point rating scale: strongly
liked, liked, indifferent, disliked and strongly disliked. It is important to notice that, for usability purposes, the proposed evaluation
scale uses qualitative values as opposed to the quantitative values
used by most of the recommendation approaches related to our
work (see Section 2.2 of this paper). Indeed, these qualitative values are internally matched to quantitative (numerical) values in
order to enable RecomMetz to process ratings. The matching is
as follows: strongly liked = 1, liked = 0.5, indifferent = 0, disliked = 0.5 and strongly disliked = 1.
As part of the information to be established during user account
creation, there also are settings related to context-aware variables.
We have proposed a 5-point Likert-type scale to represent the
importance (the weights from the user perspective) for the context-aware variables involved in the post-ltering operations, i.e.,
the location and crowd variables. It is important to notice that this

1215

scale is supplementary in the sense that the sum of the values


given separately for each of these variables must be 1. Because of
the nature of the location and crowd variables, we assume that
one of them must be more important for each user than the other.
In fact, the proposed Likert-type scale allows users to properly
determine the importance of one of the two context-aware variables respect to the other. Specically, the scale is as follows: allimportant (1.0), important (0.75), partially important (0.5), not
nearly as important (0.25) and not important (0). Furthermore,
for specic purposes of crowd-awareness, the username and password of a Foursquare account are also required by RecomMetz at
this stage of user account creation. By means of this data, RecomMetz is able to automatically communicate with Foursquare (see
Section 4.3 of this paper).
Similarly, there are complementary settings also concerning to
the context-aware side of the recommender system. Specically,
the preferred transport modality to be taken into account in the
time-related pre-ltering. All these settings can be changed later
as the user interacts with the system to rate the items of his interest because they are part of the user proles built by the user
proler.
4.5. User interface
The user interface represents the entry point of the contextaware recommendation process implemented by means of the software architecture outlined here. It enables users to interact with
RecomMetz for two main purposes: (1) request recommendations
by giving feedback on preferences and (2) give feedback on recommendations by means of entries in movie theater visiting histories.
The user can interact with RecomMetz through virtually any mobile
device equipped with a GPS receiver and an Internet connection
either via cellular networks e.g. GSM and UMTS (3G and 4G) or
wireless LANs e.g. Wi-Fi. In fact, unlike most of the proposals on
mobile recommender systems (see Section 2.3 of this paper), our
proposal is intended to provide users with native experiences.
In this context, a new trend on mobile application development
regarding the use of cross-platform Web-based application frameworks has recently emerged. This approach allows developers to
take advantage of the hardware capabilities of mobile devices
without caring about the multiple underlying operating systems.
For instance, PhoneGap is a free and open source framework for
building native applications for iOS, Android, Blackberry, WebOS,
Windows Phone, Symbian and Bada-based mobile devices starting
from the same code base written in JavaScript, HTML5 and CSS3
technologies.4
According to the above, the user interface of RecomMetz was
implemented using PhoneGap 3.0 in order to support a broad
range of mobile operating systems. The interaction ow between
a user and RecomMetz describes two main usage scenarios: (1)
giving feedback on preferences and (2) giving feedback on recommendation. These usage scenarios are depicted in Fig. 4.
The usage scenarios depicted in Fig. 4 are cyclically executed;
similarly, the stages of each usage scenario are sequentially executed as follows. It is important to notice that, the rst three stages
compose the rst usage scenario whereas the remaining three
stages compose the second usage scenario.
1. Initially, a potential RecomMetz user provides the information required to set up a new RecomMetz user account,
namely his personal information as well as the responses
to a provisional preferences questionnaire.

http://phonegap.com/.

1216

L.O. Colombo-Mendoza et al. / Expert Systems with Applications 42 (2015) 12021222

3. Arrival at a movie theater nocaon

2. Recommendaons
compung

Preferences
feedback

1. Prole
establishment

5. Pending rang
nocaon

6. Rang

GPS sensor
System
me
Recommendaons
compung

Recommendaon
feedback

4. User check-in

Fig. 4. User-system interaction workow of RecomMetz.

Fig. 5. (a) Check-in at a movie theater, (b) rating a movie showtime triple.

2. As a result, an initial set of recommendations is automatically computed by RecomMetz starting from both the
provisional preferences and some contextual information
collected from the users mobile device (GPS sensor and
system time). As it can be inferred, the potential user

becomes an actual user at this point in the user-system


interaction ow, so that the user can select one of the
top-n movie showtimes (a triple composed of a movie
theater, a movie and a showtime) from the initial set of
recommendations.

L.O. Colombo-Mendoza et al. / Expert Systems with Applications 42 (2015) 12021222

3. RecomMetz goes into the waiting for the arrival at a movie


theater (Foursquare venue), so that a notication asking for
attendance conrmation is automatically triggered when
the arrival occurs.
4. The user conrms the arrival and indicates the movie and
showtime to be attended. This stage of the user-system
interaction ow is depicted in Fig. 5(a).
5. The resulting movie showtime is marked as pending to be
rated, and a notication asking for the rating is scheduled
to be triggered at the movie end time.
6. The user explicitly states a rating for each item composing
the resulting movie showtime. It is important to notice that,
these ratings represent the real preferences of the active
user, and they are intended to replace the provisional preferences previously provided by the then-potential user. At
this point in the user-system interaction ow, RecomMetz
is able to compute a new set of recommendations starting
from both the real preferences and the current contextual
information. This stage of the user-system interaction ow
is depicted in Fig. 5(b).

1217

selected a model for software quality from Software Engineering


literature, namely, ISO/IEC 25010:2011 Systems and software
Quality Requirements and Evaluation (SQuaRE). ISO/IEC
25010:2011 is an international standard for the evaluation of systems and software quality from two complementary perspectives:
quality in development and quality in use; so that it denes two
quality models composed of general characteristics of software
which are further decomposed into general sub-characteristics
and specic attributes. In detail, we selected the performance efciency characteristic, specically the time behavior sub-characteristic, from the quality in development model. In this context, time
behavior is formally dened as the degree to which the response
and processing times and throughput rates of a software product,
when performing its functions, meet requirements. The metric
we proposed to measure the time behavior quality factor is
described in Section 5.2 of this paper.
It is important to notice that thanks to the generic nature of the
ISO/IEC 25010:2011s quality models, they can be rened according
to the type of system or software to be evaluated; therefore, they
can be applied to the evaluation of native mobile applications.

5. Evaluation

5.1. Metrics

In order to measure the performance of RecomMetz, the precision, recall and f-measure metrics from the Information Retrieval
literature (Salton & McGill, 1986) were used. In detail, these
metrics are intended to evaluate the efcacy and effectiveness of
information retrieval systems, mainly focusing on the quality of
the retrieval output.
Precision, recall and f-measure metrics have traditionally been
preferred to other methods for evaluating information retrieval
systems (Raghavan, Bollmann, & Jung, 1989). Furthermore, they
have been more recently applied to recommender systems because
recommender systems are usually considered a particular case of
personalized information retrieval systems (Fang et al., 2012; Lee
et al., 2014; Ruotsalo et al., 2013). For instance, in (Ruotsalo
et al., 2013) a two-part evaluation method comprising a series of
laboratory experiments and a set of user trials was proposed in
order to evaluate both the ltering performance of a recommender
system and its usability in real-life scenarios, respectively. Ruotsalo et al. measured the performance of a recommender system as
the accuracy of its underlying computational algorithms, namely
ontology-based reasoning, query expansion and clustering) in
terms of precision and recall.
In detail, precision allows measuring the ability of a system to
retrieve as many relevant documents as possible in response to a
request whereas recall allows measuring the ability of a system
to retrieve as few non-relevant documents as possible in response
to a request. Here, a document is considered to be relevant if it is
judged by the user to be of interest; otherwise, it is considered to
be non-relevant. F-measure is the harmonic mean of precision
and recall.
In this research, precision and recall metrics were contextualized to the Recommender Systems eld, specically to the scope
of knowledge-based recommender systems in the sense that the
outcome of the system to be evaluated are not documents
(unstructured or semi-structured data) but semantically described
items (structured data) inferred from previous user experiences.
Likewise, as usual in personalized recommendation processes,
there is no a specic query as input. The contextualization of the
aforementioned metrics is described below.
In addition, a standard software metric intended to measure the
quality of the RecomMetzs client application in terms of response
time, i.e., in terms of performance, was used as an indicator of the
computational complexity of the context-aware recommendation
algorithm implemented by RecomMetz. For this purpose, we

In this research, precision is interpreted as the RecomMetzs


ability to recommend as many relevant items as possible. It can
be calculated by means of formula (6). Here, an item is considered
to be relevant if it is liked by the user.

Precision

Correctly recommended items


Total recommended items

where Correctly recommended items is the number of items classied as relevant by the user that are recommended by RecomMetz.
Total recommended items is the total number of items recommended
by RecomMetz.
Recall is interpreted as the RecomMetzs ability to recommend
as few non-relevant items as possible. It can be calculated by
means of formula (7). As it can be inferred, an item is considered
to be non-relevant if it is disliked by the user.

Recall

Correctly recommended items


Relev ant items

where Correctly recommended items has the same interpretation


given in the case of precision metric, Relevant items is the number
of items classied as relevant by the user.
Finally, F-measure can be calculated by means of the formula
(8):

F  measure

2  precision  recall
precision recall

In this research, time behavior is interpreted as the degree to


which the response times of a recommender system, when computing recommendations, meet requirements. In this context, a
metric called recommendation response time is proposed to measure
the RecomMetzs time behavior, which is dened as the average of
the wait time the user experiences after issuing a recommendation
request until the request is completed t. It can be calculated by
using the following formula (9).

Tmean
TXmean

P
Tmean

Ti
;
N

for i 1 to N

where TXmean is the required mean response time, Ti is the


response time for ith evaluation (shot), N is the number of evaluations (sample shots).

1218

L.O. Colombo-Mendoza et al. / Expert Systems with Applications 42 (2015) 12021222

5.2. Use case


Due to the subjective judgments involved in classifying items as
relevant or non-relevant, the evaluation was conducted as a user
trial in a real-life scenario. In detail, eight students of the University of Murcia (Murcia, Spain) were asked to interact with RecomMetz for precision, recall and f-measure measurement purposes. At
the same time, these interactions were automatically recorded by
using the BlazeMeters mobile application recording technology.
BlazeMeter is a cloud-based performance testing and monitoring
platform for Web and mobile applications. It allows testing native
mobile applications by locally recording the functioning of the
applications in the target mobile devices and then directly executing the generated scripts on the platform. It uses actual remote
mobile devices and simulated mobile networks such as 4G and 3G.
With the aim of involving the context-aware side of the system,
specically the location-aware feature, the sample (eight students)
was divided into two equals groups of students geographically distributed across the two campuses of the university. In detail, four
students from the campus Campus de La Merced: two from the
faculty of law and two from the faculty of letters were asked. Similarly, four students from the campus Campus de Espinardo: two
from the faculty of informatics, one from the faculty of mathematics and one more from the faculty of education were asked. Finally,
three testing groups were formed by combining students from
both campuses; they were randomly scheduled at different times
in a day in order to involve the time-aware feature of the system;
namely, the rst group (group A) was scheduled for Saturday
evening, the second group (group B) was scheduled for Saturday
afternoon and the third group (group C) was scheduled for Saturday night. It is important to notice that, the user trial was conducted on Saturday August 30, 2014 i.e., a day after Into the
Storm and El Nio premieres (Spain). Thereby, the crowd-aware
side of the system was also involved in this evaluation. For the purposes of time-awareness, the geographical locations of the aforementioned faculties were considered the origins of the travel
routes to the movie theaters to be recommended to users, i.e.,
the origins of the travel routes to the movie theaters in Murcia,
Spain. Thereby, these movie theaters were considered the corresponding destinations.
Each participant in the trial was equipped with a GPS-enabled
mobile device, either a smartphone or a tablet PC, including a
Nexus 7 tablet, an LG Optimus L5 II smartphone, an HP Slate 7 Plus
tablet and a Samsung Galaxy S III smartphone. In most cases, these
were the devices the participants owned. RecomMetz was accessed
via the universitys Wi-Fi wireless LAN network, which offers a
20 Mbps Internet connection. The same network conditions were
simulated on the BlazeMeter platform towards the execution of
the test scripts automatically recorded during the user trial using
the aforementioned mobile devices. As it can be inferred, mobile
network simulation on the BlazeMeter platform was not required
because it was not considered in this evaluation. Finally, with the
aim of executing the test scripts under usage scenarios that are
as realistic as possible, an execution environment was simulated
for each script with up to 8 concurrent users making simultaneous
recommendation requests to the server-side of RecomMetz during
30 min.
Regarding information about locations of origins and destinations, the campus Campus de La Merced is in downtown Murcia
whereas the campus Campus de Espinardo is about 8 km. from
downtown. In addition, the nearest movie theaters to the campus
Campus de La Merced are: Cine Rex and Cines Centrofama
whereas the nearest movie theaters to the campus Campus de
Espinardo are: NEOCINE El Tiro, NEOCINE Thader and
CINESA Nueva Condomina. In detail, the nearest movie theater
to the campus Campus de La Merced is Cine Rex, which is

about 300 m. and 5 min. walking from the campus whereas the farthest movie theater to the campus is CINESA-Nueva Condomina,
which is about 9 km. and 16 min. driving from the campus. Similarly, the nearest movie theater to the campus Campus de Espinardo is NEOCINE-El Tiro, which is about 700 m. and 9 min.
walking from the campus, specically from the faculty of education
whereas the farthest movie theater to the campus is CINESANueva Condomina, which is about 6 km. and 11 min. driving from
the campus, specically from the faculty of education. Although in
this scenario the inferring of the nearest and farthest movie theaters (destinations) to a campus do not depend on the faculty that
is considered as the origin of the route, the distances as well as the
times may vary because of the proximity of the faculties in a campus. For instance, in the case of the campus Campus de Espinardo, the two faculties farthest away from one another are the
faculty of informatics and the faculty of education, which are about
1.5 km. walking from one another.
5.3. Solution
At the beginning of the user test, the mobile devices were congured to effectively communicate with the BlazeMeters mobile
application recording technology; then, the participants were
asked to create their user accounts by interacting with RecomMetz
for the rst time. As a result, the corresponding provisional user
proles were generated by RecomMetz. In detail, the participants
were indirectly asked about movies, movie theaters and showtimes they potentially like or dislike thanks to the classication
capabilities of the ontology reasoning mechanism implemented
by the system. For instance, preferences related to motion picture
technologies, seating systems at movie theaters, comfort and luxury features of movie theaters, movie genres, actors/actresses,
awards and nominations of movies and showtime periods were
collected by means of the questionnaires presented to the participants. As it was explained in Section 4.4.3 of this paper, the questionnaires served for two purposes: on the one hand, they allowed
for inferring specic preferences such as the actors a user like; on
the other hand, they allowed identifying the factors that are relevant to the users in selecting movie showtimes; these factors were
then leveraged by RecomMetz in computing the similarities
between the movie showtimes in the recommendation space.
Once the user proles were generated, RecomMetz was able to
automatically obtain recommendations. For this purpose, a search
space composed of 83 movie showtimes was rst formed starting
from four movies in movie theaters: Into the Storm, Guardians
of the Galaxy, Lucy and El Nio, the ve movie theaters mentioned in the use case and a total of 65 showtimes. Here, the movies were randomly selected. In fact, each participant in the
evaluation was previously asked to classify each one of the movie
showtimes in the search space either as relevant or non-relevant to
his interests, i.e., either as a movie showtime he would like to
attend or a movie showtime he would not like to attend.
Finally, the recommendations generated by RecomMetz were
analyzed in order to identify, among other values, the values that
were used as input parameters for metrics described in Section 5.2
of this paper, namely how many of the movie showtimes in the
search space were presented (parameter total recommended items),
how many of the movie showtimes classied as relevant (parameter relevant items) and non-relevant by the participants were presented as well as the hits of RecomMetz in classifying the movie
showtimes in the search space as relevant (parameter correctly recommended items) and non-relevant according to the classication
made by the user. This last parameter was determined by comparing the user and system classications. Once the values were
obtained, they were substituted in formulas (6) and (7). Then,
the results from formulas (6) and (7) were used as input

1219

L.O. Colombo-Mendoza et al. / Expert Systems with Applications 42 (2015) 12021222

parameters for formula (8). Thereby, the precision, recall and fmeasure of the system were determined. The analysis of the results
of these measures is presented in the following section of this
paper.
It is important to notice that, the test was repeated once with
the aim of depicting the RecomMetzs performance under normal
circumstances, i.e., under a no cold-start scenario. In detail, each
participant in the evaluation was asked to select and rate one of
the movie showtimes recommended by RecomMetz. For this purpose, the geographical positions of the participants were statically
set in order to simulate the arrivals to the corresponding movie
theaters and allow checking into them. This eliminated the need
of actually visiting movie theaters during the user test.
5.4. Results
Table 5 depicts the results obtained from applying formulas
(6)(8) to the recommendations generated by RecomMetz along
the two executions of the test for each participant. The input
parameters used in each case are also depicted. In detail, column
Relevant-User represent the parameter relevant items whereas
column Non-Relevant-User represent the opposite, i.e., the
amounts of movie showtimes in the search space classied as
non-relevant by the participants. Column Relevant-System represents the parameter correctly recommended items whereas column Non-Relevant-System represents the opposite, i.e., the
amounts of hits of RecomMetz in classifying the movie showtimes
in the search space as non-relevant according to the classications
made by the participants. Finally, column Recommended represents the parameter total recommended items.
As it can be inferred from the results of the rst execution of the
test (cold-start scenario), RecomMetz was able to correctly recommend 7 out of 8 movie showtimes in the best-case scenario, which
is equivalent to a precision value of 88% (for user G). On the
opposite, RecomMetz was able to correctly recommend 5 out of
7 movie showtimes in the worst-case scenario, which represents
a precision value of 71% (for user A). Hence, the average precision
value of RecomMetz in the cold-start scenario was 81%. In fact, the
precision rate of RecomMetz reached more than 80% in most cases
(5 out of 8 users).
Similarly, RecomMetz was able to correctly recommend 6 out of
7 movie showtimes labeled as relevant by the user in the best-case

scenario during the rst execution of the test (cold-start scenario)


which is equivalent to a recall value of 86% (for user C). On the
opposite, RecomMetz was able to correctly recommend 5 out of
9 movie showtimes labeled as relevant by the user in the worstcase scenario, which represents a recall value of 56% (for users
A and E). Hence, the average recall value of RecomMetz in
the cold-start scenario was 68%. In fact, the recall rate of RecomMetz reached 62% or more in most cases (5 out of 8 users).
It is important to notice that the above-mentioned worst-case
scenarios coincided with the setting of a weight value for the location variable greater than for the crowd variable. This is due to the
fact that in a cold-start scenario, there are no entries in movie theater visiting histories; therefore, location-related pre-ltering and
post-ltering are based only on calculating the distances between
users positions and movie theaters locations; this may be inaccurate in many cases.
Although more items were recommended by RecomMetz (from
8 to 9 movie showtimes average) along the second execution of the
test (no cold-start scenario), the trend was the slight improvement
of the precision measure rates (from 81% to 83% average). This
proved the efciency and effectiveness of the mechanism implemented by the user proler towards alleviating the cold-start problem. In addition, the worst-case scenario for measuring precision
no longer coincided with the setting of a high weight value for
the location variable. This is due to the fact that entries in movie
theater visiting histories were generated during check-in simulations so that location-related pre-ltering and post-ltering were
based on both distances between users positions and movie theaters locations and frequencies of visits to movie theaters. In fact,
the user assigned equal weight values for location and crowd variables in this case (user F). The averages of the precision, recall
and f-measure measurement results from the two executions of
the test are compared in Fig. 6.
Table 6 summarizes the trafc data obtained from the load test
automated by using the BlazeMeter workbench as well as the
results obtained from partially applying formula (9) to these data.
Under this perspective, column Average represents the parameter
Tmean for the eight shots of each of the two evaluation scenarios.
Hence, the parameter N is set to eight in both cases.
For the evaluation process of the recommendation response
time of RecomMetz, we experimentally set the parameter TXmean
to a half second (500 ms.) for the two evaluation scenarios equally

Table 5
Results of precision, recall and f-measure measurement.
Group

User

A
B
C

D
E
F

G
H

Average

Execution

Relevant

Non-relevant

Recommended

Precision

Recall

F-measure

69
67
71
71
74
73

7
9
6
8
7
9

0.714285714
0.888888889
0.833333333
0.75
0.857142857
0.888888889

0.555555556
0.888888889
0.625
0.75
0.857142857
0.888888889

0.625
0.888888889
0.714285714
0.75
0.857142857
0.888888889

78
76
74
74
77
74

76
74
73
71
72
72

5
8
6
7
5
9

0.8
0.75
0.833333333
0.857142857
0.8
0.777777778

0.8
0.857142857
0.555555556
0.666666667
0.666666667
0.777777778

0.8
0.8
0.666666667
0.75
0.727272727
0.777777778

7
6
4
5

74
75
76
75

71
73
71
71

8
7
5
6

0.875
0.857142857
0.8
0.833333333

0.777777778
0.75
0.571428571
0.625

0.823529412
0.8
0.666666667
0.714285714

5
6.5

75.5
74.625

72.125
71.5

6.125
7.875

0.814136905 (81%)
0.825396825 (83%)

0.676140873 (68%)
0.775545635 (78%)

0.735070505 (74%)
0.796230159 (80%)

User

System

User

System

1
2
1
2
1
2

9
9
8
8
7
9

5
8
5
6
6
8

74
74
75
75
76
74

1
2
1
2
1
2

5
7
9
9
6
9

4
6
5
6
4
7

1
2
1
2

9
8
7
8

1
2

7.5
8.375

1220

L.O. Colombo-Mendoza et al. / Expert Systems with Applications 42 (2015) 12021222

Table 6
Results from the load test automated by using the BlazeMeter workbench.
Group

User

Execution

B
C
B

D
E
F

G
H

Average (tmean rounded to units)

Ti (milliseconds)

1
2
1
2
1
2

610
605
605
600
595
595

1
2
1
2
1
2

600
595
650
635
590
580

1
2
1
2

635
630
615
615

1 (N = 8)
2 (N = 8)

615
610

90
80
70
60
50

Execuon 1

40

Execuon 2

30
20
10
0
Precision

Recall

F-measure

Fig. 6. Comparison of averages of precision, recall and f-measure measurement


results from the two executions of the test.

because the response time of a recommender system ideally


should not be conditioned by the availability of information about
user preferences. Hence, the recommendation response time values of RecomMetz were 1.23 and 1.22 in the rst (cold-start scenario) and second (no cold-start scenario) executions of the test,
respectively. Here, the nearer to 1.0 and less than 1.0 is the better.
As it can be inferred from the results summarized in Table 6, the
response time rates remained practically stable along the two executions of the test; in detail, they go from 595 (minimum) to
650 ms. (maximum). In fact, the average response time values of
the rst and second executions of the test were 615 and 610 ms.,
respectively. In general, there was a very slight decrease in the wait
time the users experienced between the two executions of the test.
As in the case of the performance evaluation in terms of precision,
recall and f-measure, the accurateness of the results of the rst
execution of the test prove the efciency and effectiveness of the
mechanism implemented by the user proler towards alleviating
the cold-start problem.

6. Conclusion and future work


In this paper, a context-aware recommendation approach comprising three kinds of contextual information: location, time and
crowd was proposed. This approach is aimed at recommending

movies as scheduled activities (as showtimes at movie theaters)


rather than as content so that items in search space are not single
items but composed items. This approach is focused on a modality
of leisure activity that is not directly supported by tourists as
opposed to most of the contributions in the leisure domain, including visiting theme parks, visiting museums, eating and shopping.
Contextual information may be useful for personalized recommendation, particularly for recommendation of scheduled activities. In fact, scheduled activities are usually subject not only to
time constraints but also to other physical context constraints such
as location and weather. The concept of crowd was introduced in
this paper as the amount of already served users that are crowded
around the movie showtimes to be of interest to the active user. In
detail, crowds at movie theaters are monitored in real-time by taking advantage of social networks, specically Foursquare, which is a
well-known location-based social network. Similarly, the time
dimension of the context model proposed in this paper was
addressed as a travel route recommendation problem minimizing
the amount of time required by users to get to the intended movie
theaters from their current positions in order to attend the corresponding movie showtimes. For this purpose, the Google Maps
engine was used as a geocoding and distance matrix platform. In
fact, most of the physical context constraints typically involved in
the recommendation of scheduled activities can be tracked by
means of sensors of modern mobile devices. Starting from this
asseveration, the concept of geo-recommendation as the locationbased personalized recommendation of places of possible interest
to a user was introduced in this paper by converging concepts from
mobile computing and recommender systems literature.
The context-aware recommendation approach proposed in this
research was implemented by means of a mobile clientserver
architecture leveraging the hardware capabilities, specically the
GPS sensors of mobile devices intended to run the presentation
tier. The study of other emerging geo-localization technologies
not included in the current state-of-the-art analysis because of
their novelty is considered as a future line of research towards
the implementation of a resource-saving more accurate locationaware solution. The resultant recommender system called RecomMetz was validated in the domain of the movies currently in movie
theaters in Murcia (Murcia city), Spain with acceptable results.
RecomMetz was tested on different Android-based mobile devices
although the presentation tier of the system was developed as a
multi-platform native user interface. Therefore, an additional set
of user trials considering real usage scenarios under different computational conditions is being currently designed. Similarly, real
usage scenarios involving different geographical conditions are
also being considered.
Furthermore, the proposed context-aware recommendation
approach is based on Semantic Web technologies in the sense that
a domain ontology is leveraged with the aim of identifying correlations between existing movie showtimes and inferences about
users preferences. A semantic similarity metric already proposed
in the knowledge-based recommender systems literature was
adjusted to the concept of packages of single items for the aforementioned purpose. In this context, the movie showtimes are recommended as packages composed of three single items: a movie, a
movie theater and a showtime so that they are called triples for the
case of this research. Inferences about user preferences are based
on propagating ratings explicitly given by users over the hierarchies of classes in the ontology so that the cold-start problem is
alleviated as it was demonstrated by means of the results of the
evaluation presented in this paper.
In addition, the domain ontology is also used for context modeling purposes. In fact, unlike other researches on context-aware
systems, in RecomMetz the user context is modeled as a two-part
model comprising individual context and group context. Here,

L.O. Colombo-Mendoza et al. / Expert Systems with Applications 42 (2015) 12021222

group context corresponds to crowd information, which is modeled in the domain ontology. Individual context corresponds to
the other kinds of information comprising context, i.e., location
and time, and it is modeled in each user prole. It is important
to notice that, in this research user proles are not modeled using
ontologies as it has been proposed in recent contributions on
knowledge-based recommender systems. Nevertheless, the performance implications of an ontology-based user prole approach are
currently being studied.
We are currently working on the implementation of a feedback
mechanism for the recommendation process that allows adjusting
the semantic similarity metric towards the improvement of the
RecomMetzs performance. From this perspective, the possibility
of rating items not only as a whole but also by means of a specic
feature is being studied as a line of research for this purpose.
Finally, the implementation of a complementary mechanism for
gathering implicit feedback on user preferences is currently being
studied as another line of research for the aforementioned purpose.
In addition, we are currently working on the implementation of a
complementary mechanism for modeling further domain knowledge about movie theaters besides showtimes and lms towards
the computation of more accurate crowd-aware recommendations.
In detail, this mechanism is intended to gather the information
about movie theaters that is not available online, namely, number
of rooms and seating capacities directly from movie theater workers viewed as stakeholders in the information gathering process
implemented by RecomMetz.

Acknowledgments
This work has been supported by the Spanish Ministry of Economy and Competitiveness and the European Commission (FEDER/
ERDF) through project SeCloud (TIN2010-18650). Luis Omar
Colombo-Mendoza is supported by the National Council of Science
and Technology (CONACYT) and the Public Education Secretary
(SEP). In addition, this work was supported by the General Council
of Superior Technological Education of Mexico (DGEST), National
Council of Science and Technology (CONACYT) and the Public Education Secretary (SEP) through PROMEP.

References
Adomavicius, G., & Tuzhilin, A. (2008). Context-aware recommender systems. In
Proceedings of the 2008 ACM conference on recommender systems (pp. 335336).
New York, NY, USA: ACM.
Batet, M., Moreno, A., Snchez, D., Isern, D., & Valls, A. (2012). Turist@: Agent-based
personalised recommendation of tourist activities. Expert Systems with
Applications, 39(8), 73197329. http://dx.doi.org/10.1016/j.eswa.2012.01.086.
Bechofer, S., van Harmelen, F., Hendler, J., Horrocks, I., McGuinness, D. L., PatelSchneider, P. F., et al. (2004). OWL web ontology language reference. Retrieved
from <http://www.w3.org/TR/owl-ref/>.
Berners-Lee, T., Hendler, J., & Lassila, O. (2001). The semantic web. Scientic
American, 284, 3443.
Blanco-Fernndez, Y., Lpez-Nores, M., Pazos-Arias, J. J., & Garca-Duque, J. (2011).
An improvement for semantics-based recommender systems grounded on
attaching temporal information to ontologies and user proles. Engineering
Applications of Articial Intelligence, 24(8), 13851397. http://dx.doi.org/
10.1016/j.engappai.2011.02.020.
Blanco-Fernndez, Y., Pazos-Arias, J. J., Gil-Solla, A., Ramos-Cabrer, M., Lpez-Nores,
M., Garca-Duque, J., et al. (2008). A exible semantic inference methodology to
reason about user preferences in knowledge-based recommender systems.
Knowledge-Based
Systems,
21(4),
305320.
http://dx.doi.org/10.1016/
j.knosys.2007.07.004.
Carrer-Neto, W., Hernndez-Alcaraz, M. L., Valencia-Garca, R., & Garca-Snchez, F.
(2012). Social knowledge-based recommender system. Application to the
movies domain. Expert Systems with Applications, 39(12), 1099011000. http://
dx.doi.org/10.1016/j.eswa.2012.03.025.
Carretero, J., Isaila, F., Kermarrec, A.-M., Taiani, F., & Tirado, J. M. (2012). Geology:
Modular georecommendation in gossip-based social networks. In 2012 IEEE
32nd international conference on distributed computing systems (ICDCS) (pp. 637
646). http://dx.doi.org/10.1109/ICDCS.2012.36.

1221

Cyganiak, R., Wood, D., & Lanthaler, M. (2014). RDF 1.1 concepts and abstract
syntax. Retrieved from <http://www.w3.org/TR/2014/REC-rdf11-concepts20140225/>.
Dao, T. H., Jeong, S. R., & Ahn, H. (2012). A novel recommendation model of locationbased advertising: Context-aware collaborative ltering using GA approach.
Expert Systems with Applications, 39(3), 37313739. http://dx.doi.org/10.1016/
j.eswa.2011.09.070.
Fang, B., Liao, S., Xu, K., Cheng, H., Zhu, C., & Chen, H. (2012). A novel mobile
recommender system for indoor shopping. Expert Systems with Applications,
39(15), 1199212000. http://dx.doi.org/10.1016/j.eswa.2012.03.038.
Forman, G. H., & Zahorjan, J. (1994). The challenges of mobile computing. IEEE
Computer, 27, 3847.
Goldberg, D., Nichols, D., Oki, B. M., & Terry, D. (1992). Using collaborative ltering
to weave an information tapestry. Communications of the ACM, 35(12), 6170.
http://dx.doi.org/10.1145/138859.138867.
Gruber, T. R. (1993). A translation approach to portable ontology specications.
Knowledge Acquisition, 5, 199220.
Gu, T., Wang, X. H., Pung, H. K., & Zhang, D. Q. (2004). An ontology-based context
model in intelligent environments. In Proceedings of communication networks
and distributed systems modeling and simulation conference (pp. 270275).
Jung, J. J. (2009). Contextualized mobile recommendation service based on
interactive social network discovered from mobile users. Expert Systems with
Applications,
36(9),
1195011956.
http://dx.doi.org/10.1016/
j.eswa.2009.03.067.
Lee, W.-P., Kaoli, C., & Huang, J.-Y. (2014). A smart TV system with body-gesture
control, tag-based rating and context-aware recommendation. Knowledge-Based
Systems, 56, 167178. http://dx.doi.org/10.1016/j.knosys.2013.11.007.
Liu, L., Xu, J., Liao, S. S., & Chen, H. (2014). A real-time personalized route
recommendation system for self-drive tourists based on vehicle to vehicle
communication. Expert Systems with Applications, 41(7), 34093417. http://
dx.doi.org/10.1016/j.eswa.2013.11.035.
Lpez-Nores, M., Blanco-Fernndez, Y., Pazos-Arias, J. J., & Gil-Solla, A. (2012).
Property-based collaborative ltering for health-aware recommender systems.
Expert Systems with Applications, 39(8), 74517457. http://dx.doi.org/10.1016/
j.eswa.2012.01.112.
Montaner, M., Lpez, B., & De La Rosa, J. L. (2003). A taxonomy of recommender
agents on the internet. Articial Intelligence Review, 19(4), 285330. http://
dx.doi.org/10.1023/A:1022850703159.
Moreno, A., Valls, A., Isern, D., Marin, L., & Borrs, J. (2013). SigTur/E-destination:
ontology-based personalized recommendation of tourism and leisure activities.
Engineering Applications of Articial Intelligence, 26(1), 633651. http://
dx.doi.org/10.1016/j.engappai.2012.02.014.
Noguera, J. M., Barranco, M. J., Segura, R. J., & Martnez, L. (2012). A mobile 3D-GIS
hybrid recommender system for tourism. Information Sciences, 215, 3752.
http://dx.doi.org/10.1016/j.ins.2012.05.010.
Raghavan, V., Bollmann, P., & Jung, G. S. (1989). A critical investigation of recall and
precision as measures of retrieval system performance. ACM Transactions on
Information Systems, 7(3), 205229. http://dx.doi.org/10.1145/65943.65945.
Resnick, P., & Varian, H. R. (1997). Recommender systems. Communications of the
ACM, 40(3), 5658. http://dx.doi.org/10.1145/245108.245121.
Ricci, F., Rokach, L., & Shapira, B. (2011). Introduction to recommender systems
handbook. In F. Ricci, L. Rokach, B. Shapira, & P. B. Kantor (Eds.), Recommender
systems handbook (pp. 135). US: Springer. Retrieved from http://
link.springer.com/chapter/10.1007/978-0-387-85820-3_1.
Ruotsalo, T., Haav, K., Stoyanov, A., Roche, S., Fani, E., Deliai, R., et al. (2013).
SMARTMUSEUM: A mobile recommender system for the web of data. Web
Semantics: Science, Services and Agents on the World Wide Web, 20, 5067. http://
dx.doi.org/10.1016/j.websem.2013.03.001.
Salton, G., & McGill, M. J. (1986). Introduction to modern information retrieval. New
York, NY, USA: McGraw-Hill Inc.
Schilit, B., Adams, N., & Want, R. (1994). Context-aware computing applications. In
Proceedings of workshop on mobile computing systems and applications (pp. 85
90). http://dx.doi.org/10.1109/MCSA.1994.512740.
Tsai, C.-Y., & Chung, S.-H. (2012). A personalized route recommendation service for
theme parks using RFID information and tourist behavior. Decision Support
Systems, 52(2), 514527. http://dx.doi.org/10.1016/j.dss.2011.10.013.
Turner, R. (2013). Travel & tourism economic impact 2013 World (No. 2013).
London, UK: World Travel & Tourism Council. Retrieved from <http://
www.wttc.org/site_media/uploads/downloads/world2013_1.pdf>.
Wang, S.-L., & Wu, C.-Y. (2011). Application of context-aware and personalized
recommendation to implement an adaptive ubiquitous learning system. Expert
Systems with Applications, 38(9), 1083110838. http://dx.doi.org/10.1016/
j.eswa.2011.02.083.
Wimalasuriya, D. C., & Dou, D. (2010). Ontology-based information extraction: An
introduction and a survey of current approaches. Journal of Information Science,
36(3), 306323. http://dx.doi.org/10.1177/0165551509360123.
Xie, M., Lakshmanan, L. V. S., & Wood, P. T. (2011). CompRec-Trip: A composite
recommendation system for travel planning. In 2011 IEEE 27th international
conference on data engineering (ICDE) (pp. 13521355). http://dx.doi.org/
10.1109/ICDE.2011.5767954.
Yang, W.-S., Cheng, H.-C., & Dia, J.-B. (2008). A location-aware recommender system
for mobile shopping environments. Expert Systems with Applications, 34(1),
437445. http://dx.doi.org/10.1016/j.eswa.2006.09.033.
Yang, W.-S., & Hwang, S.-Y. (2013). ITravel: A recommender system in mobile peerto-peer environment. Journal of Systems and Software, 86(1), 1220. http://
dx.doi.org/10.1016/j.jss.2012.06.041.

1222

L.O. Colombo-Mendoza et al. / Expert Systems with Applications 42 (2015) 12021222

Ylmaz, ., & Erdur, R. C. (2012). IConAwa An intelligent context-aware system.


Expert Systems with Applications, 39(3), 29072918. http://dx.doi.org/10.1016/
j.eswa.2011.08.152.
Yu, Y., Kim, J., Shin, K., & Jo, G. S. (2009). Recommendation system using locationbased ontology on wireless internet: An example of collective intelligence by

using mashup applications. Expert Systems with Applications, 36(9),


1167511681. http://dx.doi.org/10.1016/j.eswa.2009.03.017.
Zheng, V. W., Zheng, Y., Xie, X., & Yang, Q. (2012). Towards mobile intelligence:
Learning from GPS history data for collaborative recommendation. Articial
Intelligence, 184185, 1737. http://dx.doi.org/10.1016/j.artint.2012.02.002.

También podría gustarte