View with viewers

Priyanka Sinha
Innovation Labs Kolkata Tata Consultancy Services Limited 0091-8981335795

Kingshuk Chakravarty
Innovation Labs Kolkata Tata Consultancy Services Limited

kinghsuk.chakravarty@tcs.com
possible [3] and in identified movies, scene level tagging is also possible [6]. The rest of the paper is organized as follows. The next section illustrates using examples and pseudocodes how in different ways people may collaboratively view pervasively and how these can be implemented before concluding the paper. 2. PROPOSED METHODOLOGY In the proposed methodology we make assumptions: 1. 2. 3. 4. Linux is the operating system. The application is written in Qt. Phonon media player is used to view the video The device used to view the video is connected to the internet. This gives access to google search by image and video information YFrog [8] is used to upload screenshots of video Each screenshot is saved in JPEG format. IMDB is used to fetch information about the video. User (viewer) has agreed to share his/her viewing information via the public profile on Facebook [9] Google search by image [2] is used to determine video title. the following

priyanka27.s@tcs.com ABSTRACT
This paper explores collaboration amongst viewers of the same movie or tv show. It uses existing online social networks such as Facebook to like and rate scenes automatically. Upcoming interesting scenes can then be provided for the same movie. It augments video metadata with the use of IMDB like services to obtain actor, genre and rating. Thus this is a new manner of recommendation of scenes and ability to search videos based upon scenes.

Categories and Subject Descriptors

H.4.m [Information Systems]: Applications Miscellaneous. Information Systems

General Terms
Algorithms, Human Factors

Keywords
Online social networks; movie; tv show; tagging; scenes;

5. 6. 7. 8.

1.

INTRODUCTION

Movies and TV shows are consumed in many ways. Movies are watched in movie halls with a social group. TV shows are watched together in a social group and commented upon. Videos are streamed from sites such as Youtube, Netflix. They are watched on DVDs; on time-shifted DVRs; VCRs, etc. People within a social circle may influence what each of them watches. Friends viewing a video before us may function as early critics. From online social networks we can determine who our friends are and who among them have the same taste in videos as us. The videos they watch and like then become recommendations for us. There has been work on finding what your friends are watching now in [4, 5]. This allows collaborative viewing of videos. It need not require simultaneous viewing of the content by all group members. Later viewers may benefit from the viewing data of previous viewers. Collaborative subtitling is an example [7]. Recommendations may also come from rating services like IMDB [1], Rotten Tomatoes. If your friends like many scenes from a movie, that movie can be recommended to you along with the scene information. Video identification is

9.

In a pervasive application of viewing videos, video being watched would need to be identified by its title. Presently to gather more information about the video, the viewer resorts to either manually searching information using google search of looking up movie databases such as IMDB; or asking friends on Facebook; or looking up Wikipedia page of the video title. Asking friends on Facebook is a form of crowdsourcing. All of these techniques are non-linear to the video viewing experience, i.e., the viewer has to pull information manually from various sources. Proprietary solutions exist where the viewing videos title and related information is automatically identified [3]. As we consume more and more video, retrieving information related to what we are watching has become more important. Today, we like expressing what videos we like and share them on Facebook. Gathering and sharing information about the videos guides to more videos. Our solution proposes to mash up all this information for viewers convenience.

2.1
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Crowdsearch12, April 17, 2012, Lyon, France. Copyright 2012 ACM 1-58113-000-0/00/0010$10.00.

Workflow

In our application, typically, a video is being viewed on a Phonon media player on a connected device. A screenshot of the video can be saved in JPEG format and uploaded to YFrog periodically without user intervention. Once uploaded, google search by image is used on those YFrog images. Using the HTML response, IMDB is searched. Video title can be can be confirmed by repeated search in google, using different screenshots of the same video. IMDB returns for this title,

information such as actor names, genre, rating, director, plot, keywords, etc. Hereafter, this information can be referred to as the metadata of the video being watched. We use Facebook[9], to share and indicate our liking of what we do in our daily life. We would also like to share details of the content that we watch. Depending upon user permission, the video being watched can be shared on viewers Facebook wall. A particular scene that is liked by the viewer can be shared. For that, now we are uploading a screenshot which specifically contains the scene the viewer likes the most in viewers Facebook profile. As friends also upload their favorite scenes, an application with suitable permissions to Facebook accounts can now rate the scenes within the piece of video content. This application can create a list of liked scenes alongwith names of who all like that scene for a given video. This data can be used to rate scenes and get statistics such as which scenes have the viewer and his friends liked the most? It gives real time rating of scene level information about videos. This list with timestamps of the scenes can also be downloaded and used like a subtitle file to suggest upcoming good scenes to look out for. Information can be gathered by keeping count of how many times a title is being watched by a person. If the same title is being watched multiple times and is above a predefined threshold, the user is prompted to make it a favorite in Facebook.

with application name, time passed by in the video and a link to the screenshot uploaded to YFrog. A pseudocode of this is as follows: 1. 2. Upload to YFrog the scene liked and save mediaurl. Using Facebook application API key, login into current viewers account, allow suitable permissions such as publish_stream. Post onto feed message of form FBF || <timestamp> || <video title> and picture link mediaurl.

3.

Figure 1 Architecture diagram

2.2

Simulation

A movie is being viewed in phonon player in linux connected to the internet. Video frames are saved in JPEG format with 596x246 resolution. Then these images are uploaded to YFrog periodically without user intervention. Google search by image works with hosted URLs, thus, each screenshot is hosted on the YFrog and once uploaded, the corresponding URL is used. We expect this search to return the title within first 5 minutes of the video. To remove the possibility of false matches, we only consider matches that have been output 10 times by the google search engine. From the HTML response, IMDB links are searched for. Within those links, the title is searched for. Using the title, IMDB is queried for the following metadata: 1. 2. 3. 4. 5. 6. 7. 1. 2. 3. Title Genre Plot Actors Director Rating Keywords Save screenshot of video with known filename Use CURL [10] with Twitter username, password, filename and developer key to upload to YFrog. Use the mediaurl thumbnail and use CURL to google search by image (set user agent to Firefox). Save response to HTML file and parse for imdb.com/tt* Get title number; use IMDB API [11] and CURL to obtain actor, plot, keyword, genre, rating and display.

Figure 2. A sample Facebook wall post As friends upload their favorite scenes on Facebook, an application with suitable permissions to accounts can now rate the scenes within a piece of video content. This application can create a list of liked scenes alongwith names of who all like that scene for a given video. This data can be used to rate scenes and form statistics such as which scene have me and my friends liked the most? It gives real time rating of scene level information about videos. The list with the timestamps of the scenes can also be downloaded and used like a subtitle file to suggest upcoming good scenes to look out for. The pseudocode of this is as follows: 1. 2. Using Facebook API key, login to current viewers account, with suitable permissions. Read your own and friends feed messages and collect messages starting with FBF (application name), or use the posted by application field instead. Sort messages and count duplicate messages. Release statistic per title of which scenes are highest rated. Also save to file timestamp and scene for every title.

A psuedocode of this is as follows: 3. 4.

4. 5.

Information can be gathered by keeping count of how many times a title is being watched by a person. If the same title is being watched multiple times and is above a predefined threshold, the user is prompted to make it a favorite movie in Facebook.

Sharing details of the content being watched is done using a button on the video player. This button updates wall/news feed

3.

CONCLUSION

The above application is an innovative pervasive application that makes use of online social networks and online services to collaboratively enjoy a video.

[4] Miso, http://www.makeuseof.com/tag/miso-share-tvprogram-watching [5] Tunerfish. http://blog.tunerfish.com [6] P.Sinha, A.Ghose, C. Bhaumik and D. Das, Crowdsourced tagging of objects tracked on TV, IBC 2011, Amsterdam. [7] Universal subtitles. http://universalsubtitles.org/en/ [8] YFrog. http://yfrog.com [9] Facebook. http://www.facebook.com [10] CURL http://curl.haxx.se [11] IMDB API http://code.google.com/p/imdb-php/

4. 5.

ACKNOWLEDGEMENT REFERENCES

The authors would like to thank Amit K Agrawal for his inputs.

[1] IMDB, http://www.imdb.com [2] Google search by image. http://google/searchbyimage [3] Gracenote video id. http://www.gracenote.com/products/videoid