Information Retrieval Systems in Academic Libraries

The aim of this essay is to make some reflections about the way academic libraries use technology in order to satisfy users needs. While outlining general theoretical principles we will analyse two academic libraries online catalogues to which we were granted access. Most of students and researchers using academic libraries carry out projects about specific topics on which they need to gather further information. Motivated by an anomalous state of knowledge (ASK) (Belkin 1980; Belkin, Oddy and Brooks 1982) they perform an exploratory seeking (Rosenfeld and Morville 2007) formulating a query whose answer, located in a separated stored database, is subjectively right. What is needed in this kind of research environment is thus a text retrieval system that interact and communicate with the user so that the way users represent their request can be compared and matched with the way information represented in the databases by means of a dialogue which leads to retrieve the most relevant documents in the users perspective. In order to fulfill users information needs, on one side there are mathematical algorithms identifying the document to be presented; on the other, important role is played by the system design (the way for presenting results, how to group them, layout of the search-engine). Several methods can enhance the possibility of a better result: - Indexing techniques (documents are simply presented according to the precise number of terms matching). Weighting terms (documents are ranked according to their total weight). In this case each term is being given a weight according to a combination of 3 different criteria (Robertson and Jones 1994). - Iterative searching (documents, after going through statistical formulae to select a better combination of terms, are assessed by relevance). It can be performed by changing the request term weight (relevance weighting) (Robertson and Jones 1976) or changing the request composition (query expansion) (Robertson 1990). The latter reveals as a step toward more interactive systems, in that search expansion starts from a document whose relevance can be determined by the user (See Harman & Voorhees, 2006 for a review of studies of relevance feedback in the TREC project). - Other solutions helping widening the search results include: active bibliography, related documents from co-citation, cited by, research index (Rosenfeld and Morville 2007).

For the development of more effective IR mechanisms (sensu Robertson) and for helping libraries adopt one for their users, evaluation with the pivotal role played by evaluation criteria involved in the test processes become unavoidable. While in the past most of the experiments carried out to evaluate a IR system would rely mainly on the physical output given by the mechanism, evaluated against the inversely related concepts of precision and recall, nowadays many evaluation methods acknowledge the crucial importance of the user as fundamental participant of the evaluation process. This shift of perspectives, due to the advent of the cognitive, relevance and interactive revolutions (Robertson and Hancock-Beaulieu 1992) is also associated with the fact that current libraries catalogues offer such a large amount of resources (lets think to access to remote e-journals) to make difficult a laboratory-based experiment with relevance prejudgements of the whole set of documents. Since main aim of a IR system is the satisfaction of the users ASK several researches have been carried out over the last twenty years in order to combine quantitative (mechanism centered) and qualitative (user1

centered) evaluation methods (Robertson and Hancock-Beaulieu 1992, Hancock-Beaulieu 1990, HancockBeaulieu et al., 1991, Walker & Jones 1987 cited in Xie 2008). These studies advocate for including interaction and not only effectiveness factors in the evaluation process; (Su 1992, 1994, 2003; Saracevic and Kantor 1988a, b, c, Hersh Pentecost and Hickham 1996, Lancaster 1979 cited in Xie 2008). CASE STUDY: the online catalogues of the department of Archaeology of the University College London and University of Rome Sapienza were analysed. Results are summed up in the following tab: QUERY Prehistory Sahara Pottery African Pottery Africa Pottery Sapienza Catalogue
No. Documents retrieved

UCL Catalogue
No. Documents Retrieved

0 2 1
No. Documents retrieved Precision

377 8061 9591

No. Documents retrieved Precision

Prehistory Africa

12

0.6

1771

0.6

By typing any 3 terms search on the Sapienza catalogue 0 results were obtained. It is quite evident as in the Sapienza catalogue researcher face an initial serious drawback. By changing query and typing African pottery, only 2 documents (one relevant, one not relevant) were retrieved by the Sapienza catalogue. Unexpectedly, by typing Africa, the system retrieved only the not relevant one. Doing the same with the UCL catalogue, we noticed that the same book retrieved by the Sapienza system was included in both searches. We could assume that the indexing system on which the Sapienza catalogue is based does not include sophisticated stemming techniques. As far as precision calculation concerns both systems scored 0.6 (first top 10 documents were analysed). We think that precision should not be seen as a suitable criteria to assess a IR system since, as already shown by Su (1994), in an academic environment a higher recall reveals to be more helpful to students. Besides, an higher recall in the UCL catalogue led to clarify our ideas about what we were actually looking for and change our queries. An analysis of the systems design also provoked some reflections. In the Sapienza search engine typed queries are not displayed anywhere once the results come out; the advanced search is not very user friendly with too many available fields in the same screen. Besides in the UCL engine details as abstract and the fundamental recommendation are immediately readable. Furthermore two crucial features offered by the UCL catalogue allowing users to refine their search by subtopic and expand their research with suggested new searches is there missing.

In conclusion, this essay shows how strong IR systems should adopt a more user-centered oriented perspective, bearing in mind that areas such as ranking algorithms, building up of efficient access mechanisms, query processing, display and delivery of search results should be designed according to endusers needs, at the heart of any IR; secondly how systems evaluation really need to consider not only the effectiveness but also the relationship users create with the system, which if not emphatic, can lead them to opt for different sources. 2

References
http://trec.nist.gov/ 3

http://www.ucl.ac.uk/library/ http://opac.uniroma1.it/SebinaOpacRMS/Opac?sysb=GR_1

Broder, A. (2002). A taxonomy of web search SIGIR Forum Fall, 36 (2). [Online] Available: http://www.sigir.org/forum/F2002/broder.pdf (28 October 2012). Belkin, N.J. (1980), Anomalous States of Knowledge as a Basis for Information Retrieval, Canadian Journal of Information and Library Science, 5 pp. 133-143. University of Toronto Press. [Online] Available http://comminfo.rutgers.edu/~tefko/Courses/612/Articles/BelkinAnomolous.pdf (27 October 2012). Rosenfeld, L. and Morville, P. (2007), Information architecture for the World Wide Web (3rd edition): Designing Large-scale Web Sites, Sebastopol, CA.: OReilly. Saracevic, T., & Kantor, P. (1988a) A study of information seeking and retrieving: I. Background and methodology. Journal of the American Society for Information Science, 39(3), 161-176. Saracevic, T., & Kantor, P. (1988b). A study of information seeking and retrieving:II. Users, questions, and effectiveness. Journal of the American Society for Information Science, 39(3), 177-196. Saracevic, T., & Kantor, P. (1988c). A study of information seeking and retrieving:III. Searchers, searches, and overlap. Journal of the American Society for Information Science, 39(3), 197-216. Hersh, W., Pentecost, J., & Hickam, D. (1996). A task-oriented approach to information retrieval evaluation. Journal of the American Society for Information Science, 47(1), 50-56. Hancock-Beaulieu, M. (1990). Evaluating the impact of an online library catalogue in subject searching behavior at the catalogue and at the shelves. Journal of Documentation, 46(4), 318-338. Hancock-Beaulieu, M., Robertson, S., & Neilson, C. (1991). Evaluation of online catalogues: Eliciting information from the user. Information Processing and Management, 27(5), 523-532. Su, L. T. (2003) A comprehensive and systematic model of user evaluation of Web search engines: I. Theory and background. Journal of the American Society for Information Science and Technology, 54(13), 11751192. Harman, D. K., & Voorhees, E. M. (2006) TREC: An overview. Annual Review of Information Science and Technology, 40, 113-155. Lancaster, F. W. (1979). Information retrieval systems: Characteristics, testing, and evaluation. New York: John Wiley & Sons. Robertson, S.E. (1990). On term selection for query expansion, Journal of Documentation 46 (4), 359-364. Robertson, S.E. and M.M. Hancock-Beaulieu (1992). On the evaluation of IR system, Information Processing and Management 28 (4), 457-466. Robertson, S.E. and K. Sparck Jones (1976). Relevance weighting of search terms. Journal of the American Society for Information Science May-June, 129-145. 4

Robertson, S.E. & K. Sparck Jones (1994). Simple, proven approaches to text retrieval, University of Cambridge technical note, TR356. [Online] Available: http://www.cl.cam.ac.uk/techreports/UCAM-CL-TR356.pdf (28 October 2012) Su, L. T. (1992). Evaluation measures for interactive information retrieval. Information Processing and Management, 28(4), 503-516. Su, L. T. (1994) The relevance of recall and precision in user evaluation. Journal of the American Society for Information Science, 45(3), 207-217. Xie, I. (2008). Interactive information retrieval in digital environments. Hershey, PA: IGI Global [Online] http://kutuphane.ankara.edu.tr/igi/Interactive%20Information%20Retrieval%20in%20Digital%20Environm ents.pdf (29 October 2012) Walker, S. & Jones, R.M. (1987) Improving subject retrieval in online catalogues. 1. Stemming, automatic spelling correction and cross-reference tables. London: British Library.