Documentos de Académico
Documentos de Profesional
Documentos de Cultura
The evaluation process is a complex and systematic procedure followed by the assessor in order to
provide a response regarding the achievement of the performance standards by learners. The threedimensional system (measurement assessment decision), which defines the evaluation concept, must
comprise all the research, information, rationale, analysis and conclusions drawn to reach an estimated
value. In this context, the present study aims to offer a brief overview of the main stages of the
assessment process, including the techniques for developing assessment tools as well as the ones for the
interpretation and monitoring of the learning outcomes.
The entire methodological approach aims to explain the relationship between the method, the
assessment tools and the purpose and objectives of the evaluation. The detailed analysis highlights the
correlation between the learning outcomes, the assessment objectives and the specific
competencies based on a case study: the Geography of tourism subject.
The study could serve as a model for improvement of the evaluation system, due to the fact that
these tools enable the evaluator to develop a set of remedial measures aimed at optimising the
educational process. Moreover, the tools, the analysis and the interpretation techniques presented in this
paper contribute to the objective understanding of the level of acquired knowledge, as well as to the
assessment of the capabilities and practical skills gained by the students along the way.
Keywords: Assessment, Learning outcomes, Methodology.
Introduction
A correct approach for evaluation and examination aims at highlighting the efficiency and the overall
performance of the educational process. The main purpose of the evaluation strategy is to improve the
educational process in the direction chosen by the one that designs and applies it. In the academic field, due
to the specificity of the educational activities, the evaluation strategy chosen is also a tool for monitoring the
educational progress. This gives the professor the possibility of making an objective analysis, based on the
existence of clear evaluation criteria, developed and mentioned in advance.
Background: the Importance of Evaluation
It is unanimously accepted that evaluation is an integral part of teaching and learning. The subject of
teaching effectiveness has extensively been addressed in research literature (Marsh, 1987) due to the fact
125
126
that it is an essential process for improvement. According to the McKinsey report (Barber and Mourshed,
2007:43 in Malm, 2009), the quality of an educational system cannot exceed the quality of its teachers
and the only way to improve outcomes is to improve instruction. In this respect, Candlin and Edelhoff
(1982:vi) mention that learners learn most when they are quite precisely aware of . . . how their efforts
are to be judged and evaluated. Therefore, evaluation is crucial for the improvement of the educational
system since it plays a pivotal role in deciding what the learners learn and what the teacher teach
(Agrawal, 2004). Given its importance, the present article aims to propose an evaluation method that
could lead to the development of a remedial measures plan and hence to the improvement of the entire
process.
According to the National Science Education Standards, there are two major types of evaluation:
formative and summative (Coffey et al., 2001). The first one is designed to distinguish worthwhile
programs from ineffective ones and the latter is designed to help improve existing ones in order to
achieve the desirable results. The one that is currentlygoing to be addressed is the summative evaluation,
which is conducted to determine whether the subjects expectations are being met and which of its
consequences are met (Scriven, 1991). Cowan et al. (2013) focus more on the intention of the evaluation,
highlighting the fact that formative evaluation aims to identify the scope and the potential for
improvement, while summative evaluation is a judgement that usually works as a basis of one or more
decisions that will determine whether the competencies have been gained and if the candidate deserves
the promotion, award etc. Through summative evaluation, the professor can also assess the way in which
the objectives have been reached according to the difficulty level of the standardized test.
In this respect, the method described below is a useful tool we have used successfully in regular
teaching and learning situations, and which we can thus recommend for as practicable and useful.
Designing the Evaluation Test
From a quantitative point of view, the evaluation test is made of a series of items with precise elaboration
rules. In terms of qualitative research, the evaluation test becomes more relevant and efficient if the items
do not particularly focus on assessing the gained knowledge, but rather on how this knowledge could be
applied on similar or new situations. Therefore, the evaluation test has to meet some elaboration
requirements in order to meet the purpose for which it was designed, the items being selected based on a
specification matrix a relational and distribution table between the content elements and the specific
competencies.
Another essential stage is the analysis and the interpretation of the evaluation test in order to quantify
the learning outcomes. This has to focus on determining the level of knowledge acquired, on intellectual
capacity building and on the students ability to apply the knowledge acquired in specific situations. This
stage ensures an objective feedback through which the professor is systematically being provided with
information related to the learning outcomes. Therefore, through analysis, the frequent shortcomings and
mistakes are being highlighted, offering the evaluator the possibility of drawing a set of remedial
measures in order to improve the teaching methods and techniques for better results.
The Structure of the Evaluation Test
In order to assess the learning outcomes for the subject Geography of Tourism at Faculty of Geography,
University of Bucharest, an evaluation test was designed using all the types of items objective, semiobjective and subjective (Figure 1). The highest proportion was covered by the semi-objective items
(50%), items which require an answer limited in length, form and content. In this category there are
included: an item of filling in the gaps (exercise II, 10 points), a short-answer item (exercise IV, 15
points) and two items based on the analysis of the graphic/cartographic support (exercise VII, 10 points
and exercise VIII, 15 points). The advantages brought by the use of more semi-objective items is given by
127
the possibility of evaluating a high range of abilities, capacities and knowledge. Moreover, it is quite easy
to mark them since they have a high degree of objectivity in the context of a suitable grading scale.
The objective items cover 22% of the evaluation test content, comprising two categories: a multiple
choice item (exercise I, 10 points) and a selection/association item (exercise III, 12 points). The strengths
of using objective items lie in the fact that the answers have to be given in a reasonable amount of time
and space, covering a wide range of evaluation objectives and content elements.
The third category - the subjective items, which call for an open answer can be found in the
evaluation test in a proportion of 18%. This comes to complete the other selected categories presented
above due to the fact that a complex evaluation should also assess the abilities of analysis, argumentation
and synthesis. In this category there are two items: an unstructured essay (exercise V, 8 points) and a
structured essay, where the expected answer is guided by the way in which the request is formulated
(exercise VI, 10 points). In this case, the strengths of this type of items are related to the fact that they can
cover a wide range of objectives from knowledge to analysis and evaluation, measuring high-level
abilities and complex learning outcomes.
Assigned
score
22 p
TOTAL
100 p
I
II
III
IV
V
50 p
VI
VII
VIII
18%
10
10%
15
22%
18 p
50%
10 p
Objective Items
Subjective items
Granted points
20
128
c. 1975;
d. 1995.
2. In Roman times, there were several spas operating, among which on current Romanian territory:
a. Slnic Moldova i Olneti;
b. Bile Herculane i Bile Geoagiu;
3.
b. A written contract;
4.
c. A symbolic gesture;
d. A gift for the host.
10 points
1. The most important intergovernmental body that creates an overall framework for the tourism activity
in the world is. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2. Epidaurus archaeological site is famous for its best-kept . . . . . . . . . . . . . . . . . . .
3. The terms of tourism and tourist started being used in the everyday language in the midnineteenth century, a great influence in this respect having Stendhals novel. . . . . . . . . . . . . . . . . . . . .
..
4. Between Cap de Tortosa and Barcelona lies Coast
5. On the Italian coast o Adriatic Sea there is a dence network of resorts between Trieste and . . . . . . . . .
..............................................................
III. Put the following resorts into the right column:
12 points
Meribel, Interlaken, Kitzbhel, Val dIsre, Valgardena, Crans Montana, Saas Fee, Zell am See, Bormio,
Courchevel, Zermat, Jungfraujoch, Cortina DAmpezzo, Hintertux, Grinderwald.
a. Italian Alps
b. Bernese
c. Austrian
Alps
Alps
129
d. Pennine Alps
15 points
VI. Characterize briefly health tourism. The characterization should include: generalities, forms
and dedicated areas.
10 points
VIII. Analyse the cartogram below which represents the international tourist arrivals (2010) in
Europe.
15 points
130
C1
C2
C3
C4
C5
TOTAL
Competencies
C6
Contents
1. Fundamentals,
content,
terminology
I.1. (2p)
IV.2. (5p)
I.4. (2p)
IV.3. (5p)
V. (8p)
24 p
27 %
II.1.(2p)
2. Brief history
of the emergence
and evolution of
travel and
tourism
3. The dimension
of the tourism
phenomenon
nowadays
I.3. (2p)
I.2. (2p)
II.3.(2p)
II.2.(2p)
8p
9%
IV.1. (5p)
VIII. (15p)
20 p
22 %
4. Types and
forms of tourism
I.5. (2p)
5. International
tourism areas
II.4.(2p)
TOTAL
16 p
14 p
25 p
12 p
15 p
8p
90 p
(18 %)
(15 %)
(28 %)
(13 %)
(17 %)
(9 %)
100 %
VII. (10p)
VI. (10p)
22 p
24 %
III. (12p)
16 p
18 %
II.5.(2p)
* Where:
C 1. Identifying / acknowledging certain data, concepts, relations or specific categories
C 2. Describing / exemplifying certain phenomena, processes or concrete situations
C 3. Defining / characterising / comparing facts, data, properties, phenomena etc.
C 4. Using / applying specific knowledge in problem situations.
C 5. Analysing / generalising / transferring facts, processes, phenomena, situations in various contexts.
C 6. Evaluating / explaining / interpreting facts, processes, phenomena, situations in various contexts.
Figure 2. Specification Matrix
131
In the evaluation test projected and presented above the specification matrix model is composed of
five chapters which cover the content taught during the first semester that refers to the general concepts in
Geography of Tourism (see Figure 2). The distribution of the content items is balanced so that the test
would cover all the issues taught during classes. Given the fact that the subject aims to introduce and
consolidate the knowledge/the fundamentals through definitions and characterizations, higher proportions
were assigned to: 1. Fundamentals, content, terminology (27%); 3. The dimension of the tourism
phenomenon nowadays (22%). 4. Types and forms of tourism (24%). Lower content proportions were
assigned to the chapters which present examples, descriptions or localization of specific elements: 5.
International tourist areas and 2. Brief history of the emergence and evolution of travel and tourism.
Competencies:
C 1. Identification/Recognition of data,
concepts, relations, categories related to the
subject/field of study 16 p / 18 %
C 2. Description / Exemplification of certain
phenomena, processes or concrete situations
related to the subject/field of study 14 p / 15 %
C 3. Definition/ Description/ Comparison of
facts, data, properties, phenomena. 25 p / 28 %
C 4. Using / Applying specific knowledge for
problem-solving in the field of study12 p / 13 %
C 5. Analysis/ Generalization /Transfer of
facts, processes, pheomena, situations in various
contexts 15 p / 17 %
C 6. Evaluation / Explanation /
Interpretation of facts, processes, phenomena,
situations in various contexts 8 p / 9 %
Given the fact that during the second semester one of the aims of the subject is to develop the
practical abilities, it was necessary, where possible, to try a familiarization of the students with the items
which are relevant for consolidating these competencies. However, since the aim of the evaluation test is
to provide a complex summative evaluation of the knowledge acquired during the first semester, it was
necessary to also focus on other competencies, such as C4. Using / applying specific knowledge in
problem situations (13%) and C6. Evaluation / Explanation / Interpretation of facts, processes,
phenomena, situations in various contexts (9%).
132
Score range
No of
students
30 39 points
40 49 points
50 59 points
60 69 points
70 79 points
80 89 points
90 99 points
100 points
1
Figure 4. Distribution of score per number of students
For the group of students (no. 107) chosen as a case study for the current research, the graphic model
of the distribution of the score per number of students has a bell shape or a parable shape pointing
upwards. Its highest peak marks the category 80-89 points, where 6 students fit, 24% from the group total.
On the lower level of the representation lies the category 30-39 points, where a single student falls. Then,
until the highest peak of the parable the curve is growing steadily, having added one student for each
category. Below the highest peak the curve drops abruptly, 3 students falling inside the 90-99 points
category, reaching the upper end of the representation (the maximum score) where only one student falls.
The results of the group 107 show that it is an average one, given the fact that 60% of the sample obtained
scores between 60 and 89 points. Below the passing score (30-49 points) there are only 3 students, but it
is a worrying fact that other 3 students are at risk (with scores between 50-59 points). The good results,
above avera
ge, were reached by a moderate number of students of whom three fell into the 90-99
points category, and one student reached the maximum score 100 points.
133
Calculation
Total score
Average score
awarded (Gr.
107)
Rate of
achievement
Chapter 1
24 p
18,36 p
76,5 %
Chapter 2
I2 + I3 + II2 + II3
8p
7,04 p
88 %
Chapter 3
IV1 + VIII
20 p
10,92 p
54,6 %
Chapter 4
I5 + VI + VII
22 p
13,84 p
62,9 %
Chapter 5
16 p
11,72 p
73,2 %
Therefore, we focused more on the content that reinforces the fundamentals through definitions and
characterization (chapters 1, 3 and 4) and less on the ones that mainly present examples, descriptions or
localization of specific elements (chapters 2 and 5). After a brief analysis of the content-related results we
discovered that the highest level of achievement was reached for chapter 2 A brief history of the
emergence and evolution of travel and tourism. On one hand, this is due to the fact that the chapter
presents generalities and, on the other hand, the information was structured on multiple choice items, with
a relatively low difficulty level. Other good levels of achievement (around 75%) were reached for chapter
1 Fundamentals, content and terminology and for the chapter 5 International tourist areas. For these
chapters, there were several items that required defining terms (short answer items), one unstructured
essay item type and one item of association /selection. A lower level of achievement (62.9%) was attained
for chapter 4 Types and forms of tourism, mostly because the knowledge was evaluated through a
structured essay and an image-based analysis and the students were not used to this type of assessment
and interpretation of information taught during classes. The same explanation applies to the lowest rate of
134
achievement - of just 54.6% - attained for the content of chapter 3 (The dimension of the tourist
phenomenon nowadays), most of the points being awarded for an item of analysis based on map support.
Level of Achievement Based on Competencies
The competencies to be evaluated through this test were covered by different item types depending on
their characteristics and on the information taught to the students, which aimed mostly to familiarize them
with the concepts and fundamentals of Geography of Tourism (see Table 2). Therefore, the specific
competencies that meet the general aim have balanced rates of achievement, the highest being assigned to
the competence C3 - Definition/ Description/ Comparison of facts, data, properties, phenomena (28%).
Table 2. Distribution and level of achievement based on competencies
*
Calculation
Total
score
Average
score
awarded
Rate of
achievement
C1
16 p
13,68 p
85,5 %
C2
I2 + II2 + VII
14 p
10,24 p
73,1 %
C3
25 p
16 p
64 %
C4
III
12 p
8,44 p
70,3 %
C5
VIII
15 p
7,04 p
46,9 %
C6
8p
6,48 p
81 %
* Where:
C 1. Identification/Recognition of data, concepts, relations, categories related to the subject/field of study
C 2. Description / Exemplification of certain phenomena, processes or concrete situations related to the
subject/field of study
C 3. Definition/ Description/ Comparison of facts, data, properties, phenomena.
C 4. Using / Applying specific knowledge for problem-solving in the field of study
C 5. Analysis/ Generalization /Transfer of facts, processes, phenomena, situations in various contexts
C 6. Evaluation / Explanation / Interpretation of facts, processes, phenomena, situations in various contexts
As the competencies-based analysis shows, the highest rate of achievement (85,5%) belongs to C1
(Identification/Recognition of data, concepts, relations, categories related to the subject/field of study)
due to the fact that in order to cover these competencies there were used items which do not request the
interpretation of information, being mostly multiple choice items or filling-in the gaps. At the other
extreme the students faced major difficulties when it came to the map-based analysis item which covered
C5 (Analysis/ Generalization /Transfer of facts, processes, pheomena, situations in various contexts),
reaching a rate of achievement of only 46,9%. Another low rate (64%) refers to C 3 (Definition/
Description/ Comparison of facts, data, properties, phenomena), fact which indicates that students had not
developed a specialized language to help them define clearly the concepts taught or explained during
classes. The level of achievement for the other competencies is relatively good (between 70-80%);
according to the items solved most of the students were able to describe, present or illustrate various
processes, facts or concrete situations.
Conclusions
The present study presented the evaluation process and its stages, from the elaboration techniques of the
evaluation tools, to the ones for the interpretation and monitoring of the learning outcomes.
135
For the case study chosen we highlighted the correlation between the learning outcomes, the
formulated assessment objectives and the specific competencies, the whole approach aiming to explain
the relation between the method, the evaluation tool, the purpose and the evaluation objectives of the
subject Geography of Tourism.
The instruments, the analysis and the techniques helped us measure the acquired knowledge level,
the development of the intellectual capacity and the students ability to apply the knowledge. Generally,
the results obtained by the students from the group 107 on score categories showed us that the level of the
group is an average one (the average score 71,88 points) and 14 students (56% respectively) got scores
above the average, out of whom one got the maximum score. Below the passing score there were 3
students (12% of the total), and between the range of 50-71,88 points there were 8 students (32% of the
total).
The analysis of the achievement level based on item types, content and competencies helped us
identify the weak points and the strengths regarding the entire evaluation process. As strengths we spotted
the fact that students solve easily the items which require a short answer or to fill in the gaps, as well as
the multiple choice items. Given the fact that the rate of achievement for these items was above 80% this
means students have understood the content taught and that they can identify, through association, the
correct answers. Another good rate of achievement (around 80%) was attained for the items which cover
the competencies related to the assessment, explanation or the interpretation of various processes, facts or
concrete situations.
As weaknesses, we realized that students faced great difficulties in interpreting and analysing
information based on map support (rate of achievement below 50%), as well as in dealing with the
unstructured essay (rate of achievement of 50%). When it comes to competencies, the weakest result was
for C3 (Definition/ Description/ Comparison of facts, data, properties, phenomena) which covers best the
introductory character of the subject. This highlights the fact that students had not developed yet a
specialized language to help them define the concepts taught and explained during classes. All these claim
for remedial measures in the second semester of the academic year through simple, empirical
explanations of several processes and phenomena to improve and consolidate the specialized language for
most of the students. Moreover, the action plan also involves guided or structured description of specific
elements based on graphic and cartographic support as well as making students work more with maps and
images.
In conclusion, we believe that the present study could be a model for the improvement of the
evaluation system because these tools give the evaluator the possibility to design a set of remedial
measures in order to improve the educational process.
References
1.
2.
3.
4.
5.
6.
7.
Agrawal, M. 2004. Curricular reform in schools: the importance of evaluation. Journal of Curriculum Studies,
36, 361-379.
Barber, M. & Mourshed, M. 2007. How the world's best-performing schools systems come out on top,
McKinsey & Company.
Candlin, C., Abbs, B. & Edelhoff, C. 1982. Challenges: a multi-media project for learners of English.
Teacher's guide, Langenscheidt.
Centra, J. A. 1994. The Use of the Teaching Portfolio and Student Evaluations for Summative Evaluation. The
Journal of Higher Education, 65, 555-570.
Coffey, J., Black, P., Atkin, J. M., Harrison, C., Marshall, B. & Wiliam, D. 2001. Classroom Assessment and
the National Science Education Standards, National Academies Press.
Cowan, J. D., John, C., George, J. D. D. & Judith, G. 2013. A Handbook of Techniques for Formative
Evaluation: Mapping the Students' Learning Experience, Taylor & Francis.
Dulam, M. E. 1996. Didactic geografic, Editura Clusium, Cluj Napoca;
136
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
Dulam, M. E. 2002 Modele, strategii i tehnici didactice activizante cu aplicaii n geografie, Editura
Clusium, Cluj Napoca;
Dulam, M. E. 2010 Didactica axat pe competene, Editura Presa Universitar Clujan, Cluj Napoca;
George, J. W. & Cowan, J. 1999. A Handbook of Techniques for Formative Evaluation: Mapping the Student's
Learning Experience, Kogan Page.
Gheorghila A. 2014 Geografia Turismului. Metode de analiz n turism, 3rd edition, Editura Universitar,
Bucureti
Ilinca N. 2000 Didactica geografiei, Editura Corint, Bucureti
Malm, B. 2009. Towards a new professionalism: enhancing personal and professional development in teacher
education. Journal of Education for Teaching, 35, 77-91.
Marsh, H. W. 1987. Students' evaluations of University teaching: Research findings, methodological issues,
and directions for future research. International Journal of Educational Research, 11, 253-388.
Radu I.T. 2004 Evaluarea n procesul didactic, Editura Didactic i Pedagogic, Bucureti
Scriven, M. 1991. Evaluation Thesaurus, SAGE Publications.
Stoica, A. 2001. Evaluarea curent i examenele. Ghid pentru profesori.