Está en la página 1de 16

A competence-based constructivist tool for evaluation

MANUEL-MIGUEL RAMOSLVAREZ Y GLORIA LUQUE Universidad de Jaén

R AMOS -Á LVAREZ Y G LORIA L UQUE Universidad de Jaén Abstract Recently the educational

Abstract

Recently the educational system in general, and the university in particular, have suffered a transformation in the way learning, teaching, and subsequently, evaluation are considered due to changes in the European context, including the Bologna Declaration and the constructivist view of learning. There are several adjustments required to implement successfully traditional university teaching practices, such as the use of new technologies in teaching, new ways of tutoring, different groupings, the teaching of competencies and social skills, and, as a result, new forms of evaluation, which constitute the object of this paper. Thus, we show an objective alternative –or complement– to traditional evaluation: a very exhaustive tool for evaluating oral and written presentations which includes four sections: general aspects, contents, questions concerning design and layout, and psychological aspects concerning general competencies. Throughout the paper, the constructivist notions of learning, teaching and evaluation are substantiated and validated with the use of this tool for group or individual presentations or end-of-degree dissertations. Keywords: Constructivism, learning, teaching, evaluation tool, oral presentations.

Una herramienta constructivista para la evaluación basada en competencias

Resumen

Recientemente se ha producido una reforma importante en varios niveles educativos, afectando también a la Universidad. El aprendizaje, la enseñanza, y dentro de esta última, la evaluación, son procesos educativos que han sufrido grandes cambios debido, principalmente, al contexto europeo en general (Bolonia) y a la aplicación de la noción constructivista de aprendizaje a los contextos universitarios. Algunos de los ajustes más significati- vos que afectan a la educación universitaria tienen que ver con el uso de nuevas tecnologías, diferentes tipos de agrupamientos, sistemas tutoriales alternativos y entrenamiento en habilidades sociales y competencias. Todo ello conlleva una forma de evaluación diferente, aunque no por ello menos objetiva, para la que se presenta aquí una herramienta o registro. Este registro permite evaluar presentaciones orales y escritas, tanto individuales como en grupo, dentro de la carrera/grado o en postgrado. Incluye cuatro secciones: aspectos generales, contenidos, aspectos relacionados con formato y diseño, y habilidades psicológicas relacionados con las competencias. A lo largo del texto se justifica el uso de la herramienta a partir del marco teórico de enseñanza y aprendizaje cons- tructivista. Palabras clave: Constructivismo, aprendizaje, enseñanza, herramienta de evaluación, presentaciones orales.

Correspondencia con los autores: Manuel-Miguel Ramos Álvarez. Dpto. de Psicología. Facultad de Humanidades y CC de la Educación. Universidad de Jaén. Paraje Las Lagunillas s/n. 23071 Jaén. E-mail: mramos@ujaen.es.

© 2010 Fundación Infancia y Aprendizaje, ISSN: 1135-6405

Cultura y Educación, 2010, 22 (3), 329-344

330 Cultura y Educación, 2010, 22 (2), pp. 329-344

Introduction

In recent years the educational system in general, and the university, in particular, have suffered a transformation in the way learning and subsequently, teaching, are considered. The European context and the Bologna Declaration on one side, and the constructivist view of learning on the other, which in fact are interrelated, constitute the two most important causes of the reform of higher education in Europe and, as a result, in Spain. To meet the demands of the European convergence, and adjust the notion of learning-teaching, Spanish universities are currently involved in an extensive transformation of their educational system, which has been called the European Credit Transfer System. There are several adjustments required to implement successfully traditional university teaching practices, such as the use of new technologies in teaching, new ways of tutoring, different groupings, the teaching of competencies and social skills, and, as a consequence, new forms of evaluation (De Mendoza, Domínguez & Martín, 2002; Márquez, 2004; Martín, 1999), which will be the object of this paper. We aim to show an objective alternative –or complement– to traditional evaluation.

Learning: from the Objective to the Constructivist View

From the last century to the present day, the characterization of learning processes has experienced an important evolution from behaviourist to cognitive and then constructivist accounts (see Cooper, 1993; Ertmer & Newby, 1993). The table below (Figure 1) summarizes this psychological and pedagogical evolution of learning models and how teaching methodology has evolved subsequently in the past century.

FIGURE 1

Evolution of learning theories and models and corresponding methodological implications

THE LEARNING

MODEL

METHODOLOGY

& INSTRUCTION

IMPLICATIONS

IMPLICATIONS

TO THE

LEARNER

BEHAVIOURIST

(DECADES: 20th-40th)

A learning algorithm that allows the association between a specific stimulus and its correct response, by means of rules/principles (i.e. reinforcement), resulting in a repetition of the expected behaviour or outcome.

Emphasis is on Objectivism:

observable & measurable outcomes.

Prescription and repetition of answer strings, to elicit desired outcomes about knowledge and skills, mastering early steps before progressing to more complex levels of performance.

Passive Learner. Expected to reproduce knowledge transferred by teacher.

COGNITIVISM

(DECADES: 50th-80th)

A learning process that allows the acquisition of knowledge, aimed at structuring & organizing in memory the processed information.

Emphasis is on Cognitive Processes & knowledge representation: remember, retrieve & store.

Training in mental operations, to organize information in an optimal way (i.e. with conceptual maps that distinguish novice and expert learners), connecting prior knowledge to new knowledge.

Active Learner: Result depends on how teacher presents material, but also on the way the learner is able to process information.

CONSTRUCTIVISM

(DECADES: 80-Onwards)

A meta-learning process involving

a mental action guided by tools, so that the learner creates meaning according to his/her own interpretation of the word based on experiences and interactions.

Emphasis is on problem solving- decision making, using authentic tasks in meaningful realistic settings.

Gradual transfer of self-regulation processes, where teacher is a supporting-orienting-facilitator, focused on the process not the product.

Active Learner: meaning imposed

by learner, who creates meaning

rather than reproduce or acquire it.

Objetivism Introductory Level
Objetivism
Introductory
Level
Advanced Level
Advanced
Level
Constructuvism Expert Level
Constructuvism
Expert Level

First, behaviourist accounts, focusing on conditioning, associating stimuli to responses through habit formation, repetition and reinforcement, have been

A competence-based constructivist tool for evaluation / M.-M. Ramos-Álvarez and G. Luque

331

shown to be effective for teaching factual content, but there is less evidence that this instruction transfers to higher order cognitive skills (Palincsar, 1998). This lack of a satisfactory explanation of the mechanisms that account for learning gave way to new learning-teaching models (see second column in Figure 1: Cognitivism). Rejecting previous emphasis on observable and measurable outcomes, learning was considered as a process that allows the acquisition of knowledge, specifically aimed at organizing in memory the information processed by the learner. Within the second period, the cognitive-constructivist sequence evolved from Piaget’s (1970) view of accommodation rather than assimilation, to Ausubel’s (1968) meaningful learning –that arises from the learners’ background knowledge–, later still with Vigotsky’s (1962) zone of proximal development, and then Bruner’s (1983) scaffolding, where social or environmental factors again fulfil an important role in the learning process, together with internal –cognitive– features and a more humanistic view of learning and teaching. Later cognitive oriented accounts, originated with Anderson’s (1985) notions of procedural and declarative knowledge, have suggested that learning must involve cognitive processes such as directed attention, intentional focus and memory to be effective and have an enduring effect, and they must also take into account the influence of contextual factors, such as the processing constraints involved in the learning task. The second and third columns in figure 1 can be said to constitute a continuum, rather than two different models. That is to say, all cognitive science theories entail some form of constructivism to the extent that cognitive structures are typically viewed as individually constructed in the process of interpreting experiences in particular contexts (Palincsar, 1998). The individual has now become the main protagonist in the learning process –as in the second period, also–, interacting actively with his/her environment in order to learn or acquire any kind of theoretical, practical or instrumental knowledge. In this sense, the emphasis has shifted from the operational stage (what learners know and are able to do with no contextual help: i.e. knowledge of concepts) to a focus on what learners are able to do with the help of contextual clues –in social interaction– and starting from their background knowledge: in other words, to promoting those mental actions leading to self-regulation processes such as learning strategies and procedures (and, within university contexts, competencies). This transformation constitutes the so called post-constructivist views (see third column in Figure 1: Constructivism) (Palincsar, 1998). All the diverse constructivist perspectives reject the view that the locus of knowledge is the individual. Learning and understanding are social processes, and cultural activities and tools (including interpreting symbols: reading and writing) are also considered as a part of conceptual development. The emphasis now is on problem solving and decision making tasks, preferably authentic and in meaningful settings. This involves not only a higher degree of autonomy on the part of the learner, but a more skilled learner, who is expected to go beyond the reproduction of knowledge at an introductory level (Behaviourism) or the organization of this knowledge at an advanced level (Cognitivism): he/she must create meaning as a function of his/her own particular experiences and interactions with the environment, or, in other words, the student must become an expert learner (see last line in Figure 1). Accepting the post-constructivist view of learning involves shifting traditional views of teaching towards the learner and what the teacher can do to help them learn and become autonomous (De Mendoza et al., 2002).

332 Cultura y Educación, 2010, 22 (2), pp. 329-344

Teaching. From Objectives to Competencies

Palincsar (1998) suggests that:

It is interesting to consider the extent to which contemporary interest in social constructivist perspectives is propelled by recent educational reform efforts encouraging students to assume a more active role in their learning, to explain their ideas to one another, to discuss disagreements, and to cooperate in the solution of complex problems, while teachers participate in the design of these contexts and the facilitation of this kind of activities. All these notions have enormous implications for the culture of schools (p.

355).

The evolution of theoretical learning models is parallel to how teaching has been considered, so that teaching models and approaches, curricular and instructional design and educative programs in general have also experienced a transformation (see Jonassen, Mayes, & McAleese, 1993; Vighnarajah, Wong, & Kamariah, 2008; and a concrete example in Karagiorgi & Symeou, 2005). Several decades ago teaching practices were aimed at achieving objectives, whereas the current and future higher educational system involves the notion of competencies, which students are expected to achieve (see Figure 1: from operational to procedural objectives to competencies). This change is being effected through a reform of the educative systems. In higher educational levels the influence of the constructivist approach has taken place two decades later, and it constitutes the framework of the European credit transfer system, which has shifted the focus from teaching traditional contents (in terms of concepts and objectives) towards helping students acquire different competencies (see Figure 2) (Montanero, Mateos, Gómez & Alejo, 2006). This is a worldwide 1 change and a reality now in Europe and also in Spain, where the Bologna Process is producing a revision of educational structures within the European higher educational system 2 .

FIGURE 2

Evolution of teaching design and instructional implications

ACADEMIC SETTINGS

SOCIAL/PERSONAL

PROFESSIONAL

SETTINGS

SETTINGS

Knowledge: Facts, procedures, principles, & theories Understanding Used in high- order (complex) activities
Knowledge:
Facts, procedures,
principles, & theories
Understanding
Used in high-
order (complex)
activities
Repetition to
interrelations in
increase
new contexts/
automatism
situations
Re-ordering
cognition

Skill:

Abilities, & mental operations/processes

New Knowledge:

Attitudes, values, rules, & norms

New Skills:

Strategic thinking demanded in contemporary society (i.e. emotional intelligence)

New Knowledge: (meta-cognition) about functioning Very complex, Effective application: specific, changing Valued
New Knowledge:
(meta-cognition)
about functioning
Very complex,
Effective application:
specific, changing
Valued capabilities,
qualifications,
expertise, & mastery
environments
(i.e. ill-defined
problems)
Complex mental
re-structuring:
Deep conceptual
change

High-Order Skills:

For effective selection, rapid adaptation, & satisfaction Selective and coordinated use of interconnected strategies

Detailed Objectives Operational Competencies (Procedural & objectives Attitudinal) Educational reform (i.e.
Detailed Objectives
Operational
Competencies
(Procedural &
objectives
Attitudinal)
Educational reform
(i.e. Bolonia, Canada)
demanding new challenges

A competence-based constructivist tool for evaluation / M.-M. Ramos-Álvarez and G. Luque

333

Simply put, competencies go beyond learned contents-concepts, imposing a very profound reorganization of acquired knowledge and an extensive transfer to new situations and contexts (Monereo & Pozo, 2001, 2007). The problem is that the concept of competence is troublesome, basically because of its own definition, which has no explanatory power: conclusions about competencies seem to coincide with its definition (Westera, 2001, pp. 80-81). Moreover, competencies can be defined in a narrower or broader sense, within a theoretical perspective or an operational one. However, the fact that it has been definitely incorporated in educational contexts has contributed to turning this notion of competencies into a fuzzy and difficult to operativize concept (Monereo & Pozo, 2007). As a matter of fact, competencies display different interpretations in different fields, and sometimes they are confused with skills, or defined by means of dichotomies such as Chomsky’s competence vs. performance or Anderson’s procedural versus declarative knowledge (see Pozo & Monereo, 1999). Besides, official educational documents are still too general. In this sense, figure 2 is an attempt to establish the limits of the notion of competencies as compared to other related notions. Basically, the aim of educational programs is that learners acquire both knowledge and skills. Originally, these aims were limited to academic settings (knowledge about facts, procedures, models and theories, and to be skilled at some mental operations); however, with successive educative reforms that have demanded new challenges, these new knowledges and skills have been incorporated into learners curricula, including both their personal and social context (which include knowledge about attitude, norms, and new skills about emotional intelligence and procedures –being able to do something–). Finally, the university reform and the establishment of degrees as such has explicitly incorporated other aims having to do with professional settings, both as new knowledges and also as new skills. The difference between an educative model organized around objectives and another that considers competencies is that new abilities are of a higher order nature (i.e. for rapid adaptation, coordinated use of interconnected strategies, etc.), requiring a deep conceptual change and a very complex adaptation to specific, changing environments. Professional settings are aimed at the so called competencies, defined as the set of concepts/contents, procedures, attitudes and abilities acquired as a result of the learning process in multiple contexts. Within this framework, students are not only expected to learn different objectives, contents, materials, etc., but also professional and scientific abilities –competencies– which should be the outcome developed from those contents, and which are essential for successfully implementing a job in their prospective labour market. Professional and personal competencies are then the capacities or skills a learner must have acquired when he/she finishes the University. However, university studies cannot only restrict themselves to the needs of the labour market, so conceptual competencies also need to be considered. Thus, competencies could also be defined as the ability to understand and use knowledge in different contexts. For instance, to solve problems, to use new technologies, to learn autonomously, to investigate, to think creatively, to be able to communicate, cooperate,… and so on (see third column in Figure 2). The important issue is not only being able to use particular concepts or knowledge, but to be able to use them after the university, both in academic and work contexts. To sum up, according to Westera (2001), competencies involve four levels: knowledge, skills –knowing how–, attitudes and the ability throughout the learner’s whole life to learn in an autonomous way.

334 Cultura y Educación, 2010, 22 (2), pp. 329-344

Besides the problem of the definition of this term, an additional problem is involved in the lack of precision in the distinction of two types of competencies:

general and specific (Pérez, 2009). According to the Tuning Project 3 , general competencies are common to all university studies and are necessary to obtain employment. Sometimes they are also considered as transversal or instrumental competencies. Most university degrees include them, though with different predominance. Among them we can find the capacity to analyse and summarise, apply theory to practice, capacity to learn, work as part of a team, etc. Specific competencies are related to different disciplines or areas of knowledge, and they are in the process of being established by the different universities. They consist of those competencies that they would like to develop in their students (Pérez,

2009).

Despite current criticisms and problematic aspects on the notion of general competencies (see Westera, 2001; Pérez, 2009) this notion has become the definitive educational referent, it is being used in the establishment of all the new degrees, and as such, our main challenge is not to evade it, but to set its limits in a operative way. In this paper we will consider general competencies, because they can be applied to any set of contents in different studies and they can be effectively measured with the evaluation instrument we propose, which in turn can be applied in different fields. Thus, general competencies, according to the Dublin descriptors 4 (Márquez, 2004; Montanero et al., 2006), include:

1. Understanding, analysing, relating, and expressing information:

understand or express meanings in different languages (verbal, mathematic, second language), understand information from advanced textbooks.

2. Planning, decision making, and problem-solving: generalize or apply knowledge to practical areas, design projects, apply methodological procedures and algorithms in different contexts.

3. Using skills, tools or technologies: apply physical skills; use technologies and equipment.

4. Learning autonomously, being able to investigate: search, code, summarize and interpret information from different sources; deduce information from cues or evidences; draw conclusions; judge, argument and analyse critically.

5. Thinking creatively and taking the initiative: put forward personal ideas, answer to improvised situations with some security.

6. Learning to communicate, cooperate: support ideas, opinions or products in public; lead or work in group; communicate with others in formal and informal contexts.

7. Other personal, social and emotional capacities: work constantly and

persevere; motivate oneself with long term aims; be optimistic; resist frustration; adapt to new situations; act tolerantly, fairly and sympathize. These are competencies which can be ascribed to all the disciplines (prospective doctors, teachers, engineers, etc. must acquire them), particularly the first two groups, although it is an open ended list, that is to say, some more competencies could be included. Here we have mentioned those reflected by the Dublin descriptors 5 . The remaining types of competencies, although they will not be considered in this paper, include general grade competencies, specific grade competencies and specific competencies of the subjects/contents in a grade. All of them belong to particular fields or disciplines and can be consulted in TUNING

Project documentation and in the Libros Blancos 6 designed for the different profiles and grades.

A competence-based constructivist tool for evaluation / M.-M. Ramos-Álvarez and G. Luque

335

Methodology involved in the constructivist view

Traditionally, university teaching approaches have focused on the teacher developing a list of contents while the students listen and take notes (lectures). However, in a learner-centred approach which aims to achieve different competencies, both academic and professional, there are certain methodological changes. One of the most outstanding ones is the change of the teacher’s role (Acevedo, 1996; Manassero, Vázquez & Acevedo, 2004). Without abandoning

traditional but sometimes useful activities such as large group lectures, the teacher

is now expected to include alternative teaching activities, where the students’

presence is –or is not– expected, which can be practical or theoretical, and which include different groupings and different learning contexts. Among these activities we can find:

a. Coordination and evaluation, where the teacher introduces the contents and aims of the subject and of each activity, evaluates background knowledge and organizes the students’ learning tasks (related to Competence 1, in previous section).

b. Verbal exposition, where the teacher elaborates the information, solves problems, orientates towards the use of bibliographical references (Competence 1).

c. Discussions and debates, where the teacher acts as a moderator and helps draw conclusions (Competence 4).

d. Problem-solving and case studies, where the teacher shows particular cases or problems, analyses them, justifies the use of different strategies, and checks their solution (Competence 3).

e. Oral/written presentations carried out by students, with the guidance of the teacher and his/her feedback (Competencies 4, 5, & 7).

f. Practical work carried out by the students but organized by the teacher, such

as research projects, laboratory and field practices, cultural visits, tutorials, workshops (Competencies 2, 3, & 4).

g. Other autonomous learning activities guided by the teacher through group or individual tutorials (Competence 4).

As we can see here, each type of activity tends to develop one or more of the competencies described in the previous section. This list of teaching activities is not complete, and could be extended to include other tasks which are useful for achieving other general and specific

competencies. However, introducing one or many different activities is not just

a question of fulfilling an ECTS (European Credits System) framework: it

depends on the type of class, the number of students and the resources available, the cognitive style of the learners, the one of the teacher, and of course, the subject being taught within a specific grade within a particular university. However, it would be unfair, to say the least, to carry out these activities, or some of them, which are aimed to develop specific competencies, and involve a huge amount of constant work on the part of both teacher and learners, and then evaluate students with a single final test of factual knowledge, leaving aside the context, the competencies, and, in general, the learners’ features.

Evaluation

Teaching and learning outcomes –in the form of competencies– require necessarily to be evaluated following the same constructivist framework

(Moseley et al., 2005; Wakeford, 2003). The first part of this section deals with

a comprehensive notion of evaluation as it is considered in this paper, whereas in

336 Cultura y Educación, 2010, 22 (2), pp. 329-344

the second part we provide and justify a tool which can be used for this type of evaluation, but also to guide other teaching and learning processes.

A Constructivist View of Evaluation

Evaluation practices within the constructivist framework are different from other procedures in which testing contexts are designed to reduce social influences. In fact, this framework contemplates what has been termed dynamic assessment 7 (Palincsar, 1998), an approach in which the performance of the individual assessed is guided by another individual to determine the individual’s potential to profit from assistance or instruction. It is, then, a prospective measure of performance, and is predictive of how the learner will perform independently in the future. There are several models of dynamic assessment, and they vary in terms of the nature of the task, the type of assistance provided and the outcomes reported. Specifically in connection with the university educational system, dynamic assessment, which, not leaving out XX century procedures (i.e. paper-and- pencil tests, with short or long answers, theoretical problems, etc.), aims at measuring the individual performance –going beyond knowledge– through different instruments/procedures (i.e. case studies, research projects, learning dossiers, self and peer-evaluation, real problems and professional practices). This kind of assessment is more comprehensible than traditional procedures, and has the potential to evaluate new contents, a wider range of competencies and is more consequent with the role of learners in the new university system 8 .

A Tool for Evaluating through the competencies

Within a meaningful portfolio evaluation (or, from a postmodern constructivist view, dynamic assessment) which seeks to measure the learning outcomes of competencies, an alternative or complement to traditional tests is the oral/written presentation, either in group or individual. Presentations are not only a form of evaluation, but a learning activity in which students are given the responsibility of developing specific contents or topics that constitute part of the subject they are learning. Presentations offer a number of advantages for learning:

• They help develop important general competencies connected to investigating and solving problems (competence 2, see section 3), learning autonomously, investigating (competence 3), and learning to communicate and cooperate (competence 6). • The notions, topics or contents presented and organized by learners themselves are remembered and recalled better than those introduced by others. • The students take part in the teaching process and are expected to adopt a higher level of responsibility. • Presenting practical or theoretical information in front of a public audience is a professional competence most learners will need to use in their future careers (i.e. health, teaching, business and management…). However, evaluating oral presentations is, most of the times, an inaccurate task in which the teacher just gives a mark based on a general impression, without following any reliable criteria. The tool below (see Appendix), constitutes an instrument aimed at objectively measuring both written and oral presentations. There are yet few tools in the literature for the evaluation of the competences –as learning outcomes–, but some examples can be found in Vallverdú (2009a, 2009b), who shows a tool for evaluating how students can

A competence-based constructivist tool for evaluation / M.-M. Ramos-Álvarez and G. Luque

337

teach using oral presentations (Vallverdú, 2009a) and another one to evaluate beginners’ research reports (Vallverdú, 2009b). Both are very good examples, the first for evaluating oral issues and the second for searching and organizing written information and its critical analysis. These learning outcomes are also evaluated in the tool presented below. Other applications have a qualitative orientation (González, 2000, or Moallem, 2001), and constitute a general evaluation framework to be made operative in specific fields. They constitute a general guide, but not a concrete evaluation tool specifically focused on competencies 9 . However, our tool is more comprehensive and assesses other aspects, at the same time it operativizes, through the learning outcomes, the different features included in the notion of general competencies. It can also be used in multilingual classrooms, or in settings where content-based teaching is been introduced and English is being used as the lingua franca 10 . This tool includes four sections. Three of them (A, B, and C) are aimed at evaluating aspects concerning a specific subject (i.e. concepts, notions, strategies, problem-solving techniques, etc.), and, from a different perspective, they assess competencies (knowledge, skills –to know and to know how– and autonomous learning) (Westera, 2001). The items in this section have been elaborated so as to adequately consider all the competencies mentioned in

section 3. Besides, a detailed analysis of the processes involved in the elaboration

of oral and written presentations was carried out, including from more general

aspects to issues concerning contents introduced and form of presentation. The fourth section (D) –optional– addresses other competencies (attitudes) and should be implemented only when the evaluation process includes several teachers from the same area or field. It is also possible to use this fourth section

when evaluating end-of-grade dissertations or other presentations that involve coordination between several fields of work. As such, we have introduced section

D as a separate chart. Items from this section D have been considered taking

into account specialized psychological literature. It has been operativized in four different sets which include complementary psychological aspects (see terms such as Learning-to-learn, Communication Skills, Emotional Intelligence, and Leadership, on Strickland’s Encyclopedia, 2001). The first three sections receive different punctuations as a function of their relevance for the global measurement of the students’ (oral/written) presentations. The first part (A) deals with aspects related to final deadline and extension of the paper. These are formal considerations that must be taken into account when students participate in the teaching process and have to coordinate their presentations with the rest of the participants. This part only receives a 10 per cent of the total mark, as most of the competencies are assessed

in the remaining sections. The second part (B) is concerned with contents and

makes reference to which concepts have been included in the presentation, their

organization, analysis and relevance for the subject. It also deals with the ability

to look for relevant reference material, and its appropriate use. In short, it

assesses learners’ higher order cognitive skills and behaviours that represent the ability to cope with complex situations (in a semi-controlled context: the

classroom or laboratory). General competencies number 1, 2, 4 and 5 (see

section 3 above) can be assessed through this section, which covers a 60 per cent

of the mark, as most competencies can be evaluated through this section. The

third section (C) involves presentation aspects –layout and design– as unfortunately features such as grammar, spelling and writing style must be taken into account, because not all learners take care of these aspects. It also evaluates general competencies 3 and 6: to know how to explain knowledge, which

338 Cultura y Educación, 2010, 22 (2), pp. 329-344

is deemed as an essential competence in some fields of knowledge, such as psychology, teaching, etc. This section covers a 30 per cent of the mark; roughly following the same proportion of the competencies it assesses (two vs. four for the section B). The three parts are subdivided into both oral and written sections, with the same importance. For instance, within the grammar or the spelling section that typically deals with written aspects (C.1) we can also consider the oral version, because learners tend to use, for their oral presentations, some sort of written support, such as transparencies or power point slides, which may also show grammatical or spelling errors or mistakes. Administered at the beginning of the course, this tool can give feedback to the teacher –as evaluator–, to the teacher in connection with how effective teaching processes and learning systems are, and to the learners, who have time to contemplate and review what is to be expected of them in detail, so it turns into a form of formative evaluation.

Conclusions

University educational systems have turned to competencies as goals to be attained by students. Competencies are theoretical constructs involving psychological processes which include cognitive, emotional, motivational, social and behavioural components (Pérez, 2009). They are constructs very difficult to be evaluated because of its post hoc character, that is to say, competent performance presumes competence but you cannot determine its truth or untruth (Westera, 2001). Competencies can be general –for most fields of knowledge– or specific –for a specific degree in a particular university–. Traditional assessment procedures such as pen-and-pencil tests cannot assess learners’ knowledge of general competencies, which involve higher order cognitive skills such as planning, decision making, problem-solving, using skills, autonomous learning, creative and critical thinking, communication and cooperation, among others, and which are useful for both the university context and the labour market. In order to successfully carry out an assessment of learners’ competences (or successful performance) we need tools or instruments which are able not only to evaluate knowledge, but also assess how that knowledge is applied in different contexts. The tool we suggest here can assess students’ behaviours in semi- controlled situations (oral-written presentations). Thus, it constitutes a reliable tool for the observation of successful performance. In addition, it is also able to reveal how the different competences it evaluates are developed throughout time (different tutorials: see below) and hopefully, predict a similar behaviour in future complex situations or when the learner faces ill-defined problems in the future, following the notion of dynamic assessment within the constructivist framework. Therefore, this tool allows a more encompassing evaluation of different competencies because it sets up situations closer to what task demands in real life would require. Its division into sections also allows evaluating specific sub- competencies and skills, eliminating the problem of ambiguity in the formulation of competencies by means of the way in which the different sections include a detailed list of expected behaviours –learning outcomes–. In combination with other tools, it constitutes a very complete form of evaluation of competences. However, this tool is long, and it can constitute a very time-consuming form of evaluation because it is very exhaustive. In initial implementations, teachers

A competence-based constructivist tool for evaluation / M.-M. Ramos-Álvarez and G. Luque

339

might think that the evaluating process is as long or even longer than the learning process. However, positive aspects outweigh negative ones. Its regular use can partially overcome the time problem. Besides, some items can optionally be left out, depending on the type of presentation. Moreover, for the learners it is an invaluable tool, as reading and taking it into account might be a learning process in itself: it shows all the behaviours to be taken into account when carrying out oral-written presentations. Together with other measures, it can constitute part of a portfolio evaluation, particularly when it is used during the tutoring and portfolio process, where teacher and learner self-evaluate in every of the aspects considered in the tool and by means of a chart they can see their progress throughout the course. It is also dynamic in the constructivist sense, as it may predict how the learner will perform in the future in similar situations. The procedure and sequence we recommend here is:

Establish groups of up to four people, where all of them present some information. Theory:

–students read general contents and divide them among themselves –they prepare specific parts individually –group session of all theory contents Practice:

–group sessions for discussion and decision-making. How a class or therapeutic session would be carried out using ideas from research process and theory. Tutorials:

–tutorials constitute a very good opportunity to see the development of some of the competencies we mentioned in section 3, particularly competencies 5, 6 and 7. The teacher can help students during these sessions and even reformulate the concepts, reassign group work or diagnose expected problems in the oral presentation, which can be solved in advance.

Notes

1 In France, they are called as technological or constitutive abilities; in U.S.A. and Canada, basic skills; in Great Britain, core skills or industry specific standards; Australia considers them key competencies, whereas in Latin America the Ministerio de Educación (1998) has also included them.

2 See http://tuning.unideusto.org/tuningeu/

3 See http://www.reling.deusto.es/TUNINGProject/index.htm up to page 30.

4 See http://www.eua.be/fileadmin/user_upload/files/EUA1_documents/dublin_descriptors.pdf

5 See footnote 4.

6 See http://www.reling.deusto.es/TUNINGProject/index.htm and http://www.aneca.es/publicaciones/libros-blancos.aspx

7 Assessment and evaluation are usually considered as two different terms. Evaluation is the more global one, and it includes materials evaluation, the final outcome of students, the teachers success, and the adequacy of the program contents. The second term, assessment, refers to the reliable and accurate measure of the achievements of the students with tests specifically designed for this process. Assessment can be formative, continuous or summative. However, Palincsar (1998) has coined a new term, dynamic assessment, which has more to do with the notion of comprehensive evaluation than with assessment. In this paper

we are using the two words –evaluation and assessment- as synonyms, following Palincsar and to avoid repetition.

8 See http://www.aqu.cat/publicacions/guies_competencies/guia_humanitats_es.html

9 Some very good sites where evaluation sheets (rubrics) can be found, although they do not evaluate competences, are:

http://writing.colostate.edu/references/teaching/grading/pop2d.cfm. http.//www.missouri.edu/ ?pattonmd//rubrics.html 10 Many Spanish university grades are now introducing the English language for large group lectures because of the number of European (non-native) attendants. It is a form of Content-based teaching in which students are expected to learn specific content and the second language at the same time. Prestige constitutes an influential factor.

References

ACEVEDO, J. A. (1996). La formación del profesorado de Enseñanza Secundaria y la educación CTS. Una cuestión problemática (Electronic version). Revista Interuniversitaria de Formación del Profesorado, 26, 131-144.

340 Cultura y Educación, 2010, 22 (2), pp. 329-344

ANDERSON, J. R. (1985). Cognitive Psychology and its Implications. New York: Freeman. AUSUBEL, D. (1968). Educational Psychology: A Cognitive View. New York: Hold, Reinhart & Winston. BRUNER, J. (1983). Child’s Talk. New York: Norton. COOPER, P. A. (1993). Paradigm Shifts in Designed Instruction: From Behaviorism to Cognitivism to Constructivism. Educational Technology, 33 (5), 12-19. DE MENDOZA, R. A., DOMÍNGUEZ, R. D., & MARTÍN, E. (2002). Metodología docente utilizada por el profesorado universitario/ Teaching strategies used by University lecturers. Cultura y Educación, 14 (2), 177-186. ERTMER, P. A. & NEWBY, T. J. (1993). Behaviorism, Cognitivism, Constructivism: Comparing Critical Features from an Instructional Design Perspective. Performance Improvement Quarterly, 6 (4), 50-72. GONZÁLEZ, O. (2000). Evaluación Basada en Competencias. Revista de Investigación, 53, 11-31. Retrieved from

http://www.redes-cepalcala.org/inspector/DOCUMENTOS%20Y%20LIBROS/COMPETENCIAS/

EV%20BASADA%20EN%20COMPETENCIAS.pdf, December 7, 2009. JONASSEN, D. H., MAYES, T., & MCALEESE, R. (1993). A manifesto for a constructivist approach to technology in higher edu- cation. In T. M. Duffy, J. Lowyck, & D. H. Jonassen (Eds.), Designing Environments for Constructive Learning (pp. 231-248). Heidelberg: Springer-Verlag. KARAGIORGI, Y., & SYMEOU, L. (2005). Translating Constructivism into Instructional Design: Potential and Limitations. Educational Technology & Society, 8 (1), 17-27. MANASSERO, M., VÁZQUEZ, A., & ACEVEDO, J. A. (2004). Evaluación de las actitudes del profesorado respecto a los temas CTS: Nuevos avances metodológicos (Electronic version). Enseñanza de las Ciencias. Revista de Investigación y Experiencias Didácticas, 22 (2), 299-312. MÁRQUEZ, R. (2004) El sistema de créditos europeos. Su aplicación práctica a la titulación de Lingüística de la Universidad de Cádiz. Cádiz: Servicio de Publicaciones. MARTÍN, E. (1999). Estrategias de aprendizaje y asesoramiento psicopedagógico. In J. I. Pozo & C. Monereo (Eds.), El aprendizaje estratégico (pp. 339-357). Madrid: Aula XXI, Santillana. MINISTERIO DE EDUCACIÓN (1998). Currículo Básico Nacional. Programa de Estudio de Educación Básica. Caracas Segunda: Etapa. MOALLEM, M. (2001). Applying Constructivist and Objectivist Learning Theories in the Design of A Web-Based Course:

Implications for Practice. Educational Technology & Society, 4 (3), 113-125. MONEREO, C. & POZO, J. I. (2001). Competencias para sobrevivir en el siglo XXI. Cuadernos de Pedagogía, 298, 50-55. MONEREO, C. & POZO, J. I. (2007). Competencias para (con)vivir con el siglo XXI. Cuadernos de Pedagogía, 370, 12-18. MONTANERO, M., MATEOS, V. L., GÓMEZ, V., & ALEJO, R. (2006) Orientaciones para la elaboración del plan docente de una asignatura. Guía Extensa 2006. Universidad de Extremadura: Servicio de Publicaciones. MOSELEY, D., BAUMFIELD, V., ELLIOTT, J., GREGSON, M., HIGGINS, S., MILLER, J., & NEWTON, D. (2005). Frameworks for Thinking. A Handbook for Teaching and Learning. Cambridge: Cambridge University Press. PALINCSAR, A. S. (1998). Social constructivist perspectives on teaching and learning. Annual Review of Psychology, 49, 345-

375.

PEREZ, J. (2009). Competences in Language Teaching: From their Conceptualization to their Concretion in the Curriculum. In M. L. Pérez (Ed.), English Language Teaching in the European Credit Transfer System (pp. 93-110). Berlin: Peter Lang. PIAGET, J. (1970). The Child’s Conception of Movement and Speed. London: Routhledge & Kegan Paul. POZO, J. I. & MONEREO, C. (1999). El aprendizaje estratégico. Madrid: Aula XXI, Santillana. STRICKLAND, B. (Ed.) (2001). The Gale encyclopedia of psychology, 2nd edition. Detroit: Gale. VALLVERDÚ, J. (2009a). ‘Aprendiendo enseñando con el Power Point’. In G. Puigvert (Coord.). Guía para la Evaluación de Competencias en el Área de Humanidades. Barcelona: Agència per a la Qualitat del Sistema Universitary de Catalunya. Retrieved from http://www.aqu.cat/publicacions/guies_competencies/index_es.html, December 7, 2009. VALLVERDÚ, J. (2009b). ‘Redactemos un artículo’. In G. Puigvert (Coord.) Guía para la Evaluación de Competencias en el Área de Humanidades. Barcelona: Agència per a la Qualitat del Sistema Universitary de Catalunya. Retrieved from http://www.aqu.cat/publicacions/guies_competencies/index_es.html, December 7, 2009. VIGHNARAJAH, WONG, S. L., & KAMARIAH, A. B. (2008). The Shift in the Role of Teachers in the Learning Process. European Journal of Social Sciences. 7 (2), 33-41.

VYGOTSKY, L. (1962). Thought and Language. Cambridge, MA: The MIT Press. WAKEFORD, R. (2003). Principles of student assessment. In H. Fry, S. Ketteridge, & S. Marshall (2003, 2nd.Ed.), A Handbook for Teaching and Learning in Higher Education (pp 42-61). London: Kogan Page. WESTERA, W. (2001). Competences in education: a confusion of tongues. Curriculum Studies, 33, 75-88.

Appendix

Evaluation sheet: Presentations & Projects

Title:

Authors:

Section presented:

Student:

 
 

Date:

Development

Initial tutorial session

Midway tutorial session

Final tutorial: results

A. General:

A competence-based constructivist tool for evaluation / M.-M. Ramos-Álvarez and G. Luque

341

A.1. Final deadline (i.e. a month).

Written

A.2. Extension of paper within allocated time (i.e. 30 pages & 1session for presentation).

time (i.e. 30 pages & 1session for presentation). A.3. Group behaviour & organizational skills: •

A.3. Group behaviour & organizational skills:

Disconnected and inconsistent material vs. coordination within all the sections and participants.

During tutorials, teacher appreciates that one or some learners work in an isolated way, without coordination with other participants vs. ability to work in (optionally heterogeneous) groups, ability to organize, manage, plan, solve problems, make decisions, and form and develop adequate discussion groups.

A.4. Attitudes, values and ethical commitment:

• Academic-scientific attitudes related to research abilities: learner accepts information not supported with data, does not question imparted thoughts and beliefs and accepts as true one single conceptual or methodological approximation vs evidence-based, critical attitude towards external information or universally accepted truths and beliefs, & search for multi-layered knowledge. Also, learner acknowledges theoretical-conceptual and methodological-technical diversity.

• Applied attitudes following deontological or ethical considerations and depending on the content, field of study: i.e. prospective teachers should be respectful and value multiculturalism and diversity.

MEAN (A)

Oral

field of study: i.e. prospective teachers should be respectful and value multiculturalism and diversity. M EAN

B. Contents:

B.1. Originality & Creativity:

Literal copying of handbook versus integration of different materials.

Material is a copy, reproduction or imitation vs. original material, with learner’s peculiar style, and with ability to generate new ideas by means of the integration of acquired concepts.

Written

B.2. Complete & homogeneous (adequacy to the conceptual map handed out at the beginning of the process):

Loose, disconnected parts vs. conscientious treatment of contents. Concepts are unevenly handled vs. consistent treatment of theories, ideas, models, etc.

In a homogeneous way. Fragmented, heterogeneous and sloppy handling of contents vs. exhaustive treatment.

B.3. Depth of understanding-insight (also adequacy to the conceptual map):

Superficial description of contents, regardless of their difficulty or complexity vs. contents/concepts rendered in a detailed way even going beyond what was expected.

in a detailed way even going beyond what was expected. Oral B.4. Synthesis-comprehension-selection and analysis of
in a detailed way even going beyond what was expected. Oral B.4. Synthesis-comprehension-selection and analysis of

Oral

in a detailed way even going beyond what was expected. Oral B.4. Synthesis-comprehension-selection and analysis of

B.4. Synthesis-comprehension-selection and analysis of relevant aspects:

Copy or literal reading versus selection of key concepts according to their relevance. Evidence of synthesis and reorganization of concepts as a result of an analysis in depth of the material versus literal copy and paste of disconnected material showing lack of comprehension (evidence of false friends, cognates, badly translated words, conceptual or formal errors).

B.5. Multi-level assimilation of concepts and fundamental knowledge:

• Specific: learner lacks knowledge of content vs. masters knowledge of concepts regarding contents subject, discipline, and grade. Concepts explained previously either by teacher or other classmates’ presentations.

342 Cultura y Educación, 2010, 22 (2), pp. 329-344

• General: learner lacks contents knowledge vs. masters knowledge of concepts and notions from the same field or area of knowledge, of the grade, or even of other related grades (i.e. concepts from developmental psychology for prospective teachers when considering handicapped children).

B.6. Personal appraisal & critical attitude & reflection:

Listing of opinions, theories, techniques or ideas without awareness of their distinguishing features vs. critical commentary and comparison of inconsistencies, comparing and contrasting different theories or points of view (showing advantages and drawbacks).

B.7. Use of documented sources and references (relevance and extension):

Use of textbook(s) proposed by teacher versus personal search and selection of relevant references. Use of commented references and their relevance for the different sections of the presentation.

In general, references show several strategies:

Students use an adequate search and selection strategies of materials (data bases, web pages…). Students discern high and low quality material according to the objectives and contents of the presentation: read papers, books and lectures with scientific excellence. Students show their ability to read and understand one or several foreign languages in their search and comprehension of scientific bibliographic sources.

and comprehension of scientific bibliographic sources. • • B.9. Scientific rigour: • Poor language,

B.9. Scientific rigour:

Poor language, disregarding formal aspects, with incorrect references or lack of quotes, techniques are not sufficiently detailed, practical or empirical information is not considered adequately vs. the text or discourse is scientifically relevant, with selected quotes, treatment of formal aspects (i.e. formulas), and insertion of relevant techniques.

MEAN (B)

B.8. Relevance of methods and techniques (established previously through a step-by-step procedural map):

Incomplete treatment of practical cases, with loose and disconnected stages, or inadequate selection of techniques or methods (i.e. inadequate application of classroom dynamics) vs. knowledge of how to use techniques and methods adequate and relevant for the subject matter, but applying it to new practical cases or situations.

Some of these procedures require knowledge of specific computer programs (i.e. AutoCAD; Hot potatoes).

C. Design and presentation:

C.1. Layout:

Handwriting, spelling, margins, spaces, numbering, text quality, bibliographical references & quoting.

C.2. Accuracy in expression and writing style:

Written

Spelling mistakes and overabundance of expressions and structures directly translated from another language (i.e. passive voice from English) vs. cohesion and coherence, spelling, usage of specific vocabulary.

Pronunciation, clarity, accuracy, level of oral competence.

C.3. Fluency:

Literal reading of a complete text vs. fluid discourse, able to accommodate to different audiences, based on brief notes.

to accommodate to different audiences, based on brief notes. C.4. Explanation – presentation – methodology:

C.4. Explanation – presentation – methodology: (methodological adequacy and coherence about contents):

notes. C.4. Explanation – presentation – methodology: (methodological adequacy and coherence about contents): Oral
notes. C.4. Explanation – presentation – methodology: (methodological adequacy and coherence about contents): Oral

Oral

A competence-based constructivist tool for evaluation / M.-M. Ramos-Álvarez and G. Luque

343

In general. Lineal discourse which is not adapted to the relative complexity of presented contents, with ambiguities and repetitions vs. clear and accurate style, relevant and concrete, with an adequate explanation of important parts or complex concepts. There is a regular summary of contents.

Specifically, two aspects are considered:

Inclusion of examples. Literal discourse lacking examples, monotonous and boring vs. Discourse which explains or illustrates complex aspects such as formulae or techniques with examples. Usage and adequacy of support material (diagrams, illustrations, anagrams, etc.). Usage of inappropriate material, with condensed-cramped transparencies, difficult to read, copied literally from the written presentation or introduced at an inadequate moment of the session vs. material including hand-outs, slides, summaries, tables, charts etc. which is easy to read and understand, well balanced and summarized (with sequenced lists, slides that are not cramped with information) and employed at adequate moments in the oral presentation.

C.5. Coherence and integration of all the parts-sections of the work:

Coherent and continuous text versus telegraphic outlines, preserve continuity among sections, come back to nuclear ideas, introduce contents to classmates, locate sections within a general outline, etc.

The reader or listener should follow presented ideas with fluency rather than ‘jump’ from one point of view or idea to another one with no apparent connection.

C.6. Adequacy to guidance hand-out (i.e. different sections in an applied APA report):

The presentation shows it meets the sections of the prearranged hand-out with an appropriate organization.

C.7. Adequacy to the structure of a research Project (i.e. structure of a scientific APA report.):

Including the following sections: 1º) index of contents, 2º) theoretical framework –state of the art research–, 3º) methodology, 4º) data, 5º) discussion, and 6º) references.

MEAN (C)

framework –state of the art research–, 3º) methodology, 4º) data, 5º) discussion, and 6º) references. M
framework –state of the art research–, 3º) methodology, 4º) data, 5º) discussion, and 6º) references. M
framework –state of the art research–, 3º) methodology, 4º) data, 5º) discussion, and 6º) references. M

Mark= A –0.10– + B –0.60– + C –0.30–

M EAN (C) Mark= A –0.10– + B –0.60– + C –0.30– General Comment: Grade: Fails
M EAN (C) Mark= A –0.10– + B –0.60– + C –0.30– General Comment: Grade: Fails

General Comment:

Grade:

Fails (D)

Pass es (C)

B

A

0

0,25

0,5

0,75

1

Grade: Fails (D) Pass es (C) B A 0 0,25 0,5 0,75 1
Grade: Fails (D) Pass es (C) B A 0 0,25 0,5 0,75 1

344 Cultura y Educación, 2010, 22 (2), pp. 329-344

D. Psychological aspects related to general competencies:

D.1. Ability to learn and adapt to new situations:

The learner is unable to find new examples to apply acquired knowledge because he/she has not developed the ability to be autonomous; he/she is only able to memorize material explicitly handed out vs. the student is able to learn autonomously, he/she can transfer knowledge or techniques acquired in instructional settings to new situations and/or contexts, and is able to organize his/her time to learn autonomously.

D.2. Communicative ability & communication with Experts from other fields:

The learner cannot answer addressed questions repeating only memorised contents in a literal way vs. the student is able to express ideas, and makes him/herself understood. He/she shows the ability to listen, ponder and give an answer to the different addressees, defending the information he/she has acquired and imparted.

When the presentation has finished, he/she is not able to defend the position, theory and arguments he/she has presented, giving inappropriate answers or even omitting them vs. ability to answer all the questions, comprehension checks, expressing viewpoints or defending a theory/model.

D.3. Social Competence –Interpersonal/social abilities/ emotional intelligence–:

Learners show abilities concerning different levels:

– Elementary: listen, initiate and maintain a conversation, formulate questions, say thanks, express approval, introduce him/herself and others. – Advanced: ask for help, participate, follow instructions, apologize, and convince others. – Related to feelings: ability to pay attention, perceive and understand his/her own feelings and those of others –empathy–, being able to express those feelings and regulate them –both positive and negative aspects–, understand others, face their anger, express love, solve fear-related problems, give self-rewards. – Alternatives to aggression and hostile style: learners are able to ask for permission, share (i.e. material, information), help others, negotiate, possess self- control, defend his/her own rights, accept jokes and avoid problems and confrontations. – Coping with stress: learners formulate and answer grievances, show sportsmanship, overcome shame, adjust when left aside, defend friends, answer to persuasion and failure, show awareness of contradictory messages, answer accusations, face difficult conversations and group demands. – Planning: take the initiative, find out the cause of problems and know how to solve them according to their importance, establish and rank objectives, show awareness of his/her own abilities, gather information, concentrate on specific tasks, make decisions.

D.4. Ability for leadership:

Learners show the ability to plan, organize and guide group activities and projects by themselves.

They also possess a very high ability to work in group, an elevated social competence (see proceeding section) in connection with planning and organizing tasks.

High ability to develop strategies, coordinate and distribute tasks among group participants and guide them in achieving specific goals. He/she is able to establish a target, convince others to follow him/her and guide them in the achievement of that objective. That is to say, the learner has the ability to inspire others, influence them, make them trust him/her and anticipate results beyond the present moment (“have insight”).

MEAN (D)

make them trust him/her and anticipate results beyond the present moment (“have insight”). M EAN (D)

Written

Oral