INTRODUCTION TO OUTCOME-BASED EDUCATION

Year 2012

LEARNING OUTCOME
At the end of this session, participants should be able to: describe the basic concept of outcome based assessment (C2) explain assessment terms used in outcome based education. (C2) describe various methods of assessment in accordance with the course and learning outcome.(C4, A2) design rubrics for assessing learning outcomes (C3,P2)
2

ASSESSMENT IS
a way of finding out what a person knows, understands and thinks, and what he or she can do.

the systematic collection and analysis of information to monitor and improve student learning.

is not a one-time event, but a dynamic on-going process.

3

OUTCOME BASED ASSESSMENT

Focuses on providing students with multiple opportunities to practice what they need to do, and sets the provision of feedback on that practice (formative). Eventually, students must demonstrate achievement (summative).
Currently, most assessment tasks encourage the reproduction of content, whereas we should be looking for the production of new knowledge (analyze, evaluate and create parts of Blooms top level cognitive processes).
4

OUTCOME BASED EDUCATION

OUTCOME BASED ASSESSMENT

PURPOSE OF ASSESSMENT
Assessment For Learning - constructive feedback
to be given to students for use in their future learning.

Assessment Of Learning - assessing students

achievements for grading and certification

Assessment As Learning - students are capable

of becoming independent in their learning
6

PRINCIPLES OF ASSESSMENT

VALIDITY

FLEXIBILITY

PRINCIPLES OF ASSESSMENT

REALIBILITY

FAIRNESS

ASSESSMENT LEVELS
Few years after Graduation 3 to 5 years Upon graduation

Programme Educational Objectives (PEO)

Programme Learning Outcomes (PLO)

Course Learning Outcomes (CLO)

Upon subject completion

Topic Learning Outcomes

Upon topic completion

8

REFLECTION 1
What do you do when your class performs poorly in quiz/test/practical work, etc?
Adjust the scores upward Grade on a curve Adjust the difficulty level of the next assessment Give advice on study strategies Provide more practice problems, review, etc Spend more time on the materials in the future Change your teaching materials or approach

Assessment Task
Assessment task refers to summative and formative assessment or criterion referred assessment. Examples are case study, critique, debate, demonstration, drawing, sketch, essay, exhibition, interview, journal, laboratory/practical, literature review, logbook, model, oral examination, peer review, presentation, portfolio, practicum, problem solving, projects, reflection, reports, self assessment, research paper, thesis and workshop

10

ASSESSMENT TERMS
Assessment Criteria

A list of requirements that describes how well a student is able to achieve the learning outcome in order to be awarded a particular grade. Read more: http://www.answers.com/topic/assessmentcriteria#ixzz1DjoBm8oF Continuous Assessment Continuous Assessment is an assessment that takes place over a period of time. In other words you will be assessed right through your learning process and not only after the learning process. By having continuous assessment you can track the improvement (if any) of the learner, you will be able to give more support and guidance and the learner will have more opportunities to improve. http://wiki.answers.com/Q/What_is_continuous_Assessment#ixzz1DjnPB HWy
11

ASSESSMENT TERMS
Evaluation A process that enables us to measure, compare, and judge the quality of students work, schools, or specific educational programs. Measurement The process by which attributes or dimensions of some physical objects, process or opinions are determined and carried out using questionnaires and standardized test etc.
12

DOMAINS OF EDUCATIONAL GOALS

Cognitive
The Head

Affective
The Heart

Psychomotor
The Hand

3H
13

Adapted from DrJJ. UiTM.

BLOOMS TAXONOMY
Learning outcomes
Cognitive
Evaluation (C6)

DOMAINS Higher order

Affective
Internalizing (A5) Organisation (A4) Valuing (A3) Responding (A2) Receiving (A1)

Psychomotor / skills
Origination(P7) Adaptation (P6) Complex Overt Response (P5) Mechanism (P4) Guided Response(P3) Set (P2) Perception (P1)

Synthesis (C5) Analysis (C4) Application (C3) Comprehension Lower order (C2) Knowledge (C1)

LEARNING DOMAINS
DOMAIN OF LEARNING OUTCOMES (MQF) GENERIC STUDENT ATTRIBUTES (MOHE)
CRITICAL THINKING & PROBLEM SOLVING SKILLS ENTREPRENEURSHIP SKILLS COMMUNICATION SKILLS

Knowledg e of Discipline Areas Information Management & Lifelong Learning Skills

Practical Skills

Social Skills & Responsibilitie s Values, Attitudes & Professionalis m Communication Leadership & Team Skills

Learning Outcomes

Managerial & Entrepreneur ial Skills

INFORMATION MANAGEMENT SKILLS & CONTINUOUS SKILLS

MORAL & PROFESSIONAL SKILLS TEAMWORK SKILLS LEADERSHIP SKILLS
15

Problem Solving & Scientific Skills

RELATIONSHIPS BETWEEN CLOS, PLOS, PAIS and COMPLIANCE TO THE STAKEHOLDERS

Regulatory /Professional Bodies KPT/MQA IHL requirements
STAKEHOLDERS PRORGRAME AIMS PROGRAMME LEARNING OUTCOMES COURSE LEARNING OUTCOMES

Industrires students .. etc

PAI-1

PAI-2

PAI-3

PAI-4

PAI-5 .. etc

PLO-1

PLO-2

PLO-3

PLO-4

PLO-5 .. etc

Subject CLO-1

Subject CLO-2

Subject CLO-3

Subject CLO-4

Subject CLO-5

Management support and commitment

16

FORMATIVE ASSESSMENT
Formative assessment is generally carried out throughout a course or project, in which the feedback from the learning activities are used to improve the process of learning. Formative assessment is also referred to as "educative assessment, to aid learning. In an educational setting, formative assessment might be a lecturer or the learner, providing feedback on a student's work, and this would not necessarily be used for grading purposes. Formative assessments are diagnostic in nature.
17

SUMMATIVE ASSESSMENT
Summative assessment is generally carried out at the end of a learning programme. Summative assessments are typically used to assign students a course grade. Summative assessments are evaluative and often in the form of a report for a third party e.g. the Head of Department or parents. when the cook tastes the soup, thats formative, when the guests taste the soup, thats summative. Albert Stake
18

DIAGNOSTIC ASSESSMENT
Diagnostic Assessment occurs at the beginning of the teaching/learning cycle. It involves making judgments as to how a student is performing against a predetermined set of criteria. Diagnostic assessment is also a strategy to prepare students for the course requirements. It can be used as a form of orientation or induction.

19

DIRECT ASSESSMENT
Direct assessment of learning: gathers evidence, based on students performance, which demonstrates the learning itself. Examples: quiz, test, final exam, assignments, projects, Final year project, PBL, Laboratory Experiments, Exit Interview etc

Indirect assessment of learning: gathers reflection about the learning or secondary evidence of its existence. Examples: industrial training, external examiner, alumni surveys, employer surveys, student exit survey etc.

20

DIRECT VS INDIRECT ASSESSMENT

DIRECT ASSESSMENT
Quizzes Tests Final Examination Assignments Projects Final Year Project Lab Experiments PBL/POPBL Portfolio

INDIRECT ASSESSMENT
Alumni survey Employer survey Student Surveys Exit Interviews with Graduates Report and minute of meeting with Industrial Advisor Report and minute of meeting with External Examiner Percentage of students who go on to undergraduate school Retention and Transfer Studies Job Placement Statistics

21

TYPES OF ASSESSMENT

PEER ASSESSMENT SELF ASSESSMENT

AUTHENTIC ASSESSMENT

22

NORM REFERENCED ASSESSMENT

Norm-referenced assessment compares students to that or those of a larger group. or "norm group" which is a national sample representing a wide and diverse cross-section of students. Students are assigned grades according to their standing relative to other students and it normally follows the normal curve (grading on the curve). NRA is usually used to sort students and not to measure achievement towards some criteria of performance For this reason, there are strong moves internationally to move towards CRA.
23

CRITERION-REFERENCED ASSESSMENT
In criterion-referenced assessment, students' performances are compared to a specific learning objective or performance standard and not to the performance of other students For CRA to be beneficial to learning, explicit criteria are essential which provide clear learning goals for students.

24

REFLECTION 2
The following are some of the issues in assessment. Discuss in pairs some of these issues.
1. 2. 3. 4. 5. 6. Overload of students and staff Too many assignments with the same deadline Insufficient time for students to do the assignments Insufficient time for staff to mark the assignments Inadequate or superficial feedback provided to students. Overuse of one mode of assessment such as written examinations, essays or closed problems

25

FEEDBACK

FEEDBACK

Learning

Assessment

Feedback is the bridge between learning and assessment

26

FEEDBACK
Some useful guidelines for giving feedback Return all assessment scripts to students Clear and precise comments to individual student s work Include both positive and critical comments Be constructive, give advice on how to improve
Be organised in giving feedback e.g. separate / structure feedback sheet in relation to

assessment criteria involved

Open discussion and clarification of common mistakes or misconceptions Appropriately identify good performance for students to learn from each other
27

RUBRICS

28

RUBRIC
A rubric is a set of criteria and standards for assessing students work or performance. . For each criterion and standard, a rubric defines the specific attributes that will be used to score or judge the students performance and to differentiate between different levels of performances. Scoring rubrics include one or more dimensions on which performance is rated, definitions and examples that illustrate the attribute(s) being measured and a rating scale for each dimension.

Dimensions are generally referred to as criteria, the rating scales as levels, and definitions as descriptors.

29

TYPES OF RUBRIC
Holistic rubrics provide a single score based on an overall impression of a students performance on a task.

Analytic rubrics provide specific feedback along several dimensions

General rubrics contain criteria that are general across tasks. Task specific rubrics are unique to a specific task.

30

HOLISTIC RUBRIC
Description Demonstrates complete understanding of the problem. All requirements of task are included in response.
Demonstrates considerable understanding of the problem. All requirements of task are included

Score 5
4

Demonstrates partial understanding of the problem. Most requirements of task are included.
Demonstrates little understanding of the problem. Many requirements of task are missing

3
2

Demonstrates no understanding of the problem

No response/task not attempted

1
0

31

ANALYTIC RUBRIC
Needs Improvement (1)
Clarity (Thesis The purpose of the supported by relevant student work is not information and well-defined. Central ideas.) ideas are not focused to support the thesis. Thoughts appear disconnected.
Organization (Sequencing of elements/ideas)

Developing (2)
The central purpose of the student work is identified. Ideas are generally focused in a way that supports the thesis.

Sufficient (3)

Above Average (4)

The central purpose The central purpose of the student work is of the student work is clear and ideas are clear and supporting almost always focused ideas always are in a way that supports always well-focused. the thesis. Relevant Details are relevant, details illustrate the enrich the work. authors ideas. Information and ideas Information and ideas Information and ideas Information and ideas are poorly sequenced are presented in an are presented in a are presented in a (the author jumps order that the logical sequence logical sequence around). The audience can follow which is followed by which flows naturally audience has difficulty with minimum the reader with little and is engaging to the following the thread difficulty. or no difficulty. audience. of thought.
32

ANALYTIC RUBRIC
Determines your performance levels. Examples of performance levels:Excellent/ Good/Moderate/Poor/ Very Poor or Excellent/Good/ Satisfactory/Needs Improvement Numbers/numerical scores 4/3/2/1

33

STEPS TO CREATE A RUBRIC

List the criteria that will be used to assess performance.

Determine your peformance levels. Examples of performance levels. Excellent/ Good/Moderate/Poor/ Very Poor or Excellent/Good/ Satisfactory/Needs Improvement .

Write a description for each performance level.

Evaluate and revise rubric as needed.

34

ASSESSMENT TASK

35

SELECT ASSESSMENT METHODS

No. 1 Category of Learning Outcome Thinking critically and making judgments Sample of Assessment Methods essay, report, journal , present a case for an interest group ,Book review (or article) for a particular journal and write a newspaper article group work, work-based problem, analyse a case and conference paper (or notes for a conference paper plus annotated bibliography)

2.

Solving problems and developing plans

3.

Performing procedures and Demonstration, Role Play, Make a video demonstrating techniques (write script and produce/make a video), Produce a Poster, Lab report Managing and developing oneself Journal , Portfolio, Learning Contract and Group work
36

4.

SELECT ASSESSMENT METHODS

No. 5 Category of Learning Outcome Accessing and managing information Sample of Assessment Methods Annotated bibliography , Project, Applied task ,Applied problem

6.

Demonstrating knowledge and understanding

Written examination, Oral examination, Essay , Report Short answer questions: True/False/ Multiple Choice Questions (paper-based or computer-aided-assessment)
Portfolio, Performance , Presentation, Projects Written presentation (essay, report, reflective paper etc.), Oral presentation. Group work, discussion/debate/role play, observation of real or simulated professional practice
37

7.

Designing, creating, performing Communicating

8.

ASSESSMENT TASKS
Objective Tests measure the learners ability to remember facts and figures as well as their comprehension of course materials. Common variations include multiple-choice (MCQ), true-false, and matching items.

38

ASSESSMENT TASKS
Case Studies describes typical issues or problems. It gives students the opportunity to place themselves in the position of the decision maker in a specific simulated situation. The scenarios are usually taken from real life and presented from the standpoint of the decision maker. A successful case is one which offers debate on alternative courses of action, rather than offering a single settled outcome or solution
39

ASSESSMENT TASKS
Essay Questions are the most commonly-used assessment methods in polytechnics. There are two main varieties:
Unstructured questions, also known as free-response questions. Students have maximum freedom for discussion. Structured or restricted-response questions. The student is given far less freedom to determine the nature and scope of the response. Often, the questions guide both the structure and the content of the expected response.
40

ASSESSMENT TASKS
Capstone Assessment is often in final courses or internships in a program. It provides an excellent opportunity to identify whether students have acquired the central knowledge and skills that are the carefully selected outcomes expected of the program. Capstone assessment may also want to assess broader skills important to our graduates such as problem solving, computer literacy, team work, communication skills, and even the reading of professional materials. (www.indstate.edu/ academicaffairs /assessment/capstone_assessmentintro.pp/)
41

ASSESSMENT TASKS
End-of-chapter problems are used to reinforce and apply some concepts and skills learned in the classroom. As an example when a mathematical procedure is presented in the classroom, students are given numerical practice problems to solve in order, after class. These numerical problems are usually oversimplified real-life problems calling for direct substitution of values into formulae.
42

ASSESSMENT TASKS
Reflective journal is a piece of writing which allows students to record thoughts and insights about their own learning experience. This can be writing about what and how they have learned and understood a topic. It can also be a review of their learning process, selfevaluation of their performance and planning for future learning based on past learning experience, etc
43

ASSESSMENT TASKS
Seminar Presentation. Student works individually, or in teams, to investigate a topic relevant to the course and present their findings in the form of a seminar. There are other variations of the seminar presentation.

44

ASSESSMENT TASKS
A portfolio is a collection of course-related work performed by the students. It is also a written reflection in which students evaluate their own learning.. Traditionally, portfolios are used to assess learning in the arts and humanities. However, they can be equally useful in just about any discipline.

45

ASSESSMENT TASKS
Projects are popular methods of assessment in Polytechnics today. Projects can be based on literature and/or empirical research on a relevant problem. They can also be applied projects solving real-life or simulated problems, producing technical prototypes, or preparing a business plan.

46

ASSESSMENT TASKS
Examinations are used primarily for grading or selection purposes. We call this activity, in educational terms, summative assessment. It can use many of the methods we discussed above. Since examinations can take on many variations, it is important to understand the range of possibilities in order to design examinations that are compatible both with outcome-based and student-centred education.
Adapted from http://www.polyu.edu.hk/obe/ Check also Enhanced Learning and Teaching, Hong Kong University of Science and Technology at http://celt.ust.hk/ideas/afl/MExam/index.html#see
47

CONCLUSION
The OBE implementation requires a lot of coordination in planning which involves a lot of assessment. A well designed quality management system is needed to ensure that our academic programme , delivery system, assessing methods and our graduates are at high quality. A framework of continuous improvement with the involvement of the academic staff, external examiner, students participation and other stakeholders will probably contribute to the successful implementation of outcome-based assessment in polytechnics.
48

ACTIVITIES

49

ACTIVITY 1
Designing Test Items. This exercise offers you some experience in constructing test items. By referring to the JSP and JSI, outline the CLO to be covered by the test. Use the Exam Mapping Table as a guide to construct the test items.

50

EXAM MAPPING TABLE

Exam Content
Question 1 a. b. c.

Type CLO (Essay/Calc /Sketch)

PLO

Bloom Taxonomy

Question 2 a. b. c.

51

ACTIVITY 2
Think of your favourite assessment when you were an undergraduate. In groups of three or four, discuss: What makes a good assessment from your point of view?

What does it reveal about the tasks that you like doing? Solving problems? Learning, discovering, finding out, researching? Reading? Writing?

52

ACTIVITY 3
Please refer to the handout given to you.

53

ACTIVITY 4
The first step when creating a rubric is to determine what you want students to learn from the assignment. In a groups of 4 , create a Task Specific Rubric for student assessment.

54

55