Testing Spoken Language

Oral tests are qualitatively different from other kind of test because they do not easily fit the conventional assumptions about people and testing. Oral tests are more difficult to design, administer and mark Four basic kinds of Questions for test construction 1. 2. 3. 4. Proficiency: what is the learners general level of language ability. Placement: Where does this learner fit in our teaching program? Diagnosis: what are the learners particular strengths and weaknesses? Achievement: how much has the learner learnt from a particular course?

Elicitation techniques 1. Discussion/conversation The most natural thing between two people; however it is the hardest thing to make it happen in the framework of a language test. This occurs only when both parties are relaxed and confident, allowing the activity to become dominant. In practice, the success depends very much on the ability of the interviewer to create the right atmosphere. On the contrary, our inherited attitudes to tests, and the way they are usually conducted nhold learners away and prevent this positive atmosphere. Most of the time, only learners with high level of proficiency will feel confident enough to take the conversational initiative One simple rule for the facilitator is not to talk too muchbe prepared to leave short spaces or longer silences for the learner to decide what to say, or to think of the words to speak. 2. Oral Report The learner prepares and gives oral presentation. At the end of the presentation, the speaker is expected to answer querries. Making presentations is an authentic and communicative activity both for professionals and academic purposes. In a formal test procedure, the learners make the presentation directly to the interviewer. In a less formal situation, mini-presentations may be a routine part of the daily teaching schedule and may be used for testing purposes at the same time. Choosing a topic is very important, it should be relevant to the aims of the program. Ideally the topic should be chosen by the learner in consultation with is teacher who will help match the ability of the learner with the difficulty of a given topic. Marking: when giving presentations is an authentic activity for the learners, specific mark categories can be used for the different functional skills involved. When the presentation is given publicly, the assessor may back up her own impressions by watching the effect the speaker has on the listeners in terms of their comprehension, their reactions and their questions. Variations: 1. Making a mini-presentation with limited preparation time 2. Identifying a topic of personal interest 3. Learner joint discussion/decision making (group discussion/debate)

A group of two or more learners are tested together without the participation of an interviewer/.The learners have to maintain and direct their discussion entirely on their own. The task usually involves taking information from written documents and or different sources. Assessment is based on the way they express and justify their opinions, and evaluate those of others. The making could be live or taped for later marking. Marking 4. Role-play Candidates can be asked to assume a role in a particular situation. This allows the ready elicitation of other language functions. There can be a series of brief items, such as: A friend invites you to a party on an evening when you want to stay at home and watch the last episode of a television serial. Thank the friend (played by the tester) and refuse politely. Or there can be a more protracted exchange: You want your mother (played by the tester) to increase your pocket money. She is resistant to the idea. Try to make her change her mind. You want to fly from London to Paris on 13 March, returning a week later. Get all the information that you need in order to choose your flights from the travel agent (played by the tester). In my experience, however, where the aim is to elicit natural language and an attempt has been made to get the candidates to forget, to some extent at least, that they are being tested, role play can destroy this illusion. I have found that some candidates, rather than responding to the situation as if it were one they were actually facing, will resort to uttering half remembered snatches of exchanges once learned by rote. 5. Interview 6. Making appropriate responses The learner is given a number of short, unrelated situations that might occur in everyday life. He may be given the situations written out on a sheet of paper; the situations may be read out to him. The situations are designed to be easily understood, and require one to two sentences at most. They are designed to elicit functional language, for example, to ask for information, to apologize or to refuse an invitation politely. This technique aims to test his command of functional language correctly. Marking: Assessor must have a good command of the language who can assess the appropriateness of a response. Marks are awarded on a simple scale such as: --inappropriate or seriously incorrect --relevant but not entirely acceptable --appropriate and correct A prepared list of acceptable answers can be constructed from intuition and by trying each Example: Did you have any trouble finding your way here? (no, none at all) How are you? (Im thirty five) These are all authentic conversational openers and can be used as naturally as possible, in the introductory stage on any face-to-face oral tests. 7. Question and Answer

This typically consists of a series of disconnected questions which are graded in order of increasing difficulty. An experienced assessor can sometimes place a learners level more or less accurately on the basis of his answer to a question. Question and answer is a very common general purpose test technique, especially suitable for lower levels, and as an achievement test for learners who have been following a carefully graded structural syllabus. It is easy to choose and adopt questions to suit the level; if one question is not understood, the interviewer can move on to another. However, it is tester dominated and relatively unauthentic because there is no cumulative conversational meaning. Variations; 1. Using different question types Yes/No questions Simple factual questions Descriptive questions Narrative questions Speculative questions Hypothetical questions Etc, 8. Reading blank dialogue The learner is given a dialogue with only one part written. He has a few minutes to read it through and prepare the missing lines. This activity aims to give a learner a clear idea of functional meaning of the missing parts. Example: A. B. A. B. A. A. B. Can you tell me how to get to the station? _______________________________ _______________________________ The bus stop is just around the corner Oh yes, I can see it. ______________________ Im thirty one.

This technique is useful for getting the learner to ask questions. In the example, the learner must supply a suitable question rather than the usual pattern or interviewer asks, learner, answers. At a higher level, blank dialogues can be constructed using more complex functions such as excuse, polite disagreement, warning and others. The length of preparation time depends on the difficulty of the task. Variations: 1. Expressing personal opinions and attitudes at length. 2. Explaining job-related or academic terms

9. Using a picture or picture story Pictures Single pictures are particularly useful for eliciting descriptions. Series of pictures (or video sequences) form a natural basis for narration (the series of pictures on page 92 for example).

10. Giving instructions/description/explanation 11. Re-telling a story from aural stimulus

The learners hear a short passage or a recorded story. The learner then re-tell or summarize the passage/story. The instructions usually emphasize that it is the quality, rather than the quantity, of the retelling using his words that matter. This technique exercises the short-term storage and recall of information. Although the texts are short, they are clearly too long to be memorized. The task calls on the comprehensionprocessing production sequence. It is a mixed skill test for listening and speaking. Variations: 1. Discussing the context/content 2. Passing the message on 3. \ 12. Re-telling a story from written stimulus 13. Translating/ Interpreting Interpreting It is not intended that candidates should be able to act as interpreters (unless that is specified). However, simple interpreting tasks can test both production and comprehension in a controlled way. If there are two testers, one of the testers acts as a monolingual speaker of the candidates native language, the other as a monolingual speaker of the language being tested. Situations of the following kind can be set up: The native language speaker wants to invite a foreign visitor to his or her home for a meal. The candidate has to convey the invitation and act as an interpreter for the subsequent exchange. Comprehension can be assessed when the candidate attempts to convey what the visitor is saying, and indeed unless some such device is used, it is difficult to obtain sufficient information on candidates powers of comprehension. Production is tested when the candidate tries to convey the meaning of what the native speaker says.

14. Sentence Completion from written or aural stimulus 15. Sentence correction 16. Sentence transformation 17. Sentence repetition Proficiency Descriptions Accent 1. Pronunciation frequently unintelligible. 2. Frequent gross errors and a very heavy accent make understanding difficult, require frequent repetition. 3. Foreign accent requires concentrated listening, and mispronunciations lead to occasional misunderstanding and apparent errors in grammar or vocabulary. 4. Marked foreign accent and occasional mispronunciations which do not interfere with understanding. 5. No conspicuous mispronunciations, but would not be taken for a native speaker. 6. Native pronunciation, with no trace of foreign accent. Grammar 1. Grammar almost entirely inaccurate except in stock phrases. 2. Constant errors showing control of very few major patterns and frequently preventing communication. 3. Frequent errors showing some major patterns uncontrolled and causing occasional irritation and misunderstanding. 4. Occasional errors showing imperfect control of some patterns but no weakness that causes misunderstanding. 5. Few errors, with no patterns of failure. 6. No more than two errors during the interview. Vocabulary 1. Vocabulary inadequate for even the simplest conversation. 2. Vocabulary limited to basic personal and survival areas (time, food, transportation, family, etc.).

3. Choice of words sometimes inaccurate, limitations of vocabulary prevent discussion of some common professional and social topics. 4. Professional vocabulary adequate to discuss special interests; general vocabulary permits discussion of any non-technical subject with some circumlocutions. 5. Professional vocabulary broad and precise; general vocabulary adequate to cope with complex practical problems and varied social situations. 6. Vocabulary apparently as accurate and extensive as that of an educated native speaker. Fluency 1. Speech is so halting and fragmentary that conversation is virtually impossible. 2. Speech is very slow and uneven except for short or routine sentences. 3. Speech is frequently hesitant and jerky; sentences may be left uncompleted. 4. Speech is occasionally hesitant, with some unevenness caused by rephrasing and groping for words. 5. Speech is effortless and smooth, but perceptively non-native in speed and evenness. 6. Speech on all professional and general topics as effortless and smooth as a native speakers. Comprehension 1. Understands too little for the simplest type of conversation. 2. Understands only slow, very simple speech on common social and touristic topics; requires constant repetition and rephrasing. 3. Understands careful, somewhat simplified speech when engaged in a dialogue, but may require considerable repetition and rephrasing. 4. Understands quite well normal educated speech when engaged in a dialogue, but requires occasional repetition or rephrasing. 5. Understands everything in normal educated conversation except for very colloquial or low-frequency items, or exceptionally rapid or slurred speech. 6. Understands everything in both formal and colloquial speech to be expected of an educated native speaker. WEIGHTING TABLE 1 2 3 4 5 6 (A) Accent 0 1 2 2 3 4 _______ Grammar 6 12 18 24 30 36 _______ Vocabulary 4 8 12 16 20 24 _______ Fluency 2 4 6 8 10 12 _______ Comprehension 4 8 12 15 19 23 _______ Total _______ Note the relative weightings for the various components. Where analytic scales of this kind are used to the exclusion of holistic scales, the question arises (as with the testing of writing) as to what pattern of scores (for an individual candidate) should be regarded as satisfactory. This is really the same problem (though in a more obvious form) as the failure of individuals to fit holistic descriptions. Once again it is a matter of agreeing, on the basis of experience, what failures to reach the expected standard on particular parameters are acceptable. The advice on creating rating scales given in the previous chapter is equally relevant here: Calibrate the scale to be used Generally the same procedures are followed in calibrating speaking scales as were described for writing scales, with the obvious difference that video-recordings are used rather than pieces of written work. Train scorers (as opposed to interviewers) The training of interviewers has already been outlined. Where raters are used to score interviews without acting as interviewers themselves, or are involved in the rating of responses to audio- or video-recorded stimuli, the same methods can be used as for the training of raters of written work. Follow acceptable scoring procedures Again, the advice that one would want to offer here is very much the same as has already been given in the previous chapter. Perhaps the only addition to be made is that great care must be taken to ignore personal qualities of the candidates that are irrelevant to an assessment of their The total of the weighted scores is then looked up in the following table, which converts it into a rating on a scale 04+.

CONVERSION TABLE Score Rating Score Rating Score Rating 1625 0+ 4352 2 7382 3+ 2632 1 5362 2+ 8392 4 3342 1+ 6372 3 9399 4+ (Adams and Frith 1979: 358)