The Institution of Structural Engineers

AN OVERVIEW OF THE INSTITUTIONS EXAMINATION PROCESS

This paper describes the procedures that are used by the Institution in setting the AssociateMember (AM) and the Chartered Membership (Part 3) examinations and assessing the performance of candidates in such examinations. These procedures have evolved over a period of many years as a result of the ongoing review carried out by the Institutions Examinations Panel. The Panel consists of Corporate Members of the Institution who have served as Chief Examiners or who have otherwise gained extensive experience of assessment procedures and the Institutions examinations. The ongoing review process and the setting and marking of the Institutions examinations is led by the Examinations Panel and is administered by the Institutions Examinations Officer who also serves as Secretary to the Panel. The Examinations Panel is responsible for developing examination policy and for providing guidance to candidates on all aspects of the examination process. The Review Process The Examinations Panel meets at least 4 times each year to review all aspects of the Institutions examinations. The Panel reports formally to the Institutions Membership Committee which, in turn, reports to the Council of the Institution. In addition, the Panel meets with the Chief Examiners and the Marking Examiners on a regular basis to identify examples of best practice, to discuss improvements to any aspect of the examination process, to seek feedback on any draft proposals for change and to assist with the training of newly appointed Examiners. The committee structure not only considers and approves proposals for change made by the Panel but also provides guidance on how matters of Institution policy will affect the examination process. Setting the Questions Each question of each examination is set by a Chief Examiner. A Chief Examiner is always either a Member or Fellow of the Institution who has previously gained several years experience as a Marking Examiner (see below). The first draft of each question is considered in detail by the Examinations Panel. The Panel reviews each question to ensure the following:

A level of technical challenge that is appropriate to each examination; Accuracy and clarity; Consistency of style.

The development of each examination paper is co-ordinated by a member of the Examinations Panel. It is the responsibility of each examination co-ordinator to develop the final draft version of the paper taking into account the views of the other members of the Examinations Panel and each Chief Examiner following the revisions to the first draft. The final draft of each paper is approved by the Chairman of the Examinations Panel prior to printing. A further check of the final printed version of each examination paper is carried out by the examinations co-ordinator prior to the examination date.

Marking Each examination script is marked by two independent Marking Examiners. Marking Examiners are Corporate Members of the Institution with extensive design experience in one or more of the specialist subject areas covered by the examinations such as concrete, masonry, steel and timber building structures; bridges and offshore structures. The Institution uses an anonymous marking system; candidates are only identified by a reference number until the marking process is completed. Approximately 1 month after the examination date, a trial marking meeting is held. This is attended by members of the Examinations Panel, the Chief Examiners and the Marking Examiners. Following an initial briefing by the Chairman of the Examinations Panel, trial marking of each question is carried out by the Marking Examiners. This is led by the Chief Examiner for each question who produces marking guidance for each question. Even though the Marking Examiners for each question are marking to the same marking scheme identified by the Chief Examiner, it is inevitable that differences in marking will occur. To take account of this the marks awarded by each Marking Examiner for each candidate are adjusted statistically to produce a set of normalised marks (normalisation of marks is a standard technique that has been used for many years within the UK examination system). The normalisation process adjusts the mean and standard deviation of the marks awarded by each Marking Examiner to be the same as the mean and standard deviation of the marks awarded by the entire group of Marking Examiners for that question. Hence each candidate is awarded 4 marks a raw mark and a normalised mark from each of the two Marking Examiners. Approximately 3 months after the examination date, the 4 marks for each candidate are considered in detail by the Examinations Panel. The answer of any candidate that is judged to be a borderline pass or fail is identified by the Panel for adjudication by the appropriate Chief Examiner. The Chief Examiner reviews, in detail, the answer paper of each candidate that has been identified for adjudication and decides if the candidate should pass or fail. Advice is sought from the Examinations Panel when considered necessary. Throughout this process the anonymity of all the candidates is maintained. Hence, the performance of each candidate is considered by two independent Marking Examiners and any borderline candidates are considered further by the Chief Examiner. Candidates with extenuating circumstances are also considered during the adjudication process. On completion of the adjudication process, the Examinations Panel and Chief Examiners make recommendations for the award of prizes. Quality Assurance The Examinations Panel has developed a thorough quality assurance procedure for all stages of the examination process. The procedure is divided into 2 sections covering the setting of the examinations and the assessment. All aspects of the examination process are subject to independent external audit.

Feedback
Feedback is given to unsuccessful candidates (taken from a standard list and in the form of sentences) which best summarises candidate performance. Additionally, marking examiners receive feedback with regard to their marking performance. The feedback given to candidates and marking examiners is constantly under review by the Examinations Panel. Appeals The Institution has developed an Examination Appeals procedure. Examinations Panel The Institution of Structural Engineers July 2001.