18th July 2017 1:00
By Blue Tutors
Bypassing the increasing dissatisfaction and eroding confidence in the content, style and reliability of exam papers, there appears to be a desire for more transparency and scrutiny of the process of allocating and checking marks. With the mark received having potentially life changing consequences, from school choice to university admission, we would like to believe the process is fair and accurate. Think again.
The first thing to grasp is the scale of the task at hand. With over 15 million papers across hundreds of subjects at GCSE and A level, a huge number of man hours are required to get scripts marked in the space of just a few weeks; and man hours are expensive. Exam marking is worth millions of pounds a year and is looking for ways to cut this cost wherever possible. This has led to an increase in the type of questions that can be marked automatically, such as multiple choice of ‘circle the answer’.
But for those questions which require a human being, who are these examiners? The requirements to become an examiner vary wildly between examination boards. For IB, you need a minimum of a university degree in the subject, a teaching qualification and at least one year of teaching experience within the last three years. This is fairly standard for major examination bodies but there are others that are far less demanding. The truth is that the vast majority of script markers are doing it for some extra cash, not because of their love for the subject or exam fairness, or even to improve as a teacher.
Marking requires one main thing: speed. In the peak marking season in July markers can be asked to mark 500 scripts in the space of three weeks; that means 15 minutes on each script if you’re lucky, or just a few seconds per mark available. Equally, your examiner could be anywhere in the world; to prevent having to ship the many question papers around, most marking is now done online. The majority of examiners are still full-time teachers, meaning they are doing this in their ‘spare time’ at the end of the school year alongside their full-time teaching commitments. The standard in the industry is to be paid per script, usually around £4 each, meaning taking extra time over marking is at your own expensive. In order to remain fast and consistent, examiners must stick rigorously to the marks scheme. There is little space for spending time trying to work out what a candidate might be trying to say. Ironically, the better the candidate, the harder it is to mark; it takes much longer to read the answers of a fully answered paper than a half-completed one and the allocated time is just the same.
Some subjects and question styles requiring more discretion and subjectivity than others, yet allocating marks needs to be done consistently across examiners. The process to regulate this is called standardising, where potential examiners are given pre-marked scripts to mark to ensure they come out with the same scores. But, senior examiners have revealed, as the deadlines draw closer, and chief examiners become more desperate to get scripts marked, the quality of examiners allowed to mark scripts, the rigour of standardising, and the time taken per script plummet. Exam boards have even publicly admitted in 2015 to “guestimating” marks based on predicted marks.
So is it just luck of the drawer? Theoretically no, moderation is used to adjust for the generosity or ‘meanness’ of a specific examiner as scores are normalised across the entire population that took the exam. However, this assumes a consistent offset for each examiner and that they have paid the same attention to each script, from the first script they mark on a Monday morning to the ones they cram in on a Friday night or the last of many hours back to back on a Sunday. The biggest issue is those who have inconsistent marking, sometimes too harsh, sometimes too lenient, as there is no way other than remarking to correct for this.
Ultimately it is not the exact number of points gained but the grade awarded that matters. Moderated marks are amalgamated to make a distribution of marks in order to set the grade boundaries. Papers are supposed to be the same difficulty each year but in practice this is very difficult so the number of marks needed to attain a specific grade will vary slightly year on year. This entire process needs to be done in time for marks to be published. In recent years more and more marks have been challenged and requests for remarks are rising rapidly. But with exam boards charging for remarks at £40 per script, and examiners being paid 10% of that, is there much of an incentive to get it right first time?
Isn’t someone overseeing this? Yes, Ofqual is the quango tasked with ensuring that marking and levels are consistent between years not only within each exam board but between supposedly equivalent qualifications, for example A level OCR and Edexcel psychology should be of the same level of difficulty. Examiners and teachers alike argue that their broad, heavily statistical approach to the comparison only aggravates the situation as boards have a strict deadline to give marks to Ofqual or they face penalties. This puts even more time pressure on examiners to move fast.
The moral of the story is, if you think the mark is wrong, ask for a remark. Some examiners are committed, rigorous and have genuine respect for the subject, process and individual candidate; unfortunately, others don’t so if the mark looks wrong it is often worth double checking.