Let’s face it, exams are dodgy: an inadequate measuring stick which we all feel free to refer to, and then to devalue where necessary. “He got a First, but he couldn’t boil an egg,” we say. ” He/she got A stars , but has no social skills/can’t speak a coherent sentence/ doesn’t know which day of the week it is.” And so on. What an examination actually measures is mainly the ability to pass that examination. Most have little predictive accuracy, and are produced by an inherently fragile system. If you want some insight into the way they are currently being used, with “grade inflation” and all the surrounding mystification, read a very concise and cogent recent piece in the Guardian by Peter Wilby (‘Gove is stuck in the past. We must look beyond grades. ‘ Mon 27 Aug 2012 http://gu.com/p/3a264/em ).
Back in the ‘Sixties I was an A level examiner in English literature. Examinations have to be marked, and this work was done mainly by teachers trying to earn a few honest bob for their holidays: drudges in a form of legitimate moonlighting.
The parcels of scripts to be marked would thud on the floor: I reckoned that to keep up I had to mark 15 a day, at about fifteen minutes per script. If I went out for the evening there would be 30 to mark the next day. Girls wrote longer scripts, conscientiously retailing what they had been taught, but with better handwriting than the boys. The scripts came in alphabetical order. By the time you had read thirty broadly similar answers to the questions on the set books you were fairly tired of that set of responses so it was a relief to open a new packet and to read something different – though of course the novelty again wore off after few more papers. In consequence, when I wrote an article about the GCE system (The Teacher Aug 1969) I noted, among other things, that people surnamed Aardvark were likely to get rather better marks than those called Zygo, just because their answers would feel fresher. This was only part of an argument to show examinations as fallible measures, but it came out at the Silly Season, so the Papers siezed on it, there were headlines, and I was roundly denounced in the Telegraph’s letter column.
I said thirty broadly similar answers. What school candidates did was to present to the examiner what they had been taught to say, as a sort of package. They may even have absorbed this answer whole, as a model reply to a question cunningly anticipated by their teacher in his/her role of Examination Technique Coach. Candidates didn’t usually hazard their own views, though from an examiners’ point of view it would have been very interesting to read original personal responses. The aim however was, and I think still is, to cuddle up to the norm.
One part of the system that can obviously go wrong is an aberrant teacher. We had one set of papers from students who conscientiously reproduced what they had been taught about the set books: all the examiners thought these particular responses bizarre and entirely wrong: the candidates (whose fault it wasn’t) had to be marked down, and the school quietly warned about the relevant teacher’s strange views.
But it is not just teachers who can be aberrant, or old, out of touch or eccentric. An examiner can have a breakdown, get drunk continuously, or otherwise go off the rails. When I took Finals at Oxford, the literary theory paper was marked very harshly: it was assumed that the examiner had been having a bad time, poor chap. The exam Board could take note of this, without distressing fuss – Oxford was a small community.
At school level, dealing with vast numbers, the ranges of examiners’ marks were monitored and adjusted: if the curve of one examiner’s marks was well below the average, that curve would be shunted towards the norm (although, what if that examiner really did receive sets of papers worse than the rest?). If one examiner’s marks were wildly different, someone had to re-read and remark the scripts, and something was lost in the process. A kind of uniformity was preserved, but precariously. All this was done in a hurry: the results had to come out by the deadline.
Candidates can obviously experience the same distortions of their performance as teachers or examiners (or, if you like, cricketers or tennis players). The one-off examination holds up an untypical slice of a candidate’s behaviour on one particular occasion. If the candidate happens to be ill, or sorely troubled, on that one occasion, well, tough. Retake the exam or forget about it.
So far as utility goes, the skills involved in a one-off exam are not necessarily transferable. The fact that you can successfully write about some work of literature in an exam does not mean that you can write a good newspaper article, or an editorial, or a formal letter, or that you are articulate, or that you can effectively conduct business on the phone. In fact the very form of teaching and examination may well militate against the development of other skills. Richard Kostelanetz long ago pointed to the way that English reviewers were equipped to write prose pieces of around 2000 words. ..”English writers, when given the opportunity, are rarely able to do a well-organised, balanced, 5000 word essay,” by which he meant the kind of article one admires in, say, New York Review of Books or the New Yorker (‘A Critical Look at the Critics’, Twentieth Century Magazine, Spring 1966). He attributed this shortcoming to the nature of British literary journalism, then overwhelmingly Oxbridge dominated, but of course the breath-length of that writing had its roots in the Oxbridge weekly essay, read to the tutor. Nobody could have got away with reading out 5.000 words.
The main function of exams, or nowadays exam grades – for in my day, as Wilby points out, ‘A’ levels were simply pass or fail, and fewer than half the candidates passed – is to determine who goes on to the next level. They perform a mainly filtering function. Often the filtering is applied without any other benefit being sought. A requirement was pursued by the Ministry in the ’60s that applicants for the then Diploma in Art and Design should have 5 GCEs. This was in spite of official evidence showing that having such qualifications made no difference to the performance of Art Students – was indeed irrelevant- again on actual evidence, they did very well in finding work related to their education on leaving college. (The research was commissioned, would you believe, by Margaret Thatcher, but as it proved the wrong things it was sneaked out in the summer recess, when no-one would notice). The only result of the 5 GCE requirement, (followed of course by two ‘A’s) was to keep out working-class students, though this was an incidental, and not, I think, an aim as such.
The concern was then, and apparently still is, to restrict the number of students in further & higher education. Universities have limited permission to recruit. Why? Even before students were made to pay for this level of education, we had apparently abandoned the ideal that all those who can benefit should receive it. But what is the case for restricting entry if the students now have to pay for the courses? Where is the free market? (And if the students are now the purchasers, not the beneficiaries, of education, aren’t they now in a position to demand to determine the nature of the goods purchased?)
The exam system is not a shiny seamless self-consistent gold standard. It is a ramshackle workaday structure, buttressed by ifs and buts: to use it as the sole determinant of progress in the learning system is immoral. Changing the rules in the middle of the game looks even more sleazy during the Olympics – where did all that British fairness go? But of course this kind of manoevre does show how dodgy the structure is. Again, at a time of high unemployment it is also economic madness to prevent students entering courses and apprenticeships unless you really want so many people hanging around the streets that riots are inevitable.
To the statement “Everybody can’t win” we should reply “Yes they can.”
When we objected to the rigidity of entrance qualifications for art diplomas in 1968, Sir John Summerson, representing the authorities, said that they had at least made a loop-hole, a doorway – “but you want to make it a Triumphal Arch!” Thank you Sir John. That’s what we wanted then, and what we still want now..