The new Educational Policy (copies of which are increasingly hard to get – almost like education in Pakistan) has proposed a National Testing Service along the lines of the Educational Testing Service of USA, the company which conducts the TOEFL and other exams worldwide. The idea is to prepare a centralized testing system patterned along the lines SAT/GMAT are conducted. While the intention may be honourable I feel there is much to be said against a system that is already receiving criticism in its own country of origin.
Let us be clear: the examinations in this country are a farce. There is an urgent need for reform in this, as in all other sectors of the education process in this country. There is immense corruption at all levels – even the professional exams are not spared. In an effort to arrest the decline in standards and to bring some sanity at the national level, it is now being thought that only an internationally valid examination will do the trick.
It is my considered opinion that this is a misjudgment. Not that the SAT/GMAT/TOEFL are not valid or reliable for the purposes for which they are created. Only that they are PLACEMENT TESTS NOT ACHIEVEMENT TESTS. As all who know the jargon of testing, these are not the same things.
Let me elaborate. An achievement test is given at the end of a course to find how much is learnt. Of course the marks may be interpreted for any number of other variables … such as say the quality of the teaching. But the central characteristic remains … it is conducted at the end of a teaching cycle to gauge learning.
A placement test on the other hand is concerned with suitability. It asks the question: is this candidate suitable for an avowed purpose such as following a course of post-graduate studies in an American University? It is obvious that those who administer this test do not have any business teaching the candidates anything … that is, they are free to set their questions from a vast array of material which might be called “knowledge of the world”. They are certainly not tied to any fixed syllabus, nor to any time frame … a person may appear in the tests as many times as needed .. in fact more is good for business.
Even if we grant that these obvious facts will be taken into consideration and that the Pakistan tests will be suitably modified with expert help flown in for the purpose, there are other serious flaws that will hamper the streamlining of exams in this country. I ask six questions of the policy-makers and for the sake of convenience limit them to only the TOEFL.
ONE: The TOEFL is made from an item-pool of over three million items, cross-checked by computer for a variety of factors over a number of years to create a balanced test that if fair to all ethnic/cultural backgrounds. Where is the test-item pool for Pakistan going to come from?
TWO: All TOEFL materials are secret: no one can take the test papers out of a centre. The only available materials are those released by the ETS or commercially created “sample tests” to give hands on experience to those who wish to prepare for them. Can anyone guarantee the levels of secrecy that are necessary for such an enterprise in Pakistan?
THREE: The TOEFL has a multiple choice format marked by a B pencil on dotted sheets and scored by computers. All this involves expensive and reliable technology. No bad thing but, seriously, our faith in computers is rather touching. After years, indeed decades of highly subjective marking of essays we seem to intuitively realize that essay writing is one of the least reliable ways of checking learning. As one official of the Karachi Board put it: “It is only now that we have the technology to make the sequence of multiple choice questions different for a variety of learners thus making a dent in the curse of cheating.” While this faith in MCQs rings a chord among many, my concern at the moment is a deeper all-inclusive reason: Do we think that we can afford to create and maintain such technology without the usual ineptness thwarting the whole procedure in a couple of years?
FOUR: Given the present cheating syndrome that our students are so used to, how does anyone propose to ensure the privacy that is necessary for such an exam?
FIVE: There is an immense rural/urban gap in our delivery of education. Can the sophisticated ways of administering an exam like TOEFL be expected of all our students? One would think that the urban students stand at an advantage.
SIX: And lastly, do we think our teachers can manage?
I must appear to be an incredibly pessimistic person from all the above but naiveté is a greater sin.
The reform of our examination system is an urgent task, requiring the best thinking of the best minds of the nation. An alternative to the present farcical system must be sought. An alternative without the drawbacks listed above must exist. By all signs, however, the TOEFL (or SAT or GMAT) model is not the solution we seek.