A randomised control trial for synthetic phonics?

Civitas, 10 August 2010

Boris Johnson recently suggested a ‘competition’ between schools to decide which method of teaching children to read is the most effective, synthetic phonics or the medley of techniques often used in schools. Ben Goldacre of Bad Science fame was quick to castigate the mayor. Such a competition would prove nothing, only a randomised control trial could show how effective a teaching method truly is. But there is an alternative approach to gaining valid practical knowledge that Ben seems to overlook.

What are randomised control trials? They are essentially the gold standard in scientific research (especially in medicine), whereby a new technique or treatment is tested on one group, while another control group is observed without being given the treatment. The members of each group are selected randomly from the same pool to avoid biasing the treatment group with individuals with different characteristics (e.g. a different lifestyle or higher education level). Ideally, the subjects of the experiment should not even be aware in which group they are – control groups in trials for new medicines might be given sugar pills as a placebo. This controls for the often substantial psychological effects on individuals who feel better or are somehow encouraged to do well because they are being treated. The purpose of all this is to isolate the effect of the particular treatment rather than any characteristics of the subjects or other aspects of the situation they are in.

What is synthetic phonics? It is a technique of teaching literacy through learning individual letters and their associated sounds. Essentially, one learns how to read and write the word ‘cat’ through learning what the letters ‘c-a-t’ correspond to. This is sometimes opposed to other techniques (such as ‘whole language’ learning) where young readers are invited to infer what ‘cat’ on the page means, possibly from other clues in the text or accompanying pictures. In this controversy, we are not bystanders. We publish a textbook that uses synthetic phonics and make substantial use of the technique in our supplementary schools with (what we think is) tremendous success. Are we entitled to claim this without a randomised controlled trial to back it up?

In principle, an RCT should be possible. You would need to take a large pool of pupils (of five year olds if we are testing the ‘first and fast’ approach to phonics) and divide them into a treatment and control group, with the control group receiving a mix of different techniques. The treatment group would do a more concentrated and systematic synthetic phonics programme. However, it would be expensive to conduct, especially to do it with large enough numbers to demonstrate statistical significance. It would be hard to exclude several other characteristics (teacher quality and enthusiasm) and aspects of the treatment. For example, if parents were made aware of the experiment (probably a requirement on ethical grounds) they might be enthusiastic about a new technique and pass that on to their children. But it would still be a very useful exercise (perhaps the mayor should see if a rigorous trial in London would be possible).

More generally, there would be questions of how widely these results could be applied. Are their unobserved cultural or social factors that make a particular treatment work in one context, but not in another?  These problems, and more, with respect to social science were noted in a recent City Journal article by Jim Manzi: ‘The most fundamental lesson that emerges from such experimentation to date is that our scientific ignorance of the human condition remains profound. Despite confidently asserted empirical analysis, persuasive rhetoric, and claims to expertise, very few social-program interventions can be shown in controlled experiments to create real improvement in outcomes of interest.’

Is an RCT, and empirical social science more generally, the only way to judge teaching techniques? Are we lost without them? Perhaps not. Ben Goldacre is a crusader for evidence-based medicine. This usually puts him on the side of the angels, calling out quacks and snake-oil salesman who proclaim scientific evidence when it is usually nothing of the sort. But his approach sometimes seems to imply that the only valid knowledge is scientific knowledge. When it comes to our knowledge of the social, however, the limits of scientific techniques stand out. There are simply too many variables to observe or even acknowledge. All the conditions and factors effecting humans cannot be controlled or held constant. Our understanding of human life has an unavoidable element of subjectivism. How can we gain knowledge in this sort of situation?

F. A. Hayek had an answer. Using the notion of ‘tacit knowledge’ developed by the anthropologist Michael Polanyi, he argued that there is a vast array of information dispersed across society, with every individual having little pieces of practical knowledge. That is, knowledge that cannot necessarily be written down or explained in language. For example, most people know how to tie a shoelace. But it is much more difficult to write down a conscious procedure that explains exactly how to tie a shoelace. You just have to ‘do it’, while children learn through imitation and trial and error. This knowledge of implicit techniques and procedures, rarely discussed or codified, is what makes up the bulk of our social world.

Apply this to the case of reading. Synthetic phonics is, in essence, a clunky contemporary label for a technique of learning to read that has been in practice almost since writing emerged as a human practice (sounding out letters with their corresponding sounds). It was not invented, but emerged over generations of teaching and learning. It was not rolled out by a Government agency, but evolved through practice. By contrast, techniques such as whole language learning emerged as part of various ‘scientific’ theories developed by various linguists and social psychologists. There were various ideologies (of the left especially) associated with these new techniques, and they provided much of the impetus for introducing them to schools. As a consequence, support and opposition to these new techniques tended to emerge along ideological lines (we have a full discussion here).

Unfortunately, learning to read is one thing least suited to ideological analysis. Hayek understood an ideology to be a kind of overarching system of abstract thought. People tend to use an ideology to fill in the gaps left by their practical knowledge – a rough and ready way of viewing aspects of the world in which we are not experienced. The result of an ideological approach to reading was that those who had remarkably little experience of teaching tended, if they accepted the overarching ideology, to accept the validity of these new theories of education. In practice, these techniques were not very useful.

Does this mean that now is the time to purge the education system of all techniques apart from synthetic phonics? Far from it. The nature of practical knowledge means that it can be all too easily disrupted or destroyed through top-down imposition. Instead, we need to generate a system that allows for the practical knowledge of teachers and other educators to be put to best use – a decentralised system. Such an approach is part of what Will Wilkinson calls a ‘liberalism of discovery’, a set of policies that acknowledges the inherent ignorance of central government and encourages the greater use of knowledge dispersed throughout society.