‘One problem with implementing evidence based medicine is, of course, that the evidence keeps changing. An important recent example is the mounting evidence that ever tighter glucose control in people with type 2 diabetes may actually be harmful. As Richard Lehman and Harlan Krumholz point out in their editorial (doi:10.1136/bmj.b800), the evidence that tighter control might not be better was emerging just as the targets for the 2009 quality and outcomes framework (QOF) were being re-negotiated. The framework sets evidence based clinical targets for British general practices to reach (and pays them for doing so), and it has helped improve the implementation of evidence based interventions. But the 2009 version now includes a tighter target for glycated haemoglobin—just as that looks to be the wrong thing to be doing.’

Oh dear.

The QOF, for all its failings, has been relatively successful in biomedical terms largely because it is seen to be ‘owned’ by the profession and clinically relevant.  NICE, by contrast, is a fairly distant body to those in general practice and, using QALYs to reach its decisions, is not that well-placed to deal with the complexities of general practice (with its important interpersonal elements and expertise in dealing with patients with complex co-morbidities).  Thankfully, however, the contract for researching the evidence-base for indicators has, at least, been won by the National Primary Care Research Centre (at the University of Manchester), staffed largely by current or past GPs committed to quality improvement, but well aware of general practice’s historic strengths.  NICE’s work in this area is also to be chaired by a former chair of the Scottish Royal College of GPs.

Of perhaps greater concern though is the preference of the DH – expressed in the same consultation document – to consider exception reporting (that is, to remove patients from counting toward overall QOF scores, and thereby income) ‘unacceptable’.  To be sure, exception reporting opens up the possibility of gaming.  Unscrupulous GPs could use exception reporting to cheat the system and strike awkward patients off the QOF register in order to boost scores, which leaves obvious question marks about the quality of care such patients might be receiving.

But, as we argued in a recent piece on the QOF, exception reporting also has a very good clinical rationale.   There are many things beyond the control of a GP where treating a patient along QOF guidelines would be inappropriate; such as age, a lack of responsiveness to treatment, an unwillingness of the patient to be treated, and contra-indication for therapy.  If such patients were included in the QOF, it could unfairly penalise practice income, produce perverse incentives for inappropriate treatment, or encourage practices to remove ‘unusual’ patients from their lists in order to maximise payment.

To put it in medical speak, you can never perform the gold standard of random control trials on every kind of patient GPs meet every day of the week, so their freedom to use their experience and apply an ever-expanding base of sound research findings to individual patient care, particularly those with complex co-morbidities, must remain.

Exception reporting may sound a bit trivial to the bystander, but the issues surrounding it cut to the heart of what the practice of medicine is about.  To be sure, there are now evidence-based principles and protocols that, in the general case, it would be churlish bad medicine not to follow.  However, as the Harvard Medical School professor of medicine Jerome Groopman has argued powerfully, follow evidence-based too rigidly and you can undermine what may be in an individual patient’s interests – based either on the biological particularities of their case, or their specific preferences.  Algorithms, he said, ‘quickly fall apart when a doctor needs to think outside their boxes, when symptoms are vague, or multiple and confusing, or when test results are inexact’.

In fact, they can be downright dangerous.  A randomised study published in the New England Journal of Medicine last month showed more patients dying in intensive care units when patients received insulin to tightly maintain their sugar in the normal range (a clinical guideline), compared with those on a more flexible protocol.  Similarly, two studies published in June 2008 cast serious doubts about maintaining normal blood sugar in ambulatory diabetics with vascular problems (another clinical guideline) has been a key quality metric in assessing physician performance.

Medicine is, as yet, an inexact science.  If the QOF (itself a serious of guidelines with financial incentives attached) is to remain, then so must the provision to exception report.

Looking purely at quality indicators included in the QOF, the strategy appears to have been vindicated. General practice returned an average of 91.3 per cent of the maximum possible score in the first year (2004/05), rising to 96.8 per cent in 2007/08 – significantly higher than was anticipated by the Department of Health. Concomitant with this, there have been step-changes in clinical quality for patients with certain chronic diseases, such as asthma and diabetes. Health inequalities on QOF quality indicators have also fallen, with faster improvement in practices in the most deprived quintile reducing the difference in performance with least deprived from 4.0 per cent to 0.88 per cent.
However, all is not so straightforward. Quality of care is hard to conceptualise and measure in ways which capture the full range of issues that matter to patients and can be applied day-to-day. The QOF’s focus is primarily on technical effectiveness – on health promotion and the evidence-based treatment of particular chronic conditions in the biomedical model – but, as the American statistician, W. Edwards Deming famously warned, in most cases 97 per cent of what is important either isn’t measured or isn’t measurable. The QOF neither includes all medical conditions nor captures the essence of a primary care consultation, the vital relationship between doctor and patient. The fear that the financial incentives in the framework would divert attention away from these areas to meeting QOF targets is in evidence.
Quality – and improvement in quality – can be substantially worse for those with conditions outside the framework, particularly when concerning the elderly. Across eighteen practices between 2003 and 2005, achievement against fifteen indicators concerning depression and osteoarthritis (not in the original QOF) increased by just one percentage point from 35 per cent to 36 per cent, compared with a 16 percentage point improvement in incentivised indicators relating to asthma and hypertension. Consultation rates for depression and anxiety also fell; and in face-to-face interviews in the English longitudinal study of ageing, 75 per cent reported receiving endorsed quality of care for conditions included in the QOF, compared with 58 per cent for those not. For certain non-incentivised conditions, such as falls management (41 per cent) and osteoarthritis (29 per cent), the picture was worse.
More widely, the QOF encourages GPs to focus on the treatment of professionally-defined diseases, not on the health problems as patients’ experience them. Medicine is an inexact science. Otherwise sound clinical guidelines may not be appropriate in treating patients with multiple health problems. The consideration of social context, the amalgamation of appropriate consulting skills and styles, the identification of patients’ priorities and concerns, and the involvement of patients in decision-making, as well as pure technical quality, are all associated with positive outcomes; yet the QOF draws attention elsewhere and tends to place a ‘second voice’ in the head of the GP. At the extreme, some practices have reportedly started to neglect cultural attitudes towards patients and ‘bish-bang-whallop through the scoring’; but more commonly, the QOF has caused the inadvertent diversion of attention or the odd bit of ‘gaming’ the system. In one survey, 75.9 per cent of nurses reported that they felt the framework was undermining the patient focus of the NHS.
There is a strong case that the QOF should be both downscaled and downsized to give greater priority to patient-need and the professional judgment of the doctor.
• The proportion of income it is possible to derive from the QOF could be reduced, to around the seven per cent suggested by the Health Foundation’s director Professor Martin Marshall.
• The number of indicators in the QOF could be cut and – while open to new evidence – confined to clinical indicators, such as ACE inhibitors for heart failure or influenza immunisations in over 65s, which have been rigorously proven to deliver significant, cost-effective, health gain to many.
The question can be posed as this: Do we really want GPs to be a set of what the cultural critic Raymond Tallis has termed ‘sessional functionaries robotically following guidelines’?
The full report, Checking-Up on Doctors: A Review of the Quality and Outcomes Framework for General Practitioners is available here.