‘Evidence’ BMA style

Civitas, 6 August 2009

A few weeks ago now the BMA launched a campaign to end market-based reform in the NHS. Their vision: the NHS ‘restored as a public service working co-operatively for patients’, that is publicly funded through central taxes, publicly provided and publicly accountable.  Ok.  Very good.  It’s a nice idea, but we should also remember why the Thatcher/Major government and then the Blair government sort to introduce competition in the first place (and we should also remember that this is competition for provision… the service is still publicly funded).

So why did they?  Fundamentally because the NHS was widely perceived (in the words of the 2002 DH paper) as an unresponsive monolith, largely working to a rhythm all of its own.  Even after the money started pouring in, little seemed to change.  Services were (and many still are) designed around historic patterns and anomalies, often to the convenience of professionals rather than patients; hospitals with perverse incentives to treat fewer patients rather than more; providers working to the assumption of a ‘divine right’ to provide services, rather than having to prove their worth.

Now, don’t get me wrong, there are numerous problems (market failure etc.) associated with the idea of using markets in health care.  And yes, the evidence of their effectiveness is, as yet, equivocal.  For every study that finds competition driving efficiency and quality, there’s another which tends towards the reverse.  And there are also real questions as to whether markets can ever be truly effective in a system where funding still, fundamentally, comes from a big central budget: there is little direct ‘consumer’ pressure.

But I always thought that clinicians worked by high standards of proof.  Not so it this case apparently.  Yes, the government has gone for what the philosopher Karl Propper has called ‘utopian social engineering’ in plumping for a whole-sale introduction of market structures before testing it out reform – which makes it difficult to discern effects.  But the way the BMA presents the (cherry-picked) ‘evidence’ it has garnered is, quite frankly, pretty blatant propaganda.

Let’s focus on one in particular: the BMA’s supporting briefing on Independent Sector Treatment Centres and, even more specifically, the part on quality of care within this (I’ll leave the value for money part until next week).  The two points the BMA makes, dissected in turn, are:

i.    The Healthcare Commission reported its own concerns about ‘the lack of high quality, routinely-available, systematically collected data on individual patients that is essential for the assessment of the processes and outcomes of care’ available from ISTCs.

It did.  But a large part of this was due to a strange policy on behalf of the Department of Health that, as part of ISTCs’ contractual terms, required them to collect data on 24 Key Performance Indicators (KPIs) rather than (initially) reporting to the central Hospital Episode Statistics – making comparability extremely difficult.  ISTCs are now reporting to HES and the data is available.

What’s more the independent sector is taking the lead in wanting transparency and comparability of data, in commissioning the Hellenic Project.  This seeks to enable the benchmarking of independent sector providers against each other and the NHS on key indicators such as mortality rates, day case rates, MRSA and C.diff rates, and surgical site infections for knee and hip operations.  The Healthcare Commission itself also noted a ‘strong culture of service monitoring’ relating to outcomes and patient safety within ISTCs.

Also not mentioned by the BMA were the wide conclusions of the same Healthcare Commission report.  Specifically, the Healthcare Commission found length of stay and rates of readmission to hospital to be consistently and significantly lower in ISTCs than the national average – in fact standardised readmission ratios were four times higher in NHS acute hospitals than ISTCs for hernia operations and roughly twice as high for cataracts, primary knee replacements and arthroscopy.

Unpublished research by the DH Commercial Directorate (did the BMA receive this in the communiqués with the DH it mentions in the briefing and simply choose not to use it?) indicated that staff in ISTCs may be up to 60 per cent more productive than equivalent staff in NHS acute trusts and that theatre utilisation is 33 per cent higher.  (There are issues with case mix here, but addressed below.)

Onto the BMA’s second point:

ii.    More generally there has been concern that the quality of care in ISTCs might not be as high as patients should expect due to the use of doctors trained abroad and therefore unused to NHS processes and techniques, the lack of many ISTCs’ ability to manage complications due to staffing mix and the absence of necessary facilities. Some more specific examples highlight these concerns. For example, an audit of more than 200 patients who underwent knee surgery between 2004 and 2006 in an ISTC at Weston-super-Mare, Somerset, has revealed that one in five of the operations to replace knees were so bad that they had to be redone – a rate ten times the normal figure in NHS hospitals. The paper, published in the Journal of Bone and Joint Surgery, found that one out of three patients suffered a poor outcome following their knee operation.

Ok.  Let’s dissect this a bit.  The first thing to note is that the Weston-super-Mare treatment centre mentioned was not actually part of the centrally procured ISTC programme the BMA document is otherwise criticising.  It was locally procured and operated out of Weston Area Health Trust – an NHS provider.  But, yes, knee surgery was contracted out to a private company within the trust and the results were unacceptably bad; according to a rapid response from two employees to the paper due to the employment of short-term locums to perform the surgery.

But why then does the BMA choose not to mention the other side of the coin; i.e. the one clinical audit of the centrally procured ISTC programme the paper’s attention is otherwise aimed at?  This was carried out by the Royal College of Surgeons and the London School of Hygiene and Tropical Medicine.  It was based on a prospective cohort study of patient-reported (health) outcomes (PROMs) for 769 patients treated in six ISTCs and 1,895 treated in 20 NHS providers in England 2006-07 and adjusted for pre-operative characteristics and case-mix.  It found that ‘there is no widespread problem with poor quality care [in ISTCs]’.  In fact it found post-operative PROM scores in ISTC patients indicating significantly better outcomes than NHS providers for cataract surgery and hip replacements and approximately similar outcomes for the other procedures studied.   Fewer patients in ISTCs also reported a post-operative complication than those treated in the NHS.  The only blip in comparative performance was varicose vein surgery, where patients who underwent such a procedure in an ISTC were less likely to describe their operation as a success.

But the BMA prefers to refer to anecdote, that ‘there has been concern that the quality of care in ISTCs might not be as high as patients should expect due to the use of doctors trained abroad and therefore unused to NHS processes and techniques’.

And we could also look at patient satisfaction.  Here are the headline figures, complied from ISTC Patient Satisfaction survey data to February 2009 (published on NHS Choices March 2009) and from the Healthcare Commission 2008 NHS Inpatient Survey, published 13 May 2009.  The results for both NHS and ISTCs are age-sex standardised to the NHS 2007 electives weighting targets as agreed with Healthcare Commission:

Overall rating of care
96% of patients taking part in the ISTC surveys rate their care as “Excellent” or “Very good”, while the corresponding figure in NHS hospitals is 78%.

Respect and dignity
96% of ISTC patients say they are “always” treated with respect and dignity, compared with 79% of NHS patients.
79% of patients say they are “definitely” involved as much as they wanted to be in decisions about their care and treatments in ISTCs, and 52% for NHS patients.

Cleanliness
There is a significant difference between ISTCs and the NHS on cleanliness, with 92% of patients rating their room or ward in the ISTC “Very clean”, and only 60% of NHS patients giving the same rating.
Of the patients who felt confident to report on hand cleaning, in ISTCs, they thought that 92% of doctors and 91% of nurses “always” cleaned their hands between touching patients. In NHS hospitals, the figure is 74% for doctors and 70% nurses.

Nursing care
In ISTCs, 82% of patients said that nurses answered their call bells within two minutes, while the figure was 56% for NHS patients.  93% of ISTC patients “always” had confidence and trust in the nurses, the figure being 75% in the NHS.

Doctors
In ISTCs, 94% of patients “always” had confidence and trust in the doctors, while the figure was 81% in the NHS.  88% of ISTC patients said doctors “always” gave them understandable answers to their questions, while 68% of NHS patients felt the same.

Admissions
In ISTCs, 14% of patients report having their admission changed by the hospital, while 20% of NHS patients report changed admissions.

Now, the BMA’s wider point with regard to the overall value for money of the ISTC programme has a little more credence behind it (more on this next week).  But, come on.  I don’t particularly care who’s providing the care so long as it’s high quality. How about a bit of balance and use of reason; at least an impression that some kind of impartial analysis has been carried out?  Would such standards suffice in the day-to-day practice of medicine?  I hope not.