Medical science and public policy: handling uncertainty, managing transparency

Eileen Rubery | 07 May 2003

• Tweet
• 

Executive Summary

• The different perspectives of the scientist and the public can be summarised by saying that scientific experts seek to reduce uncertainty while stakeholders and the public aspire to learn the truth. The scientific method is rarely suitable to be applied by itself when taking decisions on public policy. This paper draws on the author's knowledge of how some real policy decisions were made and seeks to draw general conclusions on the best way to make judgements in these circumstances.
• The first example considered is the excess cases of childhood leukaemia near Sellafield. A scientific explanation for this excess continues to evade researchers in spite of over 20 years of investigation. But this continuing uncertainty has not prevented major improvements in the management of the radioactive discharges from the Sellafield site.
• The second example is measles: a serious and highly infectious disease, which can be prevented by a vaccine that most medical studies have found to be safe. Concerns about the safety of the triple vaccine (MMR) have little basis in scientific evidence, but are so great that uptake has fallen sufficiently to permit epidemics of measles again. The alleged links with autism and Crohn's disease are not easily resolved by further study, so parents have to decide what to do for their children in the face of this uncertainty.
• Polarisation of views often occurs because the public-health expert is thinking in terms of population benefit, while parents are more concerned about the safety of the individual child. In such situations it is easy for feelings on the two sides to start to polarise and for them to demonise each other.
• Our present desire for absolute safety can lead to an over-careful society and reduce the value placed on individual training and skills so that managers concentrate on the process instead of the people.
• One solution frequently suggested is to increase transparency, but this alone may not help. It also needs to be used intelligently.
• This brings us to the third example, the Food Standards Agency's review of BSE. All consideration of the issues was in the public domain, but this led to superficial discussion, boredom by the press and lack of interest in the final conclusions of the Review team. In a second review a combination of in camera and open meetings produced a much more rigorous assessment of the issues and there was also evidence that participants gained greater understanding of the perspectives and values of the rest of the meeting, so there was an increase in mutual understanding as well.
• Instead of an approach which tries to solve these difficult problems by more experiments and more 'scientific facts', we need to recognise that they are not resolvable by science alone but instead that the uncertainty needs managing. The term 'Wicked Problem' is proposed to describe such issues. It is suggested that they can be managed by increasing mutual understanding of the different values underlying the different conclusions of those considering the data. At the same time it has to be recognised that such problems will need to be revisited as perspectives change. This process can be called 'Pragmatic Satisficing'.

Introduction

Rather crudely one might say that scientific experts seek to reduce uncertainty, while stakeholders and the public aspire to learn the 'truth'. The problem with scientific endeavour is that its method, while powerful, requires very careful control of variable parameters during the investigation of the problem, while life has an altogether more messy feel about it and is rarely prepared to submit to such constraints for very long, if at all.

This means that many of the most interesting questions are not really amenable to scientific investigation and poses the really interesting question of what to do then? We tend to spend relatively little time thinking about how to handle these issues in the abstract, and a lot of time wrestling with them individually, at present. My contention would be that a little more time teasing out the general principles we might use to address them would help us to progress them more expeditiously and less contentiously.

Leukaemia cases at Sellafield

Let us start with the first such question that I had to deal with when I joined the Civil Service as a Senior Medical Officer in 1983. A reporter, James Cutler, had been investigating worker safety at Sellafield Nuclear Reprocessing Plant in Cumbria.

Sellafield is at the centre of the UK Nuclear Industry, producing fuel rods for our nuclear power stations, and then re-cycling the 'spent' fuel rods to obtain weapons-grade material and, amongst other things, isotopes for medical and industrial uses. Sellafield had also been the site of the Windscale Fire in the 1950s, when the nuclear reactor in the Windscale Piles caught fire and contaminated the environment to such an extent that locally-produced milk had to be thrown away.

When James Cutler talked to the locals in the pub they said that their health was OK, but there did seem to be a lot of cases of leukaemia occurring in the children living in the nearest village, Seascale. Seascale is just over 2 miles from the Sellafield Site and many of the workers at the Site lived there with their families.

James Cutler decided to change the focus of his investigation to the incidence of leukaemias in the children of the workers and did indeed find more cases than was expected due to chance. He made a TV programme 'Windscale - the Nuclear Laundry' about it. The Secretary of State for Health asked Sir Douglas Black, an eminent physician, to head a small team of six experts to advise him. I had been a cancer specialist treating people with radiotherapy before I joined the Civil Service, and so was asked to act as secretary to the Group. So started 18 years of grappling with the interface between science and policy making.

The Expert Group is a standard way for Ministers to try to get balanced advice on difficult scientific and technical issues, and over the years I spent as a Civil Servant I got involved in many such groups. Essentially such groups critically assess the information available concerning an issue; and try to say whether what is claimed is based on sound evidence or not. They are also often asked to consider what can be done. Depending on their composition they may be asked to confine themselves to scientific and practical issues, or to advise in wider contexts, taking account of public and political perceptions. Either way they should only advise, it is the Ministers' responsibility to make the final decision as our democratically elected representatives. But having said that, the fact is that Ministers are often reluctant to appear to disagree with or ignore their recommendations, so they can be pretty powerful forces for action.

In this case they confirmed the excess of leukaemia cases at Seascale- there were 6 cases over the previous 20 years where 0.6 would have been predicted: that is 10 times as many as would be have been expected by chance. They also found the nuclear discharges from the site were (absolutely legally) considerably greater than from any other nuclear site in the UK.

But correlation does not equal causation. The key question was whether the leukaemias were causally related to the radioactive discharges, and that was much more difficult to determine. There were a number of other possible hazards in the area that had been linked with causing cancer and leukaemia. For example, Sellafield is also a large industrial site and so uses large quantities of organic chemicals. Then there was also something called 'bracken fern carcinogen' that caused illness in the local sheep, and whose effect on humans was largely unknown, and there was a local industry that used depleted uranium.

The evidence for and against all of these possibilities was equally inconclusive, and the Black Advisory Group ended up saying 'it didn't know' what caused the leukaemias. In 1984 that was seen as rather daring, and one of my enduring memories is of listening to Kenneth Clarke, who was the Minister for Health in the Department at the time, defending our report to the Minister in the Department of Energy, who was convinced that once published there would be irretrievable damage to our nuclear industry because we had not explicitly and completely excluded radioactive discharges as a possible cause of the leukaemias.

I learned a number of things about handling these controversial issues from that first experience:

• saying you do not know is politically sustainable. It can even be quite a strong position: it is in fact quite difficult for the press to criticise such a position effectively;
• when producing such a report, if all the stakeholders think you have got it wrong (but in different ways), then you have probably got the balance about right;
• there are real limitations to what science can prove, especially when you are considering low-level exposures to chemicals over long periods. Even now, 20 years later, we still do not know what caused that excess of leukaemias; and perhaps most importantly
• failing to prove a causative link does not by itself justify inaction, neither does it mean no action will be taken. The Black Report was followed by prolonged action to reduce discharges, because it was perceived that, whether they caused the leukaemias or not, it was not desirable to have the environment polluted in this way in future.

Protection against measles

Let us turn now to the case of measles: quite a nasty illness, for in 1 in 3 cases side effects such as ear infection, that can lead to deafness, chest infections, fits or meningitis, occur. 1 in 1000 cases get encephalitis and about the same number have fits. Death occurs in around 1 in 1000 cases, and about half of these deaths are in previously healthy children. It is a disease best avoided if possible.

In the 30 years between 1940 and 1970 on average around 400 000, almost half a million, children used to get measles each year in epidemics occurring every two to three years. However, vaccination had strikingly beneficial effects, initially with the single vaccine and more recently with the measles, mumps and rubella (MMR) 'triple' vaccine.

In 1993, I was responsible for vaccine policy in the Department of Health. Because there had been relatively low uptake in the 1980s, our experts had warned us that an epidemic of measles in 5-15 year olds was likely within the next few years, which would probably result in between 150 000 and 200 000 cases and around 50 deaths, mostly in teenagers. They advised us to mount a campaign to increase immunisation levels in this age group.

By an enormous effort we were able to set in place a campaign to immunise 11 million children via the school health services over a six-week period and stop the epidemic in its tracks by attaining uptakes of around 95%. This sort of level of uptake is necessary with measles because it is very infectious, and so spreads rapidly in the community.

Of course within any six-week period some children will fall ill from chest infections, fits, meningitis and encephalitis and other illnesses whether they are immunised or not. We therefore kept careful statistics on the deaths and illness in the children immunised and there was no evidence of any additional serious illness or increase in death rates that appeared likely to be linked to the Campaign. However, if you were a parent and your child was immunised and then became ill in the following few weeks, no doubt you would take a lot of persuading that the immunisation was not causally linked to your child's subsequent illness. Furthermore, it is usually impossible for anyone to state categorically that there is not a link in any individual case. This is one of the reasons why worries about the safety of immunisation arise with any vaccine. But overall that campaign was very successful and confidence in the vaccine continued to be high.

Let us now turn to the present concerns about autism, Crohn's disease and the relative merits of triple or single vaccine immunisation. There are two questions, as I understand it, that parents are asking about this: are these two diseases more common in those who have received MMR vaccination? and does the risk of my child contracting either of these two diseases reduce if the three components of the vaccine are given sequentially rather than at the same time?

In answer to the first question, the general statistics collected do not suggest an increase in these diseases in those who have received MMR vaccine, but there are difficulties with interpreting information when nearly everybody has been immunised for many years, because the control group of unimmunised children becomes rather small and may be unrepresentative of the general population. We begin, once again, to come up against the limitations of epidemiological methodology here.

Because of concerns raised by some clinicians working with children with Crohn's disease and autism about a possible link between vaccination with MMR and the development of these diseases, it has been suggested that giving several vaccines at the same time is more likely to produce adverse effects than giving them one at a time. The theory underlying the hypothesis, as far as I understand it, is that if given sequentially, the immune system can cope better with making antibodies and so is less likely to develop Crohn's disease or autism. The logic of this is not entirely clear to me, but that does not of course invalidate the hypothesis or mean that a link does not exist.

But the practical issue here is that to establish whether this is the case or not requires very large numbers of children in a carefully controlled study over several years, with quite a high likelihood that the results would not be clear either way. At present the data just is not there to prove anything one way or the other. This means that the answer to the second question is that we do not know, so parents today have to decide what they are going to do for their children without this information.

There is also a longer-term issue here for the Department of Health. Public concern about possible additional risks from multiple immunisations is clearly not going to go away, and since this is likely to be the way that immunisations are offered in the future when using the next generation of vaccines, including those against meningitis and other fatal illnesses, the Government needs to take this concern seriously. Let us hope they will think more carefully about how to handle this issue in future, and not go in for the rather coercive approach they have taken recently on MMR and single vaccines, which has not been helpful to parents, children or the vaccination campaign, in my opinion.

Handling uncertainty

There are a number of reasons for the tension and concern these issues generate. One important aspect relates to the desire for an absolute assurance of safety. There is something of a refusal to recognise the fact that we are human and therefore mistakes and accidents will occur from time to time however careful we are. This may be exacerbated by the ease with which we are now able to appreciate the suffering of others when mistakes do occur, because they appear on our television. At a rational level most of us know this degree of elimination of risk is not possible. But almost helplessly we support setting up another register, or another enforcement agency, without considering whether this is really likely to achieve the outcomes we want. There is a saying in Whitehall about Ministers:

'We must do something;
This is something;
Let's do this'

which could equally be applied to the public and the media in this context. But although doing something does make us feel better, we should not confuse feeling better with having actually improved anything. And sometimes of course action makes things worse.

For in fact there is a lot of evidence that suggests that over-tight controls on professionals do not improve enforcement or reduce 'disasters'. Rather they produce a culture of carefulness as opposed to one of caring. Over-regulation or over-supervision disempowers trained and experienced staff, whose skills, usually gained at the expense of the taxpayer, are then wasted. They cease to use their training and the experience that they obtained precisely in order to permit them to make decisions based on individual circumstances. Instead they revert to blindly implementing guidelines in circumstances where they may not be appropriate. We really do not want a careful society, concentrating on the process, when a caring one, concentrating on the people, is possible.

Closely linked to this is the perspective from which different groups of people view the information available. Parents are naturally concerned for the safety of their child, that is their responsibility. The Government and public health doctors look at the issue from a population point of view, because that is what they are trained to do. Neither perspective is 'wrong', but tensions can arise if there is not some recognition of the fact that these different perspectives mean that the same data is viewed and valued differently by the two groups when making a decision. If the two sides start to lose sympathy with each other and demonise the 'other side', then rigidity sets in, people start to call for the 'truth' about the issue (always a bad sign in my experience), and progress towards an agreed way forward becomes increasingly problematic.

To help us face up to the inevitable uncertainties of life, it might help to re-define the nature of these intransigent issues. The term 'Wicked Problem' might be used for issues that:

• clearly do not have a right or wrong answer;
• have adverse effects for some groups or categories and benefits for others;
• need to be managed by discussion and accommodation of differing perspectives rather than by a flight into 'evidence-based' wrangling;
• need to be kept constantly under review so that decisions can be modified as the situation changes.

I would also suggest that the mechanism for handling such problems might usefully be called 'Pragmatic Satisficing', defined as accepting a 'good enough' way forward at any particular time, either because there is no choice, or because to obtain more information would involve an unreasonable expenditure of resources.

Defining a problem as 'wicked' and therefore not soluble might shift the focus from constantly seeking to obtain more 'evidence' when what is often needed is greater efforts to understand the key parameters surrounding the issue and so to seek an accommodation between differing values and views. Satisficing a wicked problem would aim to progress it rather than solve it. To be an acceptable way forward this process would need to concern itself with how to handle transparency, generate trust and encourage a belief that we are approaching nearer to the 'truth' while recognising the unattainability of 'absolute truth'.

Managing transparency

Transparency has been much in vogue with practitioners recently, especially with New Labour, and yet has received curiously less consideration in theoretical terms. So I want to consider transparency here in rather more detail. Transparency seems so easy. You just publish everything, invite everyone in to see what you are doing and there you are! Indeed, it sometimes seems as if everything is now being published, and everyone seems to be being consulted openly. But I am not at all sure it is as easy as this in reality.

Recently I have been working with the Food Standards Agency, who were set up in April 2000 with a specific remit to involve the consumer more in food safety issues. They have conducted two policy reviews with different approaches to transparency or openness over the last two years. They are both concerned with the much vexed question ('wicked problem') of BSE and the safety of food.

The first event was a review of the BSE Controls for the Prime Minister. It was their first major piece of work after the FSA was established, and they were naturally anxious to establish from the start a reputation for openness. They therefore set up a large group of 'Stakeholders' to conduct the review and decided that the entire review would be carried out in public.

All papers for the Stakeholder Group meetings were placed on the web before each meeting. All meetings were held in the presence of the press and the public, and the public were invited to ask questions during the proceedings. There was a web-site for comments and an open meeting for the public to make their own points relevant to their own agenda. As each draft of the report was produced it was placed on the web-site and discussed at the next meeting. Since the thorny issue of BSE and sheep was central to part of the Report this meant that press interest was initially high, and I got quite used to seeing BSE and sheep in the headlines the day of a meeting of the BSE Controls Review Stakeholder Group.

This all sounds very good, but there were disadvantages with this procedure. For one thing it was not very rigorous. With the threat of immediate transmission on the 10-o clock news of anything said at the meetings, many of the experts were rather quiet. Furthermore, holding a meeting with 30 plus people and a further 50 plus watching is not easy, and there was little meaningful discussion and certainly no in depth discussion of the complex issues under consideration. Then because the various drafts of the report were already in the public domain the final report was ignored by the press, who were clearly bored with the whole exercise by then. This evidence that interest in BSE and sheep is not inexhaustible for the press may be reassuring on one level, but it did mean that the impact of the final recommendations of the group were considerably reduced: the public did not get to hear of them and so could not exert pressure on Ministers to implement them if it thought they were important enough.

About a year later the FSA carried out a further review, this time as part of their work to decide what action they should take specifically on the possibility that BSE is in sheep. It is accepted that there has been the opportunity for sheep to be exposed to the BSE agent, but we do not have a diagnostic test to tell us whether any sheep are actually infected.

Possibly because the Agency also felt that their first essay into openness could be improved upon, this time they started by holding an open meeting at which the key experts advising the Agency presented what was known about the problem and answered questions from the floor.

The Agency then set up a 'Stakeholder Group' to produce a Consultative Report 'in camera' on the issue. I attended one of these meetings and was most impressed with the way the Chairman of the FSA, Sir John Krebs encouraged a full and open discussion of the issues - some of which were very contentious and emotionally loaded for certain of those present. (The group included representatives of the relatives of those who had died of vCJD, for example). The absence of the press and the fact that the group had agreed to keep their deliberations and discussions confidential at this stage enabled the group to have very frank and meaningful exchanges of views and resulted in real changes in opinions in many of the group during the production of the report.

The result of these meetings was a carefully considered and carefully constructed consultation document, which was then widely distributed for comment. At the end of the consultation the Report and the results of the consultation went to the FSA Board with proposals from the Executive of the FSA. The Board then made its final decision.

What impressed me about this process was that it clearly did increase mutual understanding between the farmers, the agriculture industry, the relatives of the patients and the consumer groups. It convinced me that dialogues of this nature are an important part of the way of progressing contentious issues. However, it was also clear to me that the success of the exercise depended very heavily on the skills of the Chairman of the Stakeholder Group, and on the stakeholder group being held in private; that such exercises need carefully handling if they are to have a positive outcome.

Now while my perception was that the second process was much more successful than the first one, I am not saying that open meetings do not work, nor that Stakeholder Groups need to always meet in private. The point I am making here is that transparency has to be handled intelligently just like any other process when you are dealing with a difficult or 'wicked' problem. I also suggest that the aim has to be to place stakeholders in situations where they can engage with each other and develop mutual understanding of the key issues, so that there can be movement towards a common understanding and common perceptions of what should be done. Essentially there are no 'quick fixes' or sure-fire solutions!

So in conclusion, my experience of these two activities suggests that a combination of open and closed meetings produces a more rigorous and informed result than openness all the time. But my proposal would be that 'intelligent management' of each issue is what is needed rather than adherence to rigid rules applied in all circumstances.

Conclusion

Since science cannot provide clear answers to our 'wicked problems', we need to recognise the incomplete information we do have will quite legitimately be valued differently by different sectors of society. We need to be more open about the limited nature of the 'evidence' upon which we are likely to be able to base our interim judgements and any actions. We need to learn how to take account of different perspectives and values and cultures. Perhaps out of this exercise we will indeed reduce uncertainty a little and approach slightly nearer to what the public perceives as the truth.

• Tags:
• Rubery, Eileen
• Medicine and public health

About the author

---

Eileen Rubery is a Lecturer in Public Policy at the Judge Institute of Management, Cambridge University's Business School. After completing a medical degree at Sheffield University, she trained as a cancer specialist before joining the Civil Service in 1983. Until she left in 1997, she led the Health Protection Group advising Ministers on the management of most of the major Public Health crises, including the Salmonella and Eggs crisis, Chernobyl, HIV/AIDS, vaccine policy including MMR and BSE/CJD. This article is based on the Richmond Lecture given at Downing College, Cambridge on 20 November 2002. A full transcript of the talk is available from e.rubery@ice.cam.ac.uk.

Search