How to recognise bad science

Pesticide residues in your 5 a day

If we consider all the reports we hear that wine is bad for us or alternatively that a glass of wine a day is beneficial; organic farming is environmentally sound, but, on the other hand, it means ploughing up more land to feed us all; that vaccinations have saved millions of lives, however MMR is dangerous for our children; five a day fruit and vegetables are what everyone should eat, but you must always watch out for those pesticide residues.

These examples go on and on, but is there a modicum of truth in any of the stories or are they simply poor science, poorly explained; poor experimental design, poorly interpreted; sensational stories designed to elicit improved funding; stories based on belief from single issue pressure groups who seem to work to the idea that if you say something enough times, everyone will believe it (because it has become common (if unverified) knowledge)?

There is sound science going on, but it is usual that the good scientists will publish in peer reviewed journals in a style that the general public will not understand or will very quickly tire of reading. In a world where it is acceptable (and often applauded) to wear ignorance of science as a badge of honour, while for an individual to plead no knowledge of fine art, classical music or Shakespeare’s plays is a death sentence at the dinner table. One of the problems with science and the scientist is that they spend a great deal of time talking to each other in a language unintelligible to the public rather than spreading the word. Science is exciting; it is universal; it is all embracing; it is essential for the survival of the planet, but begin a conversation on the implications of nanotechnology in formulating medicines in the pub and see how many people join in.

Obama's assassination chances

Recently on the UK's Today Programme we were told that Barack Obama is currently so unpopular that he is 400% more likely to be assassinated than was George W Bush. Shock horror!! But what does this mean? Absolutely nothing, but it is a good story. We hear of the risk of dying from a heart attack if you are a male over 50 with high cholesterol is increased by 24% if you also take ibuprofen. This is poor reporting and tardy interpretation. The figures from the trial showed that over four years there was one extra heart attack (on top of those that would occur anyway) in a population of 1,005 people and no more deaths. The truth is much less scary than the newspaper stories, but it is not news.

Clearly the world is confused by ‘bad science’ ranging from poorly undertaken experiments to inappropriate interpretation of data and ‘cherry picking’ to support a favoured viewpoint. Aside from notable exceptions, the reporting of science in the media is at best patchy and often alarmist and sensational. We are often served a recipe of opinion and belief rather than soundly determined experimental evidence. How can we tell the differences between the two and how can we recognise bad science?

The Bad Science team

Over 120 delegates were at Belgrave Square on Thursday 15 October 2009 for the first in a series of free public lectures to find out. Free public lectures were one of the ideas that came from the Members’ Forum and it represented a first for the Membership Advisory Committee.

The topic How do you recognise Bad Science? was based on the book Bad Science by Ben Goldacre, who is also a regular columnist in the Guardian. There were three speakers:

• Evan Harris MP has been the Liberal Democrat Member of Parliament for Oxford West and Abingdon since 1997. He is currently the Liberal Democrats' Science Spokesman and a longstanding member of the Science and Technology Select Committee. He has a particular interest in evidence-based policy.
• Ben Goldacre is a doctor, writer and broadcaster best known for the Bad Science column in the Guardian, and www.badscience.net, examining the claims of scaremongering journalists, quack health products, pseudoscientific cosmetics adverts, and evil multinational pharmaceutical corporations, as well as wider themes such as the medicalisation of everyday life and the psychology of irrational beliefs. He has a background in medicine and academia, trained in Oxford and London, and works full time for the NHS. In 2008 he wrote and presented The Rise Of The Lifestyle Nutritionists and The Power of Placebo on BBC Radio 4.
• Clive Cookson has worked in science journalism for the whole of his professional life. He graduated in chemistry from Oxford University in 1974. After journalism training on the Luton Evening Post, he became science correspondent of the Times Higher Education Supplement in London and then spent four years in Washington as American Editor of Times Higher Educational Supplement. He returned to London in 1981 as technology correspondent of the Times and moved to BBC Radio as science correspondent in 1983. He joined the Financial Times as technology editor in 1987 and has been Science Editor of the FT since 1991.

Spinach is good, but..

Ben Goldacre outlined some of his many problems with science and its reporting. How can a newspaper report the misleading statement that claims vitamin C is a better cure for HIV than AZT without examining the experiment that is used to support the data? Clearly, if you do the work on cells in a petri dish you will get a different result than doing it on sick people.

How can a nutritionist claim that eating spinach (pictured) is good for you because the chlorophyll that it contains ‘oxygenates the blood’ (which of course needs light, water and carbon dioxide). How much light is there in the intestines and how would this ‘released oxygen’ interact with the gas that occurs there naturally – methane?

The trouble is that the general public, with nothing else to go on, believes it because it would not be reported if it were not true and many of us do not know any better. And why are pharma companies not required to register all their clinical trials with their predicted outcomes before they begin so we can all see what trials were planned, the expected outcome of those trials and, at the end of the test period, we can see ALL the data from ALL the trials, not just those that the originating company uses to demonstrate improved efficacy?

The MMR hoax

And then of course there is the media’s MMR hoax (as it will become known) in which unpublished ‘results’ were published by the media as facts leading to a significant and dangerous fall in the numbers of young children being taken for vaccination and a subsequent surge in measles and mumps.

Clive Cookson argued that science coverage by the media was improving (although he did concede that some published pieces were not well researched and could be misleading to an uninformed public). Today all good science journalists check the validity of a news item and usually do not publish unless it has appeared in a peer reviewed journal or its authenticity has been confirmed with an independent expert in the field. To this end the Science Media Centre (SMC) has done a great deal to support good science writing and it continues to offer advice and direction to all science media correspondents.

Evan Harris spoke of the work of the Science and Technology Select Committee and how its deliberations have led to the introduction of legislation that is based on scientific fact rather than opinion or belief, but all politicians will always have one eye on their constituents and often new laws are passed that are designed to satisfy public (uninformed) opinion rather than being based on good science.

Science-educated MPs needed

This is not helped by the paucity of science-educated MPs and the decision to allow the NHS to pay for homeopathy, but to require NICE (National Institute for Clinical Excellence) to decide on the financial return associated with a new drug before it can be introduced.

As the first of a series of free science events the evening was a great success with the chairman having to stop the audience asking questions. The talks were followed by refreshments which allowed plenty of time for informal discussions and networking. Keep your eyes open for the next such event – if it is half as good it will be outstanding!

Leonard Copping and David Evans