When scientists from Bristol University questioned the accuracy of risk estimates based on the Framingham heart study, there was a collective intake of breath. As accusations go, it was up there with criticising David Attenborough or saying puppies aren't adorable. For Framingham was the most gilded of gold standard medical data sets. But it now looks as though it has been relegated to that league of greats that once ruled the world but which are now recognised as also being able to bamboozle, mislead and generally misinform.
Framingham is a middle-class, predominantly white town in Massachusetts, not far from Boston. It's where the Battle Hymn of the Republic was first sung. It's where Christa McAuliffe, who perished in the Challenger disaster, lived. It's Basingstoke, in America. Between 1968 and 1975, information relating to heart disease was collected from 5,573 citizens. The data so carefully collected have been used to frame risk scoring methods for coronary heart disease (CHD) for decades. It remains an outstanding piece of research, which has played a key role in quantifying risks such as smoking and high cholesterol.
But Dr Peter Brindle, a Wellcome Trust research fellow and GP with an inner city practice, began to question its accuracy as a relevant tool for his patients. It's the medical equivalent of someone in Leeds saying that God might not be a Yorkshireman after all.
So, he compared the Framingham data to that from its British equivalent, the British regional heart study. Using the Framingham data he over-estimated the risk of non-fatal coronary events by 57%. But, perhaps of more concern, he found that 84% of British heart deaths occurred in the 93% of men classified as low-risk by Framingham criteria.
Framingham is showing its age. "It hasn't kept up with changes in treatment - deaths from CHD have dropped by 50% since the 1970s," says Brindle. "It gives us an idea but we need to use our clinical judgment to take into account things that aren't in Framingham." Data that is past its sell-by date or even downright misleading is nothing new. The problems stem mainly from extrapolating data gathered from one population and applying it to another on the basis that one group of people is broadly equivalent to another. The result is confusion, and less than optimal treatment for some.
It's an acute concern with medicines. For example, a third of drugs in Britain are used by the over-65s, many of whom have several medical problems apiece. It's well known, too, that adverse drug effects increase with age. So which group of people have tested the drugs used most by senior citizens? Fit young men.
"Drugs need to be properly tested for those most likely to use them," says Lesley Doyal, professor of health and social care at Bristol University, who has studied the exclusion of minorities from drug trials. The situation has been recognised and should change but the continuing gulf between study samples and consumers is illustrated by the fact that only 10% of those in UK trials of additional treatments to help improve the chances of a cure (adjuvant therapy) for breast cancer were over 70, even though this age group make up about half of those with the disease.
There are similar concerns about the scarcity of children and ethnic minorities in trials, as well as that other minority, women, who have fallen particularly foul of the "one body is as good as another" approach.
Many labour ward drugs have never been formally tested on women, let alone pregnant ones. They are tested by men, which might be the subject of a few pub jokes were it not for the fact that men and women are now known to differ substantially in their reaction to pain relief. Men are more sensitive to morphine, and women are less likely to get relief from anti-inflammatories. One painkiller, used principally in dentistry, barely touches pain for men.
It isn't just drugs. Exercise tolerance tests are used to assess heart problems. For years, reference data were based on male values, leading to both under-diagnosis and misdiagnosis of heart disease in women. In occupational health, safe levels for a wide variety of workplace chemicals are based on exposure level thresholds derived from young men. Those whom such exposure is most likely to affect simply don't figure.
As we move into the 21st century, the one-size-fits-all approach is likely to give way to individualised treatments based on genetic differences. Meanwhile, Eric Brunner, reader in epidemiology at University College hospital, makes a plea for the baby not to be slung out with the bath water. "It's incorrect to say Framingham is no longer producing useful data because it is still a good way of assessing changes in risk factors. It's not invalid in the relative sense." He concedes its limitations, however, in terms of a doctor assessing a patient's chances of dying in the next five years. It's clear that data, particularly large studies, can be mesmerising. Doctors have to remember that it's an individual sitting in front of them.
It ain't necessarily so
Children
Up to 90% of medicines taken by children have been tested only on adults. Dosages are usually calculated by weight as if children were mini-adults. In fact, children's bodies use drugs very differently. Their liver and kidneys work at a different pace, depending on age, meaning that higher doses than expected might be required for older children while the reverse may be true in the very young. There are differences too in the way that their bodies absorb drugs.
Older people
Few trials include people over 70. Trials of drugs such as anti-inflammatories, used widely by older people in the treatment of arthritis, are skewed towards a younger age group. The most severe side effects (gastrointestinal bleeding) are seen predominantly in older people, the very people excluded from trials.
Ethnic minorities
Responses to drugs vary with racial origin. For instance, the incidence of "poor metabolisers" (people with low levels of liver enzymes involved in drug breakdown) varies from 4% in Caucasians to 25% in Asians. Afro-Caribbeans may get better control of blood pressure with one type of drug than another but despite these variations, ethnic minorities are rarely included in trials.
Women
The possibility of pregnancy and variations in the menstrual cycle historically meant that women of reproductive age were not included in trials though women differ fundamentally from men in their responses to many medicines.
Occupational health
Safe exposure levels for many industrial chemicals have been set using thresholds derived almost exclusively from studies of young white men.
Alcohol tolerance
For many years, calculating contrasting alcohol tolerances was thought to be a simple matter of weight correction. Today it is known that levels of an enzyme that metabolises alcohol (alcohol dehydrogenase) are key. Levels in women are about a fifth of those in men and there is also ethnic variation.
Exercise tolerance
Much data relating to treadmill tests was initially based on readings obtained only from men.