Who knows how many people have started guzzling fruit juice thanks to the blizzard of publicity surrounding an article in the American Journal of Medicine last week that suggested drinking the stuff might help fend off Alzheimer's disease. A survey of 1,836 Japanese Americans, the journal reported, found that those who drank fruit juice at least three times a week were not as likely as those who consumed it less than once a week to develop "probable Alzheimer's disease". It may be that fruit juice does protect against dementia, but my bet is that it doesn't. We've been here before.

A study like this has many potential scientific problems, but the biggest is that those who drink lots of fruit juice will be different in many ways from those who drink little. The people who drink more juice are likely to be better educated, richer, more concerned with their health, less likely to smoke, more likely to exercise, and generally healthier. It's not surprising that they are less likely to develop dementia. The authors "risk adjusted" the data, but there may be other factors that the scientists don't know about and cannot adjust for.

Another scientific and logical problem is that just because people who take a lot of x don't get disease y, it doesn't follow that giving people x will prevent y.

In the 80s, several studies showed that people who had high intakes of the antioxidant vitamin betacarotene from eating fruit and vegetables were less likely to develop cancer. The evidence was "convincing". Scientists then conducted a randomised controlled trial in which some people were given betacarotene and their chance of developing cancer was compared with those who weren't given the treatment. Such trials are the best scientific way of testing whether a treatment works. Sadly, those given betacarotene proved more likely to develop cancer.

A similar thing happened with the suggestion that people who had high intakes of vitamin E were less likely to develop heart disease. In a randomised trial, it was found that those given vitamin E were actually more likely to do so.

We've experienced the same scientific problem with hormone replacement therapy for menopausal women - only this time the idea that the therapy would reduce heart disease was pushed very strongly by the drug industry. In the 80s and 90s thousands of women were reported to be taking hormone replacement therapy. What happened to them was compared with what happened to women who didn't take it. Those taking the therapy were found to have fewer deaths from heart disease, and women were strongly advised to take it. But when the first results were announced at the end of 2002 from a large randomised trial, it emerged that hormone replacement therapy made women more likely to suffer heart disease.

It was also claimed from non-randomised comparisons that taking hormone replacement therapy decreased a woman's chance of developing dementia. But yet again a randomised study (when there is no difference among the women except that some take the treatment and some don't) showed that the treatment actually made dementia more likely.

Science is cruel and hard. Many things that seem bound to do you good when properly tested prove to be harmful. Most Guardian readers probably drink three glasses of fruit juice a week anyway, and I wouldn't stop. I drink as much myself. But nor should you bet your brain that it will prevent dementia. It may even make it more likely.

· Richard Smith is the chief executive of United Health Europe, and a former editor of the BMJ commentisfree/richard_smith.