In March 1961 – 50 years ago next month – the countries of the world, joined in their determination to stamp out drug abuse, came together to sign the United Nations Single Convention on Narcotic Drugs, which committed all of them to an outright ban on the production and supply of cocaine, cannabis, opiates and other comparable substances. Ever since, the trend in this country has been strong and unchanging: more people have been taking more drugs more frequently.

It is estimated that the number of young adults in Britain who had tried an illegal drug in the 1960s was fewer than 5%. This reached roughly 10% in the 1970s, and 15-20% in the 1980s. By 1995, nearly half of all young people said they had taken drugs. If you ever wondered how successful the Grange Hill kids' Just Say No campaign of 1986 turned out to be, then you have just read the answer.

Which is why the latest news is so surprising. According to figures released by the NHS in January, based on data from the British Crime Survey, the number of adults in England and Wales who used illicit substances in 2009-10 – 8.6% – was the lowest recorded since the study began in 1996. Among 16-24-year-olds, the picture was the same, with just 20% saying they had taken drugs in the previous year – another record low, and a third lower than the proportion 15 years ago.

Cocaine use is down, speed is down, cannabis is way down (yet again). LSD use is flat, but just one fifth of what it was in 1996. Though heroin use is also stable, fewer young people are currently requesting treatment for addiction to it. Meanwhile, in the largest ever survey of drug use among British clubbers, published in this month's edition of Mixmag, there were found to be large year-on-year falls in the number of people taking cannabis (by five percentage points), ketamine (10), ecstasy (five) and cocaine (20). The British Crime Survey tends to underestimate drug use (because it does not include people who are homeless, in prison, or living in student accommodation), and these falls are not the first, but they do cement a trend that is now too solid to ignore. In this country at least, for reasons that remain mysterious, drugs seem to be going out of fashion.

This is no small matter – the UN currently values the global drug trade at £198bn, making it the third largest industry on the planet, after oil and arms. Although, as these things do, it has crept up on us. "Something interesting is going on at the moment," says Martin Barnes, chief executive of the charity DrugScope. "We tend to get a lot of reports around drugs focusing on the bad news, and we certainly shouldn't be in the least bit complacent about the evidence of a decline, but for some people it is a surprising trend."

Confession time: I am one of those people. In 2004, I wrote an article in this paper trying to explain a recorded fall in the use of ecstasy pills among the young. The drug was not dying, I concluded, but clubbers might be turning to cocaine instead. Today, true ecstasy use is close to non-existent, yet cocaine use, as we have seen, is falling too. Shortly after writing the article, I began a novel. Telling the story of a party that goes wrong, I felt fiercely determined to describe the present day, and filled the book with all the drugs and technology that people were actually using. In 2005. Since then I have watched the emergence of YouTube, Facebook, the iPhone and Twitter turn my story into a period piece. It never seemed possible that the chemically enhanced high life it depicts could also date so quickly. Looking back, however, we can now see that the days I was describing were the apogee of British drug consumption.

In 2004, Pete Doherty (I am tempted to append "remember him?") was top of the NME's Cool List, with Mike Skinner – then an equally unashamed, if less problematic, drug-user – following in third. In 2007, the hobo stylings of Amy "Wino" Winehouse would follow.

In retrospect, this shift from the romance of Doherty to the squalor of Winehouse looks like a corner being turned. Whether Wino herself actually retoxified the image of intoxicants is hard to know, but by the time she had appeared, being sober had got cooler than it used to be. And compared with her – indeed, compared with anyone – today's rock stars are gleamingly drug-free. Laura Marling, who currently tops the NME Cool List, says she takes no drugs at all. Ditto Ellie Goulding, who spent her teenage years enjoying country walks. Tinie Tempah prays as often as he can, and has spoken of his desire to be a positive role model. Even Lady Gaga, at just 24, has found time to become a zealously reformed cocaine addict (though she remains a periodic weed-smoker). By contrast, celebrities whose heavy drug use has been public – Lindsay Lohan and George Michael, to choose two – now look something worse than wasted; they look old-fashioned.

Sadly, the decline in the use of drugs has not brought a similar decline in the damage they do. Indeed, hospital admissions for drug-poisoning rose last year by 4.8%, and for mental health problems by 5.7%. Measuring drug- related deaths is complicated, but they too seem to be rising. Explaining this is difficult. It could be a lagging indicator, a reflect-ion of the country's ageing population of heroin addicts. "It may in part be because those longer-term users are in increasingly poor health," Barnes explains.

Or, as the erstwhile chair of the Advisory Council on the Misuse of Drugs, Professor David Nutt, suggests, there might be another force at work. "We're certainly getting more alcohol deaths [even as overall alcohol consumption falls]," he says. "So binge-drinking as an alternative to taking other drugs, I think, is probably still happening. But also a culture of bingeing, of getting out of your head and taking lots of drugs, may also be contributing to deaths."

Whatever the true picture, Nutt, who now leads the Independent Scientific Committee on Drugs, is keen to stress that it does not show any kind of victory for drug policy. "The war on drugs is absurd," he says, "but if there was a purpose to it, it was reducing harm, and it's not doing that. It may be actually aggravating harm from alcohol . . . And beyond that, what about all those young people with criminal records for no other reason than there was a sniffer dog at the tube station? The obscenity of hunting down low-level cannabis users to protect them is beyond absurd."

In any case, the rate of illegal drug use in Britain is still among the highest in the world. And when one speaks to young people in the thick of things, the idea of it declining seems ridiculous. "James", for instance, is a 27-year-old from London who began taking drugs more than 10 years ago. For him, besides drinking and smoking, a good weekend with his friends will involve cocaine, ketamine, ecstasy, MDMA powder, speed, mephedrone, DMT, 2C-B or LSD, taken either individually or in combination. He does not smoke cannabis, and finds it easy to believe that others are doing so less often. But cocaine, in his experience, remains extremely popular – even if the poor quality of the powder that pretends to be it has become a standing joke. "It sharpens you up," he says. "That's the reason people still do it . . . Although yes, it's clearly not as cool as it used to be. We always take the piss and say, 'Do you fancy a line of paracetamol?'"

When considering the decline in drug use, it is important to remember this: the substances themselves really are not what they used to be. "When people were purchasing cocaine to use three or four years ago, typically the street level could be as much as 50-60%, or as little as 20%," says Dean Aimes, drugs intelligence adviser at the Forensic Science Service, which analyses drug seizures. "Gradually that has come down and down, however, and now we're talking about a situation where about 3-30% is probably the typical street range."

Over the same period, Aimes has found the purity of speed remaining steady, at 1-10%. As with cocaine, the majority of what people buy is usually caffeine, paracetamol or sugar. Meanwhile, ecstasy, the third most popular drug after cannabis and cocaine, now usually contains no ecstasy at all. "It's a rare drug now, MDMA," says Aimes. "There are hundreds of thousands of tablets in circulation in the UK that look like ecstasy tablets, but which actually contain piperazines [a formerly uncontrolled class of compounds that includes BZP]. . . Piperazines have a similarly euphoric effect, but not to the same degree that ecstasy gives you."

In recent months, the same story has been seen in heroin. "Heroin purity has dropped considerably since November," Aimes says. "Again, typical street purity would have been 20-50%, but we're now as little as 5-15% as a typical range now." This may yet prove to be temporary – there is thought to be plenty of heroin being stored in warehouses around the world – but it is another example of how good, consistent drug supplies can no longer be depended on.

In all three cases – cocaine, ecstasy and heroin – the world's enforcement agencies should claim some of the credit. Colombia's drug crop is now less than half of what it was a decade ago, thanks to the US-funded spraying of coca fields, and a series of military victories against the Farc rebel group that controlled them. One of the main transit routes for heroin into Europe, through Turkey, has been disrupted. And an improvement in cooperation between Chinese and European law enforcement agencies in 2004 has seen the supply of PMK – a vital precursor for making ecstasy – almost disappear.

But what on earth is going on with cannabis? This drug, still by far the most popular in Britain, has been losing users steadily since the mid 2000s, even while the product strengthened – by a factor of two or three, according to DrugScope. This coincides with the reclassification of cannabis, down to class C, in 2004, although it is hard to see how they might be linked – particularly since Gordon Brown's re-reclassification, enacted in 2009, has not reversed the trend.

"A lot of young people who have used the stronger stuff simply don't like it," Barnes suggests. "That could be having an impact . . . What we also have seen, and it could be linked to the overall decline in illicit drug use, is fewer young people smoking . . . We do know that, for young people in particular, if they smoke or drink they are much more likely also to be using illegal drugs. Tobacco is probably the main 'gateway drug'."

This sounds logical. The act of smoking takes a bit of getting used to; if young people are not practising on cigarettes, they are probably less likely to try joints. The fading fashion for cigarettes, in other words, might be dragging cannabis down with it. But then one never knows when an ageing fashion might perk up again.

For 18 years, Dr Fiona Measham has been visiting bars and clubs to ask people about their drug use, so she has seen the story unfold. "I've always been part of this debate about how drug use is increasing," she says, "and now I have to think, Oh hold on, it's not any more." Measham, a lecturer in criminology at Lancaster University, and the author of a new book, Illegal Leisure Revisited, recalls "an unprecedented increase" in drug use, and the acceptance of it, in the 1990s, reaching a "saturation point" around 2002. "It used to be, in surveys, that nine out of 10 people in a rave were taking drugs," she says. "Whereas now it's gone down to about half of the people in the club . . . And even among the biggest caners, they're not doing as much as we thought they were doing."

To some extent, Measham thinks this is cyclical, with each generation reacting to its predecessor and flipping attitudes roughly every 15 years. So where are we right now? "There seems to be a new moderation, at least among some groups of young people. And we can see the first signs of it from about five years ago, so we can't link it just to the economy." This is certainly borne out by the Office for National Statistics figures on alcohol, which show abstinence rates rising.

Could this be evidence of young people becoming, of all things, sensible? Taking any drug is a trade-off, after all, between risk and reward. Smoking, without its convenience or cachet, now offers too little reward to recruit new users; heroin, on the other hand, poses too much risk. Meanwhile, improved enforcement has abraded the quality of pills and coke to such an extent that they no longer glitter quite so temptingly. Saying no for one's own reasons, rather than the government's, has become a respectable response.

"I look at Pete Doherty and Amy Winehouse, and the last thing I would think about them is that they were cool," says "Edward", 21, who has taken cocaine, MDMA powder, mephedrone, acid, ecstasy and cannabis. "I've never taken heroin," he adds. "That's the one drug where I can say categorically that I don't know any young person who would think it was at all cool. I don't know anyone who would touch that with a bargepole."

This may be true, and laudable, but the hunger for intoxication never goes away. If you want proof, look at mephedrone. Under a variety of names, such as "miaow-miaow" and "M-Cat", this synthetic legal stimulant became a phenomenon in 2009, and was added to the controlled list last April. "I think people wanted to believe that there was something new coming out for this era," James remembers. "Everyone talks about how good it used to be, and I think everyone's quite pissed off that there's nothing really for us now."

Edward was at university at the time. "It was quite a widespread phenomenon," he says. "In the third year, there just wasn't much of a supply of drugs other than mephedrone, so people would do a lot of that, because they couldn't get hold of any cocaine or MDMA or whatever . . . Many people who had never really thought of taking drugs did it too. Because mephedrone was legal, I think more people were inclined to give it a go."

Today mephedrone is illegal, however, and is being cut, say the Forensic Science Service, with all the usual adulterants. Yet its popularity endures. Indeed, according to the Mixmag survey, mephedrone is the only mainstream drug that has been taken by more clubbers this year than the year before. Although it is banned – but not because it is banned – M-Cat seems to be of broadly better quality than the alternatives. In the first six months after its appearance in Britain, cocaine deaths fell inexplicably. Perhaps young people are growing up.

The Afterparty, by Leo Benedictus, is published on 3 March by Jonathan Cape. leobenedictus.co.uk