This summer, as the weather grew warmer, I prepared myself for the shock of people's shoulders, for the flashes of thigh, for the lily-white skin turning pink at high noon, for the smalls of backs, the napes of necks, the sight of hairs – normally hidden – peeping timidly out from underarms, for the dimples on upper arms, as well as legs, caused by good dinners and bad drinking. But I did not prepare myself for the Y-fronts. Walking through my local park this week, the cement of the tennis courts sizzling, the children's playground crowded and cheerful, past the mothers in shorts and the girls in bikinis, there they were. A row of men, stretched out on towels, in nothing but their tight white pants. Really, I thought. Is this what we're doing now?

The answer, emphatically, is yes. Over the last few weeks, as the temperature has reached 30C, the British love of the sun – and the associated tan – has been in evidence in parks, gardens and festival grounds, on beaches, campsites and roof terraces. Following three terrible summers, including the crushing disappointment of 2009, the British have rapidly abandoned their shirts, trousers and bras. The pants are still on, but who knows for how long?

It's proof of our deep, deep love for a deep, deep tan. Over the last decade, our commitment to tanning – whether the sun's out or not – has taken on epic proportions. The market for self-tanning products in Britain has been estimated at £41m a year, an amount that has grown five-fold since the late 1990s, and includes spray tans, applied by professionals, and all manner of fake tans applied ourselves. Popular brands such as Fake Bake, St Tropez and Johnson's Holiday Skin crowd the shelves. Fake tans come in mousses and gels, sprays and oils, all offering the spectre of summer in a can.

Many people have also taken the more controversial path of lying for extended periods, goggles on, beneath ultraviolet strip bulbs. Last year the International Agency for Research on Cancer reviewed the way it categorised sunbeds, moving from the judgment that they were "probably carcinogenic to humans", to conclude that they definitely are. They are now classed in the same highest-risk category of carcinogens as cigarettes and asbestos. Yet while tobacco has become taboo, attempts to stigmatise and subdue interest in tanning have been much less successful. The market for sunbeds remains enormous and is apparently growing. The Committee on Medical Aspects of Radiation (Comare) has estimated that there are 8,000 sunbed providers nationwide, and that the number is rising; the popularity of sunbeds is such that, in 2008, a survey of 4,000 people found that one in three women had used them. In fact, our pursuit of the perfect tan has become so dogged that in the last few years there have been reports of people injecting themselves in the stomach with an unlicensed tanning drug.

What explains this devotion? It can't be ascribed to high fashion. The most cutting-edge swimsuits of the summer have holes that highlight the waist and cutaways at the navel: a nightmare of straplines. And although tanned skin has had its moments in vogue – the modern love for the sunkissed look has often been traced back to a yachting trip Coco Chanel took in the 1920s – the current trend is for a more subtle, natural appearance. Emma Elwick-Bates, market editor at Vogue, says that the days of "fake-tan Fridays" in her office, "when lots of people would go and have a spray tan, and quite often turn into a bit of an Oompa-Loompa in the afternoon, seem to have gone". She says that, as fake tans have become darker, they have almost reached the point of parody. "I do think that look hit the wrong end of the taste barometer," she says, "and became more about cartoon kitsch than the look of a healthy woman. I don't necessarily think people want that kind of hyper-femininity any more."

That may be true in high fashion but, in the world beyond, the love of a deep tan is as popular as ever. Kiren Awan has been providing mobile spray tans for the last year; she worked in property previously. "I had to find something that was still relatively lucrative in a really bad time," she says. Spray tanning has proved the answer. Awan visits people in their offices or homes, her portable spray tanning cubicle in tow, and her clients include men and women of all skin tones. "I've got lots and lots of Indian clients," she says, "a lot of Mediterranean people, Caucasians, everybody . . . I spray-tanned two black women recently, and they absolutely loved it. Everybody feels a bit drawn when they get a little paler, and the spray tan gives a pinky-glow."

Curiously, we still associate tanned skin with good health. Our love of tanning has led to the incidence of malignant melanoma quadrupling in Britain over the last 30 years; it is now the second most common cancer in young adults in the UK. But such figures don't put us off. The Teenage Cancer Trust (TCT) is running a Shunburn campaign – encouraging teenagers to use high-factor sun cream – after its survey of 13- to 19-year-olds found that 31% admit to never using sun cream in the UK. Our devotion to tanning seems to know no bounds.

So if this isn't about high fashion or good health, what's the explanation? Robert Mighall, a cultural historian and author of Sunshine: Why We Love the Sun, believes the rush to tan is explained partly by the simple fact that the sun feels so good on our skin. "There are various chemicals that it triggers – serotonin and beta-endorphins – so we all respond to that", he says. Then there is the scarcity of hot weather in the UK. "We value what we can't have, and we value it more if we get a little glimpse and it's taken away. So, great, we're having a decent summer this year, but we've had three lousy ones. That's what really keeps us on our toes. If something flirts with us the way the sun does, we want it all the more."

The problem with this scarcity is that it prompts us to behave as we do with alcohol, food and drugs in the UK – as a binge nation. "It's very simple," says Mighall. "The more we're denied something, the more we want it." And so we go overboard. We head into the parks, strip off and get burned, and for many of us, the scarcity of the sun probably makes this seem justifiable – we may have bad habits, but if they're confined to just a few weeks every four years, surely that can't hurt?

But, says Simon Davies, chief executive of the TCT, "the reality is that one serious burning could develop into a melanoma in years to come, so it's very risky behaviour. And it's unfortunate that people think the sun isn't as dangerous in this country as they believe it to be abroad. That's not the case."

Of course, many of those who love tanning would attribute their obsession specifically to how it makes them look. "Generally, people who have a spray tan look better afterwards," says Awan, "and they feel better. I have people who say 'I love it, because when you spray my legs, you're colouring in my cellulite.' They feel thinner, like they look a better colour. They feel like they can get their legs out and wear dresses that they couldn't wear otherwise".

But I still wonder why we are so attracted to tanned skin at this moment in history. Tanning has always gone in and out of fashion. When workers typically toiled outdoors, the tan was deemed declassé, and the aristocracy kept their skin as pale as possible. When workers moved into factories, tans became associated with leisure, and the upper classes embraced them. We are now living in a cultural moment in which the absolute majority of us work indoors. Denied the sun, denied the outdoors, perhaps a tan – however fake – gives us a much-needed flavour of summer every time we look in the mirror.

As Mighall has pointed out, sunshine is "a prime metaphor for happiness", a state associated with childhood, beaches, holidays and ice creams, with warmth, picnics and long, cool, calm blue swimming pools. Sunshine bathes our happiest memories (even if they took place in a seething winter gale, that's the light we remember them in), and is associated with relaxation, ease and bliss. On that basis, it doesn't seem so surprising that we'd want to wear it on our skin. But as the hit song reminded us, there is one proviso we should always keep in mind. Wear sunscreen.