Even though I always made it clear that I only used the place for its swimming pool, people often seemed a little surprised when I told them I had a gym membership. I'm not out of shape, so I wonder if their reaction was down to the fact that I have a beard and somewhat wild hair. I'd be the first to admit that - especially when accompanied by the sizeable flares I invariably wear in my landlocked life - it's not a look that suggests aquatic dynamism.

Nonetheless, for the last three and a half years, I have triumphed against the odds, refusing to see my hirsuteness as a "disability", and managing to propel myself through 25 metres of chlorinated water on the outskirts of Norwich, sometimes over 40 times in a row.

I suppose you could say I recently decided to cancel my gym membership for the clichéd reasons. My gym had started to feel high-maintenance and needy, demanding that I visit it at least three times a week to make our relationship "work". Even when I announced that it was over, it kept calling, to the point where I wondered if I would have to be brutal, and tell it the lie that I'd met a fitter, brighter gym, with massive boobs. There were other, smaller factors. When I'm in the changing room, I'm like most people: I keep my head down, my business to myself, and my turning circle tight, but when a man is standing three feet away from me, blow-drying his pubic hair, I can't help but notice. Also, it was getting kind of tiresome hiding underwater when they played that Duffy song over the PA system.

Another influence on my decision not to swim indoors any more has been Roger Deakin's 1999 book, Waterlog. Deakin died in 2006, but lived a couple of miles away from me in Suffolk, and could well be my new hero. The book tells the story of his journey around Britain, swimming in lakes, rivers, estuaries and the sea, having been inspired by The Swimmer, a story by one of my all-time favourite writers, John Cheever.

Aptly, Deakin was the least dry of nature scribes, and comes across as a rebel spirit, defying local authority jobsworths to swim in rivers, namechecking the Who and Pink Floyd, and even at one point venturing alone into some terrifyingly steep pothole/waterfall combination in North Yorkshire. Essentially: living the kind of free-spirited rural life I would like to, if I was a little less worried about what people thought of me. Were he still alive, I'd probably nip down the road to his farmhouse right now and ask him to be my uncle.

I'd initially been a bit sceptical about wild swimming. I saw it as a slightly overegged term – a bit like describing lawn mowing as "wild vacuuming" - that got dropped into the conversation by trendy people in Hackney who would never actually get around to doing it. However, a leap into the bracingly cold waves on the Suffolk coast can shove such reservations to the sidelines astonishingly quickly.

It's early days for my own wild swimming adventures, and I've only taken baby steps so far. I donned a wet suit for the first time in my life, and threw myself into a choppy Atlantic near my friend Emma's house on Cornwall's north coast, but quickly got knocked back sprawling on to the shingle by a wave slightly bigger than Padstow, the feeling of suddenly being ten years old again amplified when I was tutted at by an authoritative-looking surfer.

Here in Norfolk last week, I edged tentatively into the Waveney, Deakin's favourite river, in a reedy patch between Diss and Bungay, and did an hour's breaststroke in just my trunks, feeling simultaneously preposterous and pleasantly naughty. I emerged with silty toes and a gash on my knee that I had no recollection of sustaining.

I've also toyed with the idea of having a dip in the lake at the end of my garden. I decided it was best to give this a miss, after reminding myself just how much litter it contained, but in my test paddles did find out that a giant moustachioed fish was living on my property. This might not have advanced my relationship with water, but has made me start to newly think of myself as the kind of person who lives with a giant fish with a moustache, which is arguably just as good.

Like so many authors who work from home, I've been spending too much time on The Internet recently, my attention span seduced, manipulated and ravaged, in particular, by Twitter. Swimming outside has provided the extreme antidote I need. I've not felt this antidote's impact any more acutely than during a pair of recent two-hour sea swims at Dunwich, on the Suffolk coast.

"There is no anti-depressant like sea swimming," said Deakin. From time immemorial, fed up people have found staring at the sea's vastness a very quick way to reduce the scale of the problems in their mind. To make the next step and actually get in it is an amplification of that: an abrupt, invigorating reminder how easily it is, in a technologically enslaved modern life, to slip gradually away from a true sense of being alive.

My swims at Dunwich made me realise how rarely I'm truly there, in the present, how often I'm fantasising about an optimistically edited past or worrying about the future. This was perhaps because of the balmy quality of the day, the fresh fish I'd not long ago eaten, the lack of crowds, and the nature of Dunwich itself: a place of rare marshy beauty, one of Britain's biggest ports before it crumbled into the sea 700 years ago, where, legend has it, the bells of sunken churches still ring underwater.

I feel sure, though, that the secret was in the water itself. I'd felt a minor version of this back in the pool in Norwich, leaning back and watching the sunlight make pretty patterns on its surface, until a blood-caked plaster floated past my face, disturbing the reverie. But the sea was magical. Back in the summer of 2000, in Whitstable, it had cured the worst hangover of my life, instantly. What other explanation was there for that, aside from magic? There was something primal and Pagan and purifying here that I spent too long out of touch with, I thought, as I floated, on my back, with the upside down view of Southwold lighthouse in the background.

And even though a stray, floating lunchbox, containing the remains of what appeared to be a corned beef sandwich, did choose that exact moment to bob along and hit me in the cheek, it was not quite enough to dissuade me from my convictions.
• Follow Tom Cox on Twitter or at tom-cox.com.

Wild Swim by Kate Rew is published by the Guardian and covers the best spots in the UK for swimming outdoors. Available for £9.99 (save £4.99) from the Guardian Bookshop