You don't get many casual visitors where I live. Home is down a dirt track, three kilometres from the ­nearest village. The ­neighbours are five ­minutes' walk away, and when the woods are in leaf you can't see another building. By day you may hear a chainsaw in the distance; by night only the deer barking. I'm on my own here, if you don't count the cat, and right now it's a week since I saw another human.

But it's a beautiful spot, in a lovely bit of France, and every now and again a rambler will find his way to it. If I'm outside, and don't have time to hide, his first remark will usually be: "This is a little bit of heaven."

I'll feel a little bashful, as if I somehow shaped this world with my own talented hands, and then he'll say: "Mind you, I could never live here."

After seven years of this, I no longer ask why. Most people, it is clear, are happy to spend a few hours far from the crowds, but the thought of days or weeks like this fills them with dread. They'd be afraid, they say. They wouldn't be able to sleep. Most of all, they'd be lonely.

Now, I'm not an anchorite. I do have a car, a landline, an internet connection. Neighbours come for dinner, my daughter visits for long weekends, and in school holidays friends arrive with their kids. But when I explain this, it doesn't cut any ice. Poor sod, I see ­people thinking. He lives on his own. In this emptiness!

You know what, though? It can be marvellous. For some of us, as Anneli Rufus puts it, solitude is "just what we need, the way tuna need the sea. Here we are, not sad, not lonely, having the time of our lives."

You haven't heard of Rufus? She's the American author of Party of One: A Loners' Manifesto, a fierce defence of all us misunderstood and misrepresented independent spirits. Being alone, she says, "feels calming to me, and invigorating, and most of all ­normal. The rest of the world can get by without us".

What's so great about the life less shared? Let's start with a trivial ­example. I'm writing this article in my pyjamas, though it's late afternoon. I've brushed my teeth, and ­shuffled into some slippers, but I haven't washed or broken out the deodorant. If I shared my home with someone, I'd be dressed by now – not because I wanted to, but because it would be expected. If a stranger were to knock at the door, I'd probably feel obliged to make up some lie about being ill. That may seem like a petty hardship – it is a petty hardship – but so long as no one else suffers there's a lot to be said for doing precisely what you want, when you want, how you want.

Richard Byrd would back me up on that. In 1934 the great explorer spent five months alone in a hut in Antarctica, manning the southernmost outpost of an American expedition. His wife and four children were in the United States; the nearest human was 200km away. "I wanted something more than just ­privacy in the ­geographical sense," he explained in his bestselling book Alone. "I should be able to live exactly as I chose, ­obedient to no necessities but those imposed by wind and night and cold, and to no man's laws but my own."

After a few months of splendid ­isolation, Byrd felt able to draw some conclusions. "Solitude is an excellent laboratory in which to observe the ­extent to which manners and habits are conditioned by others," he wrote. "My table manners are atrocious – in this respect I've slipped back hundreds of years; in fact, I have no manners whatsoever. If I feel like it, I eat with my fingers, or out of a can, or standing up – in other words, whichever is ­easiest. What's left over, I just heave into the slop pail, close to my feet."

Is there anyone who lives on their own, with a sink full of dirty crockery, who can't relate to that?

Then there's the quiet. I moved from Britain to France to be nearer my daughter, who was growing up here with my ex. But if I ended up in such seclusion, it wasn't because I was a stupid foreigner leaping on the first shack the estate agent suggested. We loners are driven to escape the empty chatter that fills most lives. We may occasionally catch ourselves talking to our pets, or muttering when we can't do the crossword, but we don't want endless conversations about the price of Friskies, or how we thought it was going to be nice but actually it turned out a bit cloudy. We may not get round to thinking deep thoughts, plumbing the depths of our souls, getting in touch with our muses, but that will be down to indolence or insensitivity, not to the white noise that fills more ­sociable heads.

And every now and again, one of us does get round to producing that Great Work, like Sara Maitland with her Book of Silence. Divorced, her two children long since grown up and gone, she lives on a Scottish moor with a ­little dog called Zoe. Her neighbours are barn owls. She has no TV, radio, hi-fi or ­mobile phone, and tries to limit all social activities to six days a month. I attempted to have a chat with her, but – unsurprisingly – she never replied to my emails.

Maitland's book, however, ­explains how she, a deeply committed ­Christian, spends three hours a day praying. She has fallen in love with silence. It "can be calm or frightening, lonely or joyful, deep or thin. There is religious silence; a self-emptying silence, and romantic silence – what Wordsworth called the 'bliss of solitude'."

Socialising – as exhausting as giving blood

The bliss of solitude! When the man from Scotland on Sunday forced his presence on Maitland in 2008, he reported that she was hoping to spend the third Christmas in a row on her own. "Her life is, by modern standards, one of extraordinary isolation and ­self-denial," he wrote. "Maitland lives more like an early Christian monk or biblical hermit than a 21st-century woman."

If it wasn't for the fact that neither of them enjoys heart-to-hearts, she'd have plenty to chew over with Anneli Rufus. Rufus says she finds socialising as exhausting as giving blood: "After three hours I'm drained, even if I love the person I'm with. People assume we loners are misanthropes, just ­sitting thinking, 'Oh, people are such a bunch of assholes,' but it's really not like that. We just have a smaller tolerance for what it takes to be with others. It means having to perform. I get so tired of communicating."

Remarkably, Rufus a) is married, and b) lives with her husband. People call her a hypocrite, she says, but her husband's a loner too, and their lives are as separate as cohabiters' can be. "We don't have people over, or do things in groups, and when we're at home ­together we'll often be at different ends of the house. We're not here sitting talking all day and playing games."

Once you've finished with that ­vision of domestic bliss, meet another great introspector – the Canadian Bob Kull, who chronicled his own retreat from society in Solitude: Seeking ­Wisdom in Extremes. In 2001, Kull's desire to ­understand himself and the world took him to an uninhabited ­island off the coast of Chile, 150km from the nearest settlement. He spent just one year here, building a home, fishing, collecting firewood, pulling his own rotten teeth, getting in touch with nature. He had just one visit, from a group of officials checking that the "lunatic gringo" would ­survive the winter. It was hard and ­dangerous, especially for a middle-aged man with a prosthetic leg. As well as the ­elements and assorted injuries, he had to contend with mood swings. "We all carry some pretty dark shadow material in us, and it starts to bubble up," he says. But he got through it, though one call on the satellite phone would have brought a boat to whisk him away.

Rufus, Byrd, Maitland, Kull, all those round-the-world sailors, all those buckskinned mavericks who trapped and traded their way across the New World: they show that solitude can be not just survived but embraced. It helps, of course, that this was something they chose. It must be immeasurably harder if you find yourself alone because of bereavement, old age, or the end of a love affair.

Last Christmas, half a million ­elderly Britons reportedly had no one to celebrate with. "I've had some very good friends in my time," one 89-year-old told Radio 4. "They're all bloody dead." That's not a life anyone would envy. Nor would you want to cut yourself off from the world when your health begins to fail. Yet most people find it hard to contemplate even the kind of watered-down, voluntary ­solitude that my life consists of. ­"People who are not this way really don't get it," Rufus suggests. "And when you don't understand something you fear it."

According to the anthropologist Robert Sussman, humans evolved not as hunters – as we like to imagine – but as prey, easy meat for wild dogs, ­crocodiles and hyenas. We became "social animals", as biologists describe us, not to catch dinner, but to avoid ­becoming it – out of fear, in other words. Perhaps that fear still marks us, and perhaps that's why we still feel uneasy when there's no one around to watch our backs.

I asked a few friends when they had last spent 24 hours without human company. "That's a tough one," one 40-year-old woman said. "A whole day, you mean?" No, a whole day, evening and night. "I simply couldn't!" She has a young son, which would make things difficult right now, but what about before he came along? "Twenty-four hours, without seeing anyone at all? It's never happened to me." Elsewhere, a few people suggested that, they guessed, it might possibly, perhaps have occurred a decade or two ago, when they were living on their own, or sharing with friends who had pushed off for the weekend. They were definitely ill, or they'd have invited someone over, or gone a-visiting.

Are people uncomfortable with solitude because they so rarely experience it, or do they so rarely experience it ­because they are uncomfortable with it? What is clear is that most of us persist in equating aloneness with loneliness, and company with companionship, despite a lifetime of evidence to the contrary. "We are for the most part more lonely when we go abroad among men than when we stay in our chambers," is how Henry David Thoreau put it after two years as the sole inhabitant of a house he had built in the Massachusetts woods. You're never more alone than when you're in a crowd. A cliche, perhaps, but most of us recognise the truth in it.

Before moving to the back of beyond, I spent almost 40 years ­surrounded by people, first as one of five children, then in shared houses, and finally in a succession of London flats. I had girlfriends, a daughter, flatmates, people to the left of me, people to the right of me, people in front, behind and, in the more pleasant moments, under or on top of me. I sometimes feel unloved now, but I sometimes felt unloved then. Doesn't everyone?

Do explorers and hermits miss those they leave behind? Of course they do. "I can't take my loneliness ­casually," Byrd wrote. "It is too big. But I must not dwell on it. Otherwise I am undone." Maitland admits: "There are times when you need somebody's hand to hold."

'You're not responsible to anyone but yourself'

But the reality of solitude is often less daunting than the perception. During his year on the desert island, Kull, a likable man, was buffeted by loneliness. It didn't help that he had begun a "pretty intense" love affair in the time between planning his ­retreat and embarking upon it. But he came to regard his longings as a storm that would sweep in, turn everything ­upside down, then sweep out again. The best thing was simply to enjoy the spectacle: "Instead of running from that feeling of loneliness, whenever I allow myself to settle into it, and really, fully experience it, it opens out into a sense of wonder and peace." After three or four months, he recalls, "I became more engaged with the world around me – the trees, the ocean, the sky. My whole orientation changed. We are social beings – there's no doubt of that. But even when we're alone, we're engaging with the world."

In the 1980s and 90s a British man called Les Powles sailed three times round the world – always single-handedly, once non-stop. He couldn't afford a radio transmitter, and on his greatest adventure he didn't speak to anyone for 329 days. At 84, his ­circumnavigating days are now behind him, but he still lives on his boat, the Solitaire. What's the ­appeal of sailing, I asked him. "It's the solitude. When you're out at sea on your own, there's no government or bankers to worry about. You're not ­responsible to anyone but yourself."

Powles has been married twice, and has friends in many countries. Doesn't he miss people? "I get lonely at ­airports, or saying goodbye to somebody, but not when I'm at sea. There's no comparisons, you see. I'm not ­seeing people with their families. I'm so isolated out there."

There was one moment on that non-stop trip when Powles ended up in tears – on Christmas Day, just off ­Australia, with his transistor radio picking up some seasonal nonsense. Friends in Britain had given him a ­special meal to open on the day, ­complete with greetings card. "I was lonely then, I must ­admit. I just wanted to say 'Merry Christmas' to someone." But he'd recovered by the time he reached dry land. "When I got back," he says, "I would have happily set off again."

And that's after almost a year of proper hardcore solitude, the sort so few of us will ever experience. The worst we have to fear is a kind of ­solitude lite, where even if we don't see anyone for a few days, we still have the phone, email, Facebook, instant ­messaging, Twitter.

It's a great life if you don't weaken. Every now and again, the need to scratch out a living forces me out of my lair. "A week in London," I tell myself. "I can do that standing on my head." And, to tell the truth, I can. I enjoy the chance to catch up with people. Like many loners, I don't have that many friends, but the ones I do have, I value. I talk with them, eat with them, drink with them – all those things normal people do. Still, whenever I return home, it's with relief that I shut the door on the world. It's a fantastic place to visit, but I wouldn't want to live there.

And now, if you'll excuse me, I'm ­going to put on my warmest coat, grab a beer and go outside to watch the sun go down. We loners love our rituals, and this is something I try to do come rain, shine or biting cold. I'll be on my own, of course, but channelling ­Richard Byrd.

On April 14 1934, with the temperature far below zero, Byrd enjoyed a magical sunset, the ice like platinum, Venus an "unblinking diamond" in the sky. "I paused to listen to the silence," Byrd wrote. "My breath, crystallised as it passed my cheeks, drifted on a breeze gentler than a whisper . . . The day was dying, the night being born – but with great peace . . . In that instant I could feel no doubt of man's oneness with the universe."

There was, you will note, no one standing beside him complaining about the weather.