Before I have time to ask the Covid-era question “Are we doing handshakes?”, Jimmy Carr has thrust out his arm and grasped my hand. Then, suddenly, he lets go and screams: “Oh God, no! My hand’s covered in Covid!”

Comedians aren’t supposed to be walking versions of their act – Carr says that he’s not. But my first impression is of a man bursting with one-liners. As we walk through his PR’s art-filled offices he gives me a breathless guided tour full of trivia – “I think that’s the actual drum from the sleeve of Sgt Pepper” – and more gags. “People think that one’s good value,” he says as we pass a Damien Hirst sheep in formaldehyde, before glancing back at the other side, which has spent some time decomposing, “but you’re only getting half of it.”

Yet jokes take a bit of a backseat in Carr’s new memoir, Before & Laughter, which he has written as a self-help book. Initially I thought he might be spoofing the genre, but no: it’s an utterly sincere guide for people to achieve bigger things in life, complete with notes on healthy eating, travel tips and – somewhat hilariously given the huge scandal around his tax avoidance in 2012 – managing finances. It’s a strange read. At points it resorts to rather woolly cliches: seize the moment, save for the future, pretend to be happy and you may just become happy. For a man who once had a reputation as the dark prince of comedy, and whose jokes can be edgy, shocking and sometimes plain cruel, it seems very off-brand. What made him want to write it?

“I could have phoned in a showbiz biog of 60,000 words, stuck a couple of pictures in, cash the cheque, great. But I didn’t want to shortchange anybody and in the end it became a labour of love.”

Carr, 49, turns out to be a huge fan of self-help books. He read lots when he was in his early 20s, trudging to his marketing job at Shell each day, dissatisfied with his life and longing for some excitement. “Self-help opened my eyes a little bit to the idea that the rules that affect our lives aren’t written. ‘I’m not the sort of person that goes on stage, because I’m not from a theatrical background.’ Or: ‘I’m not the sort of person who can be on telly because they must be special magical people.’ You’d never think: ‘Oh, no, I could give that a go.’”

Maybe it was those books that made Carr a star. Aged 25, he jacked in his job and – despite having no comedy experience – tried his hand at writing jokes. One-liners were his thing, clever wordplay mixed with a desire to test the boundaries of political correctness. Within a couple of years, he had been nominated for the Perrier comedy award at Edinburgh. Rather disarmingly, he stresses repeatedly throughout his book that anyone could have done this, and that he had no supernatural talent for comedy. “Or any talent at all,” he says. “I’d never written a joke before I was 25. And now I’m good at writing jokes. It’s a learnable skill. And I learned that skill.”

Carr is an engaging presence – friendly, enthusiastic, happy to answer uncomfortable questions, albeit with an unnervingly intense stare at times. He seems a little dejected when I tell him I was more interested in the memoir sections of the book. But he rarely gives much of himself away, so it’s interesting to read such personal material. The book covers most of his life, from growing up in Slough and going to university at Cambridge to meeting his partner of 20 years, the TV producer Karoline Copping, and hosting shows such as 8 Out of 10 Cats. He may not think his showbiz stories are particularly interesting, but I loved hearing about his friendship with Stephen Hawking, whom he would take out for a curry and a musical.

“I think if you have a friend that’s tetraplegic you have to be quite chatty, because obviously the typing takes him so long,” he says, in a remark that feels like one of his jokes, but isn’t. “We’d do shots together sometimes too. His care team said tequila would be too much so he’d be on the Cointreau.”

Carr opens up admirably in the book about his mental health, his problem drinking and the grief he experienced when his mother died in 2001, just as his comedy career was beginning. “I found the book incredibly cathartic to write,” he says. “Especially about my mother. There’s that lovely phrase, that you die twice – once when you die, and again the last time someone says your name. So I loved that thing of being able to talk about my mum.”

The book recounts how he became a sort of surrogate partner to his mum after she separated from his father in 1994 and struggled with depression. As we talk about her, Carr turns the tables and asks me a question: do I interview a lot of comedians? Occasionally, I say. “Well, can I make a suggestion? Ask them which one of their parents was sick. That’s it. That’s the question. It tends to be that they had to make things OK with humour, and then they became comedians.”

Perhaps this unusual devotion to his mum (“I suppose a therapist would tell you I was ‘enmeshed’”) is why he found himself still a virgin at 26, although he says the situation never bothered him. “It’s like, not everyone’s doing that at the same time. But if you’re watching Euphoria on TV as a 16-year-old you’re gonna think: ‘What the fuck? I’ve never had a threesome – what’s going on?’”

He laughs: “Is this making me sound like an incel elder? I did have opportunities but I was bad at reading the signs and I would friend-zone people. A lot of girls I was very, very close to growing up, we had incredibly intimate relationships, but we didn’t have a physical relationship and it was lovely …”

He trails off and reconsiders what he’s just said. “No, I think I probably was a little bit stressed about it, a bit down about it,” he decides. “But it was probably a good thing because if things had been a bit better in my early 20s, I might not have quit my job for comedy.”

Another thing Carr talks openly about in the book is his experience of anxiety (a “constant hum in my life”) and depression (he gets occasional crashes). He describes his first descent into the latter vividly: “Like vertigo … the sensation of falling within a black void.”

“I was happy to throw my hat into that ring,” he says now. “Because can you remember there was an ad campaign about five years ago saying that one in four people have experienced a mental health problem? Well, I think after Covid, we can round that up to … fucking everyone.”

Despite the crashes Carr says he’s “lucky” when it comes to depression. “I can white-knuckle it. I can kind of get through it without meds. A lot of people can’t.”

How often does it happen? “I try not to think about it, because I think you’d be looking for it. But it happens occasionally.”

Before he moved into comedy, Carr had what he thinks was a drink problem. “No great rock-bottom story,” he says, “and certainly not a lot of alcohol. But drinking for the wrong reasons, to get out of my head.”

As a result he spent his first 12 years of comedy success avoiding alcohol completely. “Which was much better. You have better conversations. The only thing about being sober around comedians is that, around 2am, you might as well fuck off home. You’re just gonna be told the same anecdote again.”

As for drugs, “I’ve tried everything once, but I’m not a drug person,” he says. “I’ve met people who are funnier after a couple of pints. But I’ve never once in my life met someone and gone: ‘Oh, he’s a bit quiet, but you’ve got to meet him after he’s had some cocaine.’”

Carr is estranged from his father, and mentions him just briefly in the book. He doesn’t want to say much more today but talks about “detaching with love” – having no ill feeling for his father but being unable to have him in his life. Carr became a father himself two years ago, although it was only recently reported in the media.

“Well, there had been pictures of me pushing a pram in the pictures,” he says. “What did people think was in that? Old CDs?”

He’s a smitten dad, telling me his son’s funny words for different foods (“He says ‘wubbwubb’ for toast … I’m sure everyone is rolling their eyes: ‘Yeah, you’ve had a kid’”). The pandemic gave him more time to spend with his son, and he admits that he would desperately like another but doesn’t know how possible that is. “We were very lucky to do it at all,” he acknowledges. Copping is only a couple of years younger than him.

I wonder if all this talk of wubbwubb has softened his act. “No. I’ve been writing new stuff and it’s brutal. My sense of humour doesn’t change.”

Carr has regularly been in trouble for his gags, from both the left for “punching down” – often towards minorities such as travellers, disabled people, fat people, gay people, victims of sexual violence – and the right. One infamous joke ran: “Say what you like about those servicemen amputees from Iraq and Afghanistan, but we’re going to have a fucking good Paralympic team in 2012.” A recent Guardian review said: “Many of his one-liners are barely jokes at all, just boorish cliches.” But Carr is unflappable when it comes to defending his act. “To be punching down you need to be looking down. And it’s saying you can’t joke about those people, because they can’t take it … whereas actually, some people with disabilities like really rough, dark stuff.”

And, of course, many don’t. In 2019, the charity Little People, which was co-founded by the Harry Potter actor Warwick Davis, criticised one of his jokes that targeted people with dwarfism. “We are saddened that in 2019 we have to deal with such prejudice to the dwarfism community (and other minority groups) in a time where we all strive to live equal lives and celebrate diversity without fear of humiliation or preconception.” Carr uses parts of his book to justify his comedy, but his arguments – “people know it’s just a joke” – can feel flimsy. Today he says: “It’s not as if I’m shouting these things through people’s letterboxes,” which is true, although he acknowledges that with social media the jokes no longer stay within the confines of his show.

Sometimes, though, his jokes land a punch that rings true in a gallows humour kind of way: the Afghanistan joke, for all the furore, seemed to me to be more about the calamitous invasion than mocking the veterans who served in it. There was a brutal honesty to it and Carr has an anecdote to back that up. He recalls being at the GQ awards two or three years later; Seb Coe was onstage with a group of Paralympians. “He talked about how incredible they were at 2012, and said: ‘Obviously the cause of a lot of their injuries were military service.’ I was sitting next to Michael McIntyre and he turned to me and said: ‘He’s doing your bit!’”

Carr says that he’s actually an “uber-liberal” when it comes to politics, but when I ask about the current government he’s remarkably sympathetic to its situation. “I think when we look back on the pandemic, it’ll seem like it was handled pretty well,” he says. “I mean, what else could they have done? It’s not enough just to piss and moan. Not unless you have a strong alternative idea.”

He seems to have witnessed a different pandemic to me, in which well-signposted icebergs were frequently ploughed into and alternative ideas were plentiful, yet often ignored or mocked. Perhaps what frustrates the left isn’t that Carr is unfunny – he’s clearly very skilled with wordplay and springing a surprise punchline – but that he doesn’t use his obvious talents to target the powerful more often.

But then Carr in real life is a much more empathic figure than the one on stage. One thing not covered in Before & Laughter is his relationship with Sean Lock, a team captain on 8 Out of 10 Cats, whose recent death prompted a huge wave of public affection. “After he died I looked back and went: ‘Hang on – was I in a double act?’” he says today, his eyes welling up. “We did 250 TV shows together and I sort of didn’t notice.” He pauses and smiles: “I mean, I don’t think he’d view me as a comedy partner. He’d say: “Get over yourself – I’m much funnier.’”

Carr knew the bad news was coming – Lock had been diagnosed with cancer for some time – but he hadn’t bargained for how hard it would hit him. “I got wiped out. It was the same thing when my mother died and I had to go to bed and sleep for 17 hours.”

When I ask how it felt to be roasted by Lock over the K2 tax scandal (“We all like to put a bit of money away for a rainy day, but I think you’re more prepared than Noah,” joked Lock on the show that followed the revelations), Carr’s voice breaks completely. “When that’s happening to you, you never forget the people who were good to you,” he says. “And he was so good to me. He said: ‘Are you OK? All right, we’ll handle this.’” There’s a little sob, a pause to gather himself. “And then he went out there, was super-funny about it, no judgment.”

Carr believes that today’s climate offers little room for redemption, but in his case he got lucky. “I was publicly shamed for tax avoidance and it was very clear what I had to do: say sorry and pay it back. I didn’t have to. It was entirely legal. But morally I paid everything back. It was as if I’d been on PAYE all the way through. I did that, and then people took the piss, which is the modern equivalent of being in the stocks. I took a tomato to the face from Sean Lock.”

He says the uproar was far worse than anything he has faced over offensive jokes. Even the then prime minister, David Cameron, got involved.

“I’m not an expert on tax accounting, but I think if the prime minister of the country where you live has broken off from the G20 in Mexico to talk about your personal tax affairs and called you out and named you, that might be a problem.”

Surely he must have allowed himself a wry smile during Cameron’s recent lobbying scandal with Greensill? “No, I don’t wish him any ill at all,” he says, before adding some advice that could come straight from his own book. “I think it’s that thing of letting go. If you’re bitter and angry, it’s best to let go.”

He seems to mean it, too. It may not have been for me, but if the book helps anyone else reach the same level of contentment as Carr seems to have, maybe he’s on to something.

• This article was amended on 1 October 2021. Some text was changed to be consistent with our editorial guidelines on references to disability.

Before & Laughter: A Life-Changing Book by Jimmy Carr is out on Thursday, published by Quercus, priced at £20.00. To support the Guardian, order your copy from guardianbookshop.com. Delivery charges may apply.