My old friend Mick calls me with an invitation to his 50th birthday party. It sounds brilliant. Mick has rented a house for a week. Lots of people I know will be there. I want to go. I really, really want to go.

“So is that a yes?”

I want to say yes. I’m about to say yes. But there’s a problem. Part of me wants to say no. Part of me wants to spend that particular week writing my book. Those are days I’ve promised to my book.

At this exact moment, the yes I want to say feels several times stronger than the no I also want to say. There is a yes part of my brain, and it’s overriding the no part of my brain, and I am briefly filled with a blinding flash of “Yes – to hell with it!”

But I don’t say yes. I hold back for a second. What’s important to me here? My book, or a party? On the other hand, why do I write in the first place? To communicate with people, of course! And where better to communicate than at Mick’s party?

But I don’t say yes. Not yet. The no part of my brain is still in the fight, a boxer taking punches. The no inside my brain is resilient. Will you really enjoy the party, it’s asking me. Think of the book. You’ll never write the book if you spend your time going to parties!

Yes or no – the central dilemma. For most of my lifetime, yes has been winning the race. But recently no has made a strong comeback. The forces of no tell you not to give in to your impulsive side, but to look elsewhere in your brain for guidance. There are good reasons for this, mostly having to do with the fact that the world has recently changed very fast. We used to live in a world in which we didn’t need an inner no, because no was all around us. Now we live in a world designed to give us what we think we want. Now yes is all around us.

An outer yes requires an inner no.

Recently, an article in the Financial Times declared that no was the new yes, and cited books with titles such as The Power of No, and How to Say No Without Feeling Guilty. These are trendy self-help books. But I’ve been reading books on the no side of the equation for a few years now. The importance of saying no to your impulses is more than fashionable – it’s a necessity.

In 2002, the behavioural psychologist Daniel Kahneman won a Nobel prize for his work on saying no to your impulses; in 2017, Richard Thaler won another. Dozens of researchers and writers have picked up on the work done by Kahneman and his late colleague Amos Tversky. I have a whole shelf of their books. Of course, there’s Kahneman’s Thinking, Fast and Slow, and Thaler’s Misbehaving. But also The Marshmallow Test by Walter Mischel, The Impulse Society by Paul Roberts, Irresistible by Adam Alter, Your Money and Your Brain by Jason Zweig, Wait by Frank Partnoy. And there’s The Life-Changing Magic of Tidying by Marie Kondo, for those who have said yes too many times, and need to clear their clutter.

You have gut feelings, these authors tell us. You have impulses. Don’t say yes to these impulses. Think again. Say no.

Mick says: “So – is that a yes?”

I’m thinking that I’d love to go to Mick’s party – and also that the word yes is actually affecting the way I think. Over the last few years I’ve been to several sales seminars hosted by Jordan Belfort. The Wolf of Wall Street says that, if you want to generate a yes, it helps to use exactly the right tone of voice. Then ask the person a few innocuous questions and get them to agree with you – these utterances are called “micro-agreements”. Saying yes makes it harder to say no.

Yes! I want to say the word. I want to be marinated in a mutual moment of yes.

But I don’t say the word. Not just yet.

The psychologist Steve Peters gives one of the clearest metaphors for the impulsive part of the brain. He calls it the chimp. Everybody has an inner chimp, he says. Actually, he’s talking about the limbic system, the centre of emotions. We all have an inner human, too. Your inner human is located in the prefrontal cortex. The chimp makes snap judgments. It’s obsessed with food, sex and immediate danger. It’s there for a reason. Back when we were chimps, it was right most of the time. It made you fight when you needed to fight. It made you kill when you needed to kill. It was grabby, jealous, greedy. It still is.

The prefrontal cortex, on the other hand, is what makes you human. Some people, Peters points out, might say it’s where your soul is. It seeks the truth. It looks for evidence. It argues rationally and sees the world in what Peters calls “shades of grey”.

One of the most important things Peters says is this: the inner chimp is much more powerful than the inner human. When information reaches the brain, it gets to the chimp first. The chimp has first dibs. Which means that everything you see or hear is framed by your impulses. When you see something you want, yes comes first. Yes is louder. No is a distant second.

One thing you might notice at this point is that we seem to be living in a world designed for the inner chimp in all of us. Politics is for chimps; Twitter is for chimps; clickbait is for chimps. In a broader sense, the internet is for chimps. Not all of it, of course. But watch the blogger Tim Urban’s brilliant Ted talk on procrastination. You start out with the best intentions. But it’s easy to fall down a rabbit hole.

I once had a conversation with the tech guru Jaron Lanier, who has strong feelings about this. These rabbit holes are not just based on our whims. They are engineered for us by expert robots. We are like rats in an experiment designed to get us to say yes. The experiment is relentless; it has almost unlimited resources and all the time in the world.

In The Impulse Society, Paul Roberts writes about a man called Brett Walker, who became addicted to the online game World of Warcraft. “For four years,” writes Roberts, “even as his real life collapsed, Walker enjoyed a near-perfect online existence, with virtually unlimited power and status akin to that of a Mafia boss crossed with a rock star.” In real life, Walker was “physically weak, financially destitute, and so socially isolated he could barely hold a face-to-face conversation”. The inventors of World of Warcraft had designed an astonishingly effective product.

“On the surface,” writes Roberts, “the tale of someone like Brett Walker may not seem relevant to those who don’t spend our days waging virtual war.” But that’s just on the surface. Roberts makes a powerful point: the rest of the world is more like an addictive game than you might think. Not just Netflix and Amazon, which are guided by rabbit-hole algorithms. It’s more than that, he says. It’s the whole consumer society. It’s the way we “fine-tune our lives”.

It’s a world geared to the chimp – or, in psychologist Walter Mischel’s terms, the “hot system” of impulse, rather than the “cool system” of taking stock, looking for evidence, thinking about what’s best in the long term. Daniel Kahneman refers to “system one” and “system two” – gut reaction versus taking a moment or two to think things through.

Jason Zweig refers to “thinking” and “feeling”. I think that Paul Roberts has put his finger on the reason lots of people are interested in this distinction. It’s the world. It’s changing. It’s designed for the chimp in all of us.

I interviewed Mike Morhaime and Frank Pearce, the inventors of World of Warcraft. “As I got older,” Morhaime told me, “I became fascinated with how computers worked. How everything evolved. And so I decided to go into electrical engineering. I was just so fascinated,” he said. He looked down at the table, and then up again. “I’ve just always been interested in the capabilities of different devices,” he said. “And getting them to talk to other devices. All the inputs and outputs on them, you know.”

Morhaime and Pearce didn’t have an office. They had a campus in Irvine, California. Lovely modern buildings and neat lawns. Frank Pearce’s office was full of plastic figurines of characters from the game – orcs and elves. And, of course, humans. “Mike is better at relationships and interacting with people than I am,” he told me. I was trying to find out why the game was so popular; why millions of people spent 20 or 30 hours a week playing it. Morhaime and Pearce had manipulated algorithms and got them to talk to each other. They had fine-tuned the algorithms for years. Eventually they had found a sweet spot in the human brain.

In the lobby was a map of the world that lit up. It was a data field of where people were saying yes to the game, in real time. They were saying yes in Los Angeles, Boston, Caracas, and Buenos Aires. They were saying yes in Madrid and Beijing.

Morhaime and Pearce are only the beginning. That’s what I imagine Jaron Lanier would say. That’s what Paul Roberts would say.

Mick says: “I’d love you to come.”

And I would love to come. I can always write my book another time. My brain runs calculations. It’s doing what behavioural psychologists call “future-discounting”. I want to say yes now. If I say yes now, I’ll be happy. Then I’ll be able to look forward to the party. Then I’ll go to the party. That’s a lot of happiness. The chimp in my head is totting everything up. But there’s a problem. He’s using chimp maths. Chimp maths is not real maths. It’s what you use when you’re wondering whether to have the salad or the fries. Look at those fries! They are 10 times better – order them! That’s chimp maths.

Chimp maths says yes to the party. But my prefrontal cortex knows that if I say no now, I’ll be happier – but not until later. The question is: how much is deferred happiness worth? And how do you persuade yourself to say no to happiness now?

I interviewed Frank Partnoy, the author of Wait, a book about the benefits of not capitulating to your inner chimp. Before I met him, he sent me an email: “Given the crush of technology, email, social media, and 24-hour news, most of us react and decide too quickly. We are hard-wired to snap respond to fast, salient stimulus, even when it is to our disadvantage.” In other words, the world has got too fast for us. Things have evolved to grab the chimp’s attention. Food is fast. It’s engineered to get glucose into our blood as soon as possible. Social media is fast. It stirs us up. It engages the chimp.

To thrive, we must slow down. Everybody else is led by their inner chimp. So slow down. If you wait, you’ll win.

In The Marshmallow Test, Walter Mischel tells us about his famous experiment involving children and marshmallows. Partnoy describes it, too. It’s one of the most influential experiments in recent behavioural psychology. In the experiment, conducted at Stanford University in 1960s, four-year-old children were given a choice. A marshmallow was put in front of them, and they were told that they could eat the marshmallow now, or wait 15 minutes, after which they would be given two marshmallows.

The kids were left alone in the room with the marshmallows. Some said yes. Some said no, for the whole 15 minutes. The kids were monitored over the years. Those who had said no got better grades. Also, they “were less prone to impulsive behaviour”, and, according to tests, were “more likely” to be well-adjusted.

Party or book? “The hotter and more salient the desired reward,” writes Mischel, “the more difficult it is to cool the impulsive reaction to it.”

On the other hand: “The power is not in the stimulus… but in how it is appraised.”

And, of course: “The power resides in the prefrontal cortex, which, if activated, allows almost endless ways of cooling hot, tempting stimuli by changing how they are appraised.”

As Frank Partnoy would say: wait. As Steve Peters would say: allow time for the information to move beyond the chimp. You don’t have to be “hijacked by your chimp”. You can let yourself feel your chimp’s emotions, and wait for them to pass. Be rational. Say no to the marshmallow right now, and you might find yourself with two marshmallows in the future.

I’m desperate to go to the party. I really am. It would be such fun. Why not just go for a couple of days? Why not? But that’s chimp maths. I can see what would happen. I’d have to make arrangements, pack, drive to the place. I’d be anticipating it for a couple of days beforehand. I’d be excited and anxious. My focus would go. Then it would take me a couple of days to wind down. I’d lose a week.

“The thing is,” I say to Mick, “I can’t.”