In the very first instalment of my column for the Guardian’s Weekend magazine, a dizzying number of years ago now, I wrote that it would continue until I had discovered the secret of human happiness, whereupon it would cease. Typically for me, back then, this was a case of facetiousness disguising earnestness. Obviously, I never expected to find the secret, but on some level I must have known there were questions I needed to confront – about anxiety, commitment-phobia in relationships, control-freakery and building a meaningful life. Writing a column provided the perfect cover for such otherwise embarrassing fare.

I hoped I’d help others too, of course, but I was totally unprepared for how companionable the journey would feel: while I’ve occasionally received requests for help with people’s personal problems, my inbox has mainly been filled with ideas, life stories, quotations and book recommendations from readers often far wiser than me. (Some of you would have been within your rights to charge a standard therapist’s fee.) For all that: thank you.

I am drawing a line today not because I have uncovered all the answers, but because I have a powerful hunch that the moment is right to do so. If nothing else, I hope I’ve acquired sufficient self-knowledge to know when it’s time to move on. So what did I learn? What follows isn’t intended as an exhaustive summary. But these are the principles that surfaced again and again, and that now seem to me most useful for navigating times as baffling and stress-inducing as ours.

There will always be too much to do – and this realisation is liberating. Today more than ever, there’s just no reason to assume any fit between the demands on your time – all the things you would like to do, or feel you ought to do – and the amount of time available. Thanks to capitalism, technology and human ambition, these demands keep increasing, while your capacities remain largely fixed. It follows that the attempt to “get on top of everything” is doomed. (Indeed, it’s worse than that – the more tasks you get done, the more you’ll generate.)

The upside is that you needn’t berate yourself for failing to do it all, since doing it all is structurally impossible. The only viable solution is to make a shift: from a life spent trying not to neglect anything, to one spent proactively and consciously choosing what to neglect, in favour of what matters most.

When stumped by a life choice, choose “enlargement” over happiness. I’m indebted to the Jungian therapist James Hollis for the insight that major personal decisions should be made not by asking, “Will this make me happy?”, but “Will this choice enlarge me or diminish me?” We’re terrible at predicting what will make us happy: the question swiftly gets bogged down in our narrow preferences for security and control. But the enlargement question elicits a deeper, intuitive response. You tend to just know whether, say, leaving or remaining in a relationship or a job, though it might bring short-term comfort, would mean cheating yourself of growth. (Relatedly, don’t worry about burning bridges: irreversible decisions tend to be more satisfying, because now there’s only one direction to travel – forward into whatever choice you made.)

The capacity to tolerate minor discomfort is a superpower. It’s shocking to realise how readily we set aside even our greatest ambitions in life, merely to avoid easily tolerable levels of unpleasantness. You already know it won’t kill you to endure the mild agitation of getting back to work on an important creative project; initiating a difficult conversation with a colleague; asking someone out; or checking your bank balance – but you can waste years in avoidance nonetheless. (This is how social media platforms flourish: by providing an instantly available, compelling place to go at the first hint of unease.)

It’s possible, instead, to make a game of gradually increasing your capacity for discomfort, like weight training at the gym. When you expect that an action will be accompanied by feelings of irritability, anxiety or boredom, it’s usually possible to let that feeling arise and fade, while doing the action anyway. The rewards come so quickly, in terms of what you’ll accomplish, that it soon becomes the more appealing way to live.

The advice you don’t want to hear is usually the advice you need. I spent a long time fixated on becoming hyper-productive before I finally started wondering why I was staking so much of my self-worth on my productivity levels. What I needed wasn’t another exciting productivity book, since those just functioned as enablers, but to ask more uncomfortable questions instead.

The broader point here is that it isn’t fun to confront whatever emotional experiences you’re avoiding – if it were, you wouldn’t avoid them – so the advice that could really help is likely to make you uncomfortable. (You may need to introspect with care here, since bad advice from manipulative friends or partners is also likely to make you uncomfortable.)

One good question to ask is what kind of practices strike you as intolerably cheesy or self-indulgent: gratitude journals, mindfulness meditation, seeing a therapist? That might mean they are worth pursuing. (I can say from personal experience that all three are worth it.) Oh, and be especially wary of celebrities offering advice in public forums: they probably pursued fame in an effort to fill an inner void, which tends not to work – so they are likely to be more troubled than you are.

The future will never provide the reassurance you seek from it. As the ancient Greek and Roman Stoics understood, much of our suffering arises from attempting to control what is not in our control. And the main thing we try but fail to control – the seasoned worriers among us, anyway – is the future. We want to know, from our vantage point in the present, that things will be OK later on. But we never can. (This is why it’s wrong to say we live in especially uncertain times. The future is always uncertain; it’s just that we’re currently very aware of it.)

It’s freeing to grasp that no amount of fretting will ever alter this truth. It’s still useful to make plans. But do that with the awareness that a plan is only ever a present-moment statement of intent, not a lasso thrown around the future to bring it under control. The spiritual teacher Jiddu Krishnamurti said his secret was simple: “I don’t mind what happens.” That needn’t mean not trying to make life better, for yourself or others. It just means not living each day anxiously braced to see if things work out as you hoped.

The solution to imposter syndrome is to see that you are one. When I first wrote about how useful it is to remember that everyone is totally just winging it, all the time, we hadn’t yet entered the current era of leaderly incompetence (Brexit, Trump, coronavirus). Now, it’s harder to ignore. But the lesson to be drawn isn’t that we’re doomed to chaos. It’s that you – unconfident, self-conscious, all-too-aware-of-your-flaws – potentially have as much to contribute to your field, or the world, as anyone else.

Humanity is divided into two: on the one hand, those who are improvising their way through life, patching solutions together and putting out fires as they go, but deluding themselves otherwise; and on the other, those doing exactly the same, except that they know it. It’s infinitely better to be the latter (although too much “assertiveness training” consists of techniques for turning yourself into the former).

Remember: the reason you can’t hear other people’s inner monologues of self-doubt isn’t that they don’t have them. It’s that you only have access to your own mind.

Selflessness is overrated. We respectable types, although women especially, are raised to think a life well spent means helping others – and plenty of self-help gurus stand ready to affirm that kindness, generosity and volunteering are the route to happiness. There’s truth here, but it generally gets tangled up with deep-seated issues of guilt and self-esteem. (Meanwhile, of course, the people who boast all day on Twitter about their charity work or political awareness aren’t being selfless at all; they are burnishing their egos.)

If you’re prone to thinking you should be helping more, that’s probably a sign that you could afford to direct more energy to your idiosyncratic ambitions and enthusiasms. As the Buddhist teacher Susan Piver observes, it’s radical, at least for some of us, to ask how we’d enjoy spending an hour or day of discretionary time. And the irony is that you don’t actually serve anyone else by suppressing your true passions anyway. More often than not, by doing your thing – as opposed to what you think you ought to be doing – you kindle a fire that helps keep the rest of us warm.

Know when to move on. And then, finally, there’s the one about knowing when something that’s meant a great deal to you – like writing this column – has reached its natural endpoint, and that the most creative choice would be to turn to what’s next. This is where you find me. Thank you for reading.

• Oliver Burkeman’s book Four Thousand Weeks: Time Management For Mortals will be published next year by The Bodley Head. Find out more at oliverburkeman.com