Are you weary of "new year, new you" positive thinking exercises? Tired of trying to feel Tiggerish in the cold, dark, midwinter mornings? Why not try this quick experiment to redress the balance. All you have to do is imagine that something great has happened in your life: maybe you've run into an old friend; perhaps you've been promoted at work; or you're about to head off on holiday. Now ask yourself what could go wrong. In what awful ways could it all fall apart? What disastrous chain of events might unfold? Don't think solutions, think problems. Open the worry floodgates and allow yourself to be swept away.

As you're likely to discover when you attempt this depressing little exercise, with a sufficiently negative outlook even the happiest moments in life can become a source of anxiety and stress. When we worry, we become preoccupied with an aspect of our lives, desperately trying to anticipate what might go wrong and what might happen if it does. Although we might believe worry is constructive, actually all it usually does is lower our mood. And when we start worrying it can be difficult to stop.

So worry can be an immensely powerful psychological mechanism, but might it also be a defining factor in the development of mental illness? Can studying it deepen our understanding of what mental illness is, how it comes about, and how it differs from psychological health?

As you'll know if you read about the hoo-ha following publication of the latest edition of the Diagnostic and Statistical Manual of Mental Disorders, the number of officially recognised psychiatric disorders has mushroomed in recent years, and now stands at around three hundred. That giant total has attracted a lot of criticism – and with some justification – but in fact many of these conditions are pretty similar. It is better to think instead of three main groupings of disorders: internalising (most commonly, depression and anxiety); externalising (addiction, for instance, or anti-social behaviour problems); and psychosis (with its characteristic symptoms often bracketed under the label of schizophrenia). However, even these three broad groupings share many of their causes, which has led some researchers to speculate that underlying and unifying all mental illness may be a single cause: the so-called "p factor of psychopathology".

At the social level, we know that poverty, isolation, and negative life events all elevate the risk of mental health problems. But when it comes to the psychological p factor, there is increasing evidence that it may be excessive worry. When worry gets out of hand, it now appears, a very wide range of mental health problems can follow in its wake.

This kind of "transdiagnostic" approach represents a major shift in the way we think about worry. Traditionally, problematic worrying has been demarcated as a specific condition: generalised anxiety disorder. And in that box it has remained. (The exception to this rule is depression, for which persistent worry about the past is a recognised symptom. But it's not called worry: it's called "rumination". "Worry" is defined as anxious thinking about the future.)

Yet real life seems to show a lamentable lack of respect for systems of psychiatric classification. Rather than being a separate disorder, excessive worry has been shown to play a significant role in the development and persistence of paranoid thinking, post-traumatic stress disorder, alcohol and drug dependence and insomnia. It has also been linked to the incidence of eating disorders.

The idea that many psychological problems have excessive worry in common seems plausible. As most of us know from bitter experience, worry brings the most unlikely – and unpleasant – ideas to mind, keeps them there no matter how hard we try to shake them off, and convinces us that the events we dread really may happen.

If persistent worry is potentially so damaging to our mental health, what can be done to combat it? Interestingly, we tend to worry less as we grow older. People aged 65-85, for example, report fewer worries than those aged 16-29. But besides simply waiting for the years to pass, the evidence is strongest for an adapted form of cognitive behavioural therapy. This relatively brief, one-to-one treatment is based on a detailed model showing how problematic worry is caused, maintained and overcome. Patients are helped to notice when they're worrying, to interrupt this habitual thinking style, and then try alternative ways of reacting to life's problems.

So far this kind of CBT has mostly been used with people suffering from generalised anxiety disorder. A recent meta-analysis of 15 studies, for example, showed that CBT was far more effective than other therapies (or than a non-treatment control) at helping people recover from generalised anxiety disorder and stay well.

But it is now beginning to be piloted for other conditions – the Oxford Cognitive Approaches to Psychosis Group, for example, is testing its efficacy in severe paranoia.

How does CBT tackle worry? For one thing, it helps people to re-evaluate their beliefs about its benefits. Like many of us, individuals who are prone to excessive worry tend to assume that it helps them. They may believe, for instance, that worrying helps them to anticipate and solve problems; that it provides the motivation necessary to tackle those problems; or that it prepares them for the worst if a solution can't be found. They may even feel that by worrying about an event they can prevent it occurring – despite realising that it's pure superstition. Learning to challenge these kinds of beliefs can be a huge step forward.

CBT also teaches us to confine our worrying to a regular set period of 15 minutes or so each day. When worrying thoughts arise at other times, the trick is to save them for later and let them go. "Expressive writing" can be effective too: you describe your worries in as much detail as you can, focusing on what it feels like, and resisting the temptation to analyse what's causing your thoughts. And don't underestimate the power of distraction: work out when you're most likely to worry and plan a pleasurable, absorbing activity you can do instead.

Many of CBT's techniques for tackling worry are not rocket science: with the right guidance we can all put them into practice. By doing so we're not merely sparing ourselves hours of futile fretting. If excessive worry is truly the p factor it seems to be, we'll also be addressing one of the key determinants of our mental health.

Daniel Freeman is a professor of clinical psychology and a Medical Research Council Senior Clinical Fellow in the department of psychiatry, University of Oxford. Jason Freeman is a psychology writer. On Twitter they are @ProfDFreeman and @JasonFreeman100. They are the authors of The Stressed Sex: Uncovering the Truth about Men, Women, and Mental Health