If you’ve ever experienced a feeling of paranoia – that’s to say, an unrealistic or exaggerated belief that other people mean you harm – you’re certainly not alone. Around one in four people have regular thoughts filled with suspicion, and almost all of us experience paranoia at some point in our lives.

For most people these thoughts are temporary and relatively mild. But for a small minority they’re persistent, powerful, and profoundly distressing. In psychiatry the experiences at the most debilitating end of the paranoid spectrum are termed persecutory delusions and they’re associated with a variety of serious problems, including anxiety, depression, and suicidal thoughts. As a result, people with severe paranoia are often admitted to psychiatric care, typically with a diagnosis such as schizophrenia, and are treated with anti-psychotic drugs.

But as we’ve noted previously on Guardian Science, anti-psychotics don’t work for everyone. And their side effects can be so unpleasant that many people refuse to take them. Moreover, there’s compelling evidence to suggest that the concept of “schizophrenia” doesn’t stand up scientifically, operating instead as a catch-all for a variety of distinct and frequently unrelated experiences.

This is why scientists have increasingly focused on understanding and treating those experiences in their own right, rather than assuming they’re simply symptoms of some single (albeit nebulous) underlying illness. So what have we discovered by applying this approach to paranoia?

Well, we know now that paranoia is far more common than previously assumed. At its core is a deep-seated belief that we’re in danger – that we’re not safe. That belief appears to be partly genetic in origin and partly the result of the things that happen to us in our lives (bullying, for example, suffering an assault, or being brought up in challenging urban environments).

Importantly, there are a range of so-called “maintenance factors” that increase the chances of paranoia taking hold: sleeplessness; thinking negatively about ourselves and others; a tendency to “reasoning biases” such as jumping to conclusions, not considering alternative explanations, and belief confirmation; and avoiding other people’s company.

From this it follows that if we are able to tackle an individual’s maintenance factors their paranoia should improve too. This is exactly the strategy we’ve been pursuing in recent research, carefully altering one maintenance factor at a time, observing the effect on the suspicious thoughts, and thus aiming to develop a precisely targeted – and therefore more effective – psychological treatment for paranoia.

The results of some of that research are published in the journal Lancet Psychiatry. In what is the first large randomised controlled trial dedicated to severe paranoia, we focused on one contributory causal factor: worry.

Excessive worry, as we’ve noted previously, is associated with a host of psychological problems including post-traumatic stress disorder, alcohol and drug problems, insomnia, and eating disorders. That it should also play a significant role in paranoia is hardly surprising: after all, worry tempts us to give houseroom to the most implausible and distressing ideas. Patients have told us:

“It’s totally um … drowning. The fears.” And: “It’s a general feeling that your state of mind is in control of you rather than you in control of it.”

Our trial (a multi-disciplinary collaboration between the universities of Oxford, Southampton and Manchester funded by the UK’s Efficacy and Mechanism Evaluation programme) involved 150 patients with persistent paranoid beliefs. Most had experienced problems for many years, were taking anti-psychotic medication, and hadn’t previously received help from a clinical psychologist. They were worriers, as pretty much all patients with these delusions are.

We wanted to see what would happen if we could reduce these people’s level of worry (but without attempting to persuade them that their paranoid thoughts were wrong). To find out, we randomly allocated half of the group to a six-session course of cognitive behaviour therapy, plus their usual treatment; the other half carried on as they had been doing.

The CBT treatment took place over eight weeks and was specifically aimed at tackling the patients’ worry. Participants were taught about the causes and effects of worry; they were helped to identify and evaluate their positive and negative beliefs about worry; and to think about the kind of events that typically triggered their own bouts of worrying. They learned how to restrict their anxieties to brief daily “worry periods”, and tried scheduling enjoyable and absorbing activities for the times of the day when they were most prone to worry. Participants also practised “letting go” of their worry: understanding that thoughts are not facts and that we can learn to watch them come and go in our minds without becoming distressed.

Participants were assessed before the trial, at its conclusion, and then again at 24 weeks. This was a “single blind” study, meaning that the assessors didn’t know which of the patients had received the CBT.

The CBT sessions proved popular with the patients. Most importantly, they led to a significant improvement in levels of both worry and paranoia, and the gains could still be seen at the 24 week assessment. These benefits were what scientists call “moderate” – not a magic bullet, but with meaningful effects nonetheless – and are comparable with what’s seen with many anti-psychotic medications.

Mediation analysis (essentially a sophisticated statistical procedure) showed that two-thirds of the improvement in delusions was the result of the change in worry. This is convincing evidence that worry doesn’t merely tend to crop up alongside paranoia: it can be a cause. Incidentally, the CBT didn’t just help with worry and paranoia; it was also somewhat effective for levels of well-being and psychiatric symptoms.

It’s worth noting that we don’t know which elements of the CBT were most effective. It may be, for instance, that patients benefited to some extent from time with a skilled therapist. And although the gains were substantial, participants still experienced high levels of worry and paranoia. That suggests that the intervention is best regarded as part of more effective therapy, rather than the sum total of that therapy. Tackling additional maintenance factors (sleep problems, for instance, or reasoning biases) is likely to be a productive approach – and one that we’re currently piloting.

Nevertheless, the trial does show that a treatment targeting just one maintaining factor can bring about real and lasting improvements in paranoia. It’s a point brought home by “Chris”, a participant in the study:

I needed that kind of therapy at the time because if I didn’t have that therapy at that time I wouldn’t be here. It was therapeutic talking about things. I listened to what you had to say and wrote down how I felt. I also tried relaxing to the tape and I ignored people when they were horrible to me. It was hard becoming disciplined but we worked as a team, that’s what I liked about it. … I couldn’t have been able to do it by myself, no way. I thought a lot about what I thought the therapy did - it decreased my worrying but in other ways it built my confidence.

Follow @ProfDFreeman and @JasonFreeman100 on Twitter