Barbara Oliver has had an intriguing relationship with her identical twin sister, Christine, over the decades. Throughout their childhoods, they were effectively treated as two versions of the one person: they were dressed in exactly the same manner and were given the same hairstyles. "Our parents did everything to stress how similar we were," Barbara recalls.

But when Barbara and Christine reached adolescence in the 60s, the pattern changed. The girls could choose their own clothes and adopted very different fashions. "I wore short skirts. Christine had longer dresses and jackets," says Barbara. At the same time, differences in their personalities became more apparent. "Christine is more conscientious about what she does. I am more confident. That became increasingly obvious as the years went by," says Barbara.

Christine agrees. "I am more self-conscious and I suffer from serious depression. There is no sign of that in Barbara. We may be identical twins but we are very different in many ways."

Such a divergence might seem odd. After all, as identical twins, the pair have exactly the same genes. They are clones of each other. They also had an upbringing that accentuated their similarities. Nature and nurture would appear to have dealt them identical hands. Yet Barbara and Christine have ended up as dissimilar individuals.

Nor are they unusual, says Professor Tim Spector, head of twin research at King's College, London. Barbara and Christine, who enlisted with the college's twin studies unit several years ago, are like many identical twins. In some ways, they are very, very alike, in looks, for example. But in other ways, they are noticeably dissimilar – and that is far harder to explain. "We see it in so many different ways," says Spector. "For example, our research has shown that twins rarely die of the same disease. Yet they share many other features, such as height. It is not a straightforward business."

It sounds baffling. After all, identical twins have the same genes, share the same womb and usually experience the same childhoods. "Most of the twins recruited to our study went to the same school and lived together, eating the same food for the first 18 or so years of their lives," says Spector, whose pioneering study celebrates its 21st birthday next month. "But the outcomes of their lives are often very different indeed."

It is an intriguing discovery and it forms the core of a new awareness of the behaviour of genes and their interactions with the environment, one that may explain the baffling roots of human variation, though Spector, when he started his study, had slightly less ambitious goals. In the 1990s, he was studying the roots of common ailments such as cataracts and arthritis. At the time, doctors dismissed these conditions as the results of wear and tear on patients' bodies as they grew older. These ailments were just something that people had to put up with. "However, I wanted to know why some people got hit quite early and not others," says Spector.

And that is where his study of twins began. By comparing identical and fraternal twins and their sensitivities to illnesses, it is possible to separate the genetic roots of conditions from their environmental influences. So Spector began recruiting them for his research and set up his unit at St Thomas' hospital, London. He did so at a time when the first breakthroughs in modern genetics were taking place. In the late 80s and early 90s, researchers – using the tools of modern molecular biology – were starting to pinpoint single genes that were responsible for deadly but relatively uncommon inherited ailments such as cystic fibrosis, Huntington's disease and muscular dystrophy. The roots of widespread ailments such as heart disease and diabetes, although more complex and possibly involving up to a dozen genes, would soon follow, it was expected.

"Scientists would analyse genes and find a link between a group of them with a disease," says Spector. "Thousands of these gene studies were carried out but most of them were false because the researchers did not or could not replicate their results. This was an era of hype. If you got a negative result, you simply didn't publish it: 90% of publications turned out to be rubbish."

One such study involved Spector's twin research. Work with them suggested that the vitamin D receptor gene was the single genetic cause of osteoporosis. An error in one variant of the gene made people susceptible to the condition, it appeared. The story made the cover of Nature, every scientist's dream, except that in this case, as in many others, the claim proved to be wrong. "It was a false dawn," Spector says. "We were simply mistaken."

Then came the Human Genome Project, in which the entire 3bn units of DNA that make up an individual's set of genes were decoded. That sequencing technology and more rigorous replication has transformed the study of individual variation and the work of the twin studies group. "We have 7,000 individual twins – and therefore 3,500 pairs of siblings – on our books here. Of these, about half have had their entire genome sequenced," added Spector.

Every year, day-long measuring sessions are held for groups of twins and a host of different parameters are studied. Blood samples are taken, bone density is calculated, lung function is assessed, x-rays taken and full body scans carried out, along with series of psychometric tests. "We go to these events about once a year and they are really great fun. I get to spend all that time with my sister," says Christine.

But what was thrown up by these sessions and the research carried out on the twins left geneticists puzzled. Instead of finding a dozen or so genes for common conditions such as obesity, researchers found that hundreds were involved. "In the case of osteoporosis, which we once thought was caused by a single mutant gene, we now believe that there may be 500 genes involved – interacting to trigger the disease in people at different ages," says Spector.

"These are genes that individually only account for 0.1% of susceptibility for a condition. And even then, these genes, in total, only seem to account for a fraction of the variance we see in the prevalence and severity of these conditions in the population. This phenomenon has a name: it is called missing heritability."

It is an effect you can see directly from the studies of identical twins carried out at St Thomas'. "We now began to look not at the similarities between identical twins but the differences. It was a shift in perception really. Our work shows that the heritability of your age at death is only about 25%. Similarly, there is only a 30% chance that if one identical twin gets heart disease the other one will as well, while the figure for rheumatoid arthritis is only about 15%."

It is a baffling observation: individuals with identical genes and often very similar conditions of ubringing but who experience very different life outcomes. What could be the cause? The answer, says Spector, came to him in a Damascene moment four years ago. The causes of these differences were due to changes in the human epigenome, he realised.

"Essentially, epigenetics is the mechanism by which environmental changes alter the behaviour of our genes," he says. "This involves a process known as methylation, which occurs when a chemical known as methyl, which floats around the inside of our cells, attaches itself to our DNA. When it does so, it can inhibit or turn down the activity of a gene and block it from making a particular version of a protein in our bodies." Crucially, all sorts of life events can affect DNA methylation levels in our bodies: diet, illnesses, ageing, chemicals in the environment, smoking, drugs and medicines.

Thus epigenetic changes produce variation in disease patterns. And recent experiments carried out by Spector and his colleagues, in which they have looked at methylation levels in pairs of identical twins, back the theory. "We have studied identical twins who have different tolerances to pain and shown that they have different states of methylation. We have also produced similar results for depression, diabetes and breast cancer. In each case, we have found genes that are switched on in one twin and switched off in the other twin. This often determines whether or not they are likely to get a disease."

Epigenetic changes are not just simple environmental changes, however. They influence a person's genes and can have an effect that can last for two or three generations in extreme cases. For example, studies of the children and grandchildren of pregnant women who endured starvation in the second world war and in China in the 50s have revealed they tended to be smaller and more prone to diabetes and psychosis. These trends are put down to epigenetic changes.

"Essentially, they are a way to make short-term changes to a generation," says Spector. "A famine strikes but you cannot instantly alter your genes. But epigenetic changes allow you to produce children who are fatter or skinnier or whatever is best suited to the new circumstances. These changes will last for at least two or three generations, by which time you would hope the change in the environment will have passed. It may not, of course."

If nothing else, the idea of epigenetic changes explaining the variability in twin behaviour and illness strikes a chord with Christine. "The idea that I am different from my identical twin sister being due to life events makes sense. Barbara got married first. Many twins will tell you that when that happens the other twin is left grieving. That is how it felt to me. And later I suffered from leukaemia and I have also been divorced. That would leave a mark on anyone. Luck plays its part."

Identically Different: Why You Can Change Your Genes by Tim Spector is published in paperback by Phoenix