Steve Christmas was born, in 1958, in unpromising circumstances. His father was a heavy drinker who abused his mother. Neither parent was interested in Steve’s education and bailiffs were regular visitors to the family door. The years of his youth were miserable.

Yet Steve triumphed. He was a hard worker and took jobs in farming before landing work as an insurance salesman and later a manager. He married, had a daughter and led a life graced by achievement – and with some scientific influence, as we shall see.

Many others have also overcome early disadvantage, of course, though the overall picture is not inspiring. Grim beginnings tend to inflict deep wounds. Low social status and income produce individuals who tend to be smaller, slower at reading, poorer in health and younger at death than their more privileged counterparts. As Helen Pearson, the chief features editor at Nature, notes: “Poverty can seem like an inescapable trap.”

Yet this awareness has only been acquired relatively recently and is due, in large part, to a uniquely British experiment: the birth cohort study. These mighty undertakings have been set up by different official groups – the UK has launched five since 1946 – to analyse the lives of thousands of babies from birth through life in staggering detail, from records of birth weights and ages of weaning to reading skills and employment in later life.

“No other country has anything like it on the same scale,” says Pearson in this spellbinding, carefully sourced and elegantly outlined account of these birth cohort studies. “The findings fill over 6,000 published papers and 40 academic books and the results have rippled out to touch almost everyone in Britain today.”

Consider the first cohort study: 13,687 mothers of children born between 3-9 March 1946 were initially surveyed by project pioneer James Douglas and questioned in detail about the costs of their pregnancies, the locations of their lavatories (inside or out), other children, income, and general health.

No computers were available to Douglas to analyse this data, just a crude tabulating machine. Yet just about every result that tumbled from his machine showed a country divided by class. Working-class mothers accessed medical care later in pregnancy, were less likely to see a doctor for the birth and were less likely to deliver their baby in a hospital. As a result, their babies were 70% more likely to be born dead than those born to mothers in the most prosperous class.

For the heads of Britain’s fledgling health service, this information was startling and precious. So in 1958, a second cohort – which included Steve Christmas as a new-born subject – was launched to tease even more data. Again its results were striking. By the early 70s, doctors had realised mothers who smoked during pregnancy ran high risks of miscarriage. But were there also long-term effects?

To find out, doctors went back to the 1958 cohort children – now aged 14 – and discovered that, regardless of class, those born to smoking mothers were shorter, performed poorly in intelligence tests, had fewer educational qualifications and tended to be obese. Around 40% of expectant mothers smoked then, a figure that has since been reduced to a still disturbing 12%.

It was also found that breastfed children tend to have better health than those who are not. So the NHS started to employ midwives at hospitals to encourage new mothers to breastfeed – but later removed the service at weekends because of government funding cutbacks. Statisticians were quick to pounce. Using 2001 cohort data, they compared those who had been born on weekdays with those born at weekends and found that the latter – whose mothers had had no midwife support and were less likely to breastfeed – not only had worse health but poorer cognition, thus exposing “the massive, unexpected repercussions of money-saving in healthcare”, says Pearson.

Thanks to cohort studies, we also know now that reading regularly to children has real educational benefits; that babies are safest sleeping on their backs; and that air pollution, even at trace levels, has lethal consequences. These findings, and many more, may sound familiar but it took the cohort study to prove them. They have played a vital role in shaming governments into helping to make Britain a healthier place – not just by throwing up immediate, striking results, but by proving a vast data set that can be returned to over the decades.

An example is provided by Douglas’s 1946 cohort of babies. They will celebrate their 70th birthdays next week and data about their past lifestyles is already being studied in order to unpick lifestyle factors that might be involved in their subsequent development of dementia and other ailments of old age.

Cohort studies have saved thousands of people from early deaths by alerting parents to potential harmful practices and alerting doctors to unexpected dangers. It also provided individuals like Steve Christmas with an added interest in his life. He keeps a filing cabinet filled with reports gleaned from the study in which he has participated all his life. Yet it is not clear whether we can rely on them to guide us for much longer. Last year the UK’s latest cohort study had to be abandoned because not enough mothers-to-be would sign up. People’s lives are busier, Pearson notes, while the nation is less encumbered by the sense of duty that permeated the postwar years. Yet health problems – obesity and mental illness, for example – still threaten society. Without new cohort studies we will struggle to cope, as Pearson makes clear in this cogent, persuasive book.

The Life Project: The Extraordinary Story of our Ordinary Lives is published by Allen Lane (£20). Click here to order a copy for £16