One of the weirdest experiences for me in early motherhood was a recurrent image or sense, when I was walking down the street, pushing the baby in the pram, that a slight breeze could disintegrate me, dissolve me into fragments or dust.

I imagine some of this was due to the startling (to me) metamorphosis of becoming a mother, psychologically, physically and socially. But looking back, I’m sure it was also to do with loneliness.

It feels shameful to admit, but new motherhood was the loneliest time of my life.

This took me by surprise. While I had a supportive partner and co-parent, family and friends, access to a library and baby groups, and a tendency towards introversion, the isolated arrangement of modern motherhood was a shock. One study found that more than a third of new mothers in the UK spend eight hours a day alone with their babies, and this was often the case for me.

My baby was spectacular, as they all are, but not talking to adults for hours at a time, most days of the week, was a peculiar experience. It gnawed at me. I lost social skills and confidence. I fell silent and, for a while, withdrew. I struggled to ask for, or accept, help. I didn’t know how to talk about the chaos of childbirth, the effect on my body and mind – which I’m sure contributed to periods of depression and anxiety.

I thought there was something wrong with me. Wasn’t this supposed to be the happiest time in my life?

We have known about the dangers of loneliness for health for a while. Its impact is compared to the effect of smoking 15 cigarettes a day. But until recently, little was known about perinatal loneliness. In fact, what I was feeling was not unusual at all.

Some research suggests 80 to 90% of new mothers feel lonely. One study found that 43% of mothers under the age of 30 in the UK felt lonely all the time. Loneliness is more prevalent in new parenthood than in the general population, and it is more common among certain at-risk groups, such as young parents, immigrant and refugee mothers, mothers without supportive partners, mothers looking after a child with significant health problems, and transgender and non-binary parents.

Loneliness, unsurprisingly, can lead to mental illness. In 2023, a review of evidence carried out at University College London found that loneliness was a key risk factor for perinatal depression. The study’s lead author, Dr Katherine Adlington, said: “We found that loneliness was central to the experiences of expectant and new mothers with depression. Chronic loneliness – feeling lonely often or always – is now experienced by around a third of new parents.

This level of loneliness and isolation doesn’t seem at all ideal for a species that relies on social contact for health, wellbeing, survival, and learning. So why are mothers in our society so lonely?

Some existential loneliness in new motherhood might be inevitable. For example, I found there was an individual existential process I needed to go through to make sense of bringing a life into the world through my body. But much of the isolation felt by new parents is a consequence of how some contemporary societies are structured.

Take the immediate postnatal period. In most cultures across the world, including in China, Japan, India and South America, new mothers follow similar practices or rituals after giving birth that acknowledge the emotional heft that becoming a mother brings, as well as what a body goes through in pregnancy and childbirth.

These often involve some combination of nutrient-rich meals, massages, drinks and herbal baths prepared by close female relatives, women in the community, or hired postnatal assistants who will care for the mother and baby for about 40 days. The mother is expected to rest and be looked after, while feeding and bonding with her baby.

In Vietnam, the period is called namo, meaning “lying in a nest”. In Nigeria, omugwo is the name of the Igbo cultural practice of postpartum care given to the new mother and baby by her mother or elder female relatives. In Mexico, a “closing of the bones” ceremony helps to emotionally and physically close the woman’s body after the extreme opening of birth.

In western societies, a new mother is mostly left to her own devices after birth. She will normally give birth in hospital and go home shortly afterwards. If there is a partner, he or she will usually have two weeks’ parental leave from work and then the primary caregiver – most usually a mother – will care for the baby at home for a period of time. A partner might be off work for just one week: in a government consultation published in June 2023, one of the proposed “reforms” was to allow statutory paternity leave to be taken in two separate week-long blocks as opposed to the two weeks consecutively. As anyone who has had a caesarean or a birth injury will know, it often takes more than seven days to even be able to walk properly again.

But in late capitalism, time is money, and people are time-poor. In a 2019 survey of parents, 14% of fathers who did not take any statutory leave reported that they were too busy to take time off work.

A parent at home alone with an infant is a relatively new construct. For most of our evolutionary history, humans lived in small groups. This meant that women mothered alongside others, or foraged together with their babies close by. Our brains and nervous systems evolved in collective child-rearing societies.

This kind of collective raising of children still exists today. A recent study led by Nikhil Chaudhary of the University of Cambridge on the Mbendjele BaYaka hunter-gatherer communities living in rainforests in the northern part of the Republic of the Congo, found that multiple caregiving adults, known as “alloparents” – responded to a crying child more than 40% of the time, as well as providing close care and physical contact.

Childcare support, wrote the researchers, seems to have been substantial in our evolutionary past. “WEIRD (western, educated, industrialised, rich, and democratic) parents” today “are faced with the challenge of child rearing outside the cooperative childcare systems that have been so key in our species’ evolution.”

In the early months of motherhood, I sometimes felt that my nervous system simply hadn’t evolved for the continuous physical, psychological one-on-one care that my baby needed, often without any kind of break. It became a vicious circle. Isolation meant exhaustion, which meant little energy for anyone else, which meant loneliness. Frankly, even with my children now out of the baby stages, it sometimes still does.

“If anybody was looking above our world, they would think this is utter madness, because here are all these mothers in these little boxes,” says Andrea O’Reilly, a professor in the School of Gender, Sexuality and Women’s Studies at York University, Toronto, and the founder of motherhood studies, an academic discipline that researches topics related to motherhood and matricentric feminism (a mother-centred feminism). “How we live in our homes, how we arrange our neighbourhoods, how we organise work is all fed into this absolute privatisation of family.”

In 2021, the BBC Loneliness Experiment found that unsurprisingly, around the world, those who lived in individualistic cultures are more likely to be lonely.

For a long time, I thought this was my fault, my failing. But since writing my book Matrescence, which is about the transition into motherhood and how it affects the mind, brain and body, I am in correspondence with hundreds of women and I’ve learned that many of them feel the same.

There is a distinct relational aspect to the loneliness of new mothers. A pattern I’ve seen in the messages I’ve received from readers over the past year or so since Matrescence was published is a feeling of shame and stigma around the subjective experience of new motherhood, and then a silencing of the self and isolation, often alongside distress, shock and sometimes mental illness.

A 2021 study led by clinical psychologist Dr Billie Lever Taylor at King’s College London illuminates the social aspects of postnatal mental distress. All mothers felt shame and anxiety about being judged “inadequate” but mothers from ethnic minority backgrounds, deprivation, or who were single or young, felt a stronger fear of being judged to be “bad mothers”.

“There was a sense of loneliness in the mothers’ beliefs that they were alone in their feelings, and ought not admit to them, preventing honest, authentic interactions with others,” Taylor and her team wrote. New mothers who felt distressed would actively withdraw through fear of “being a burden”.

The UK’s individualistic culture can come as a shock for people from more collectivist cultures. In Taylor’s study, a mother who identified as Black African and Muslim compared the social support for mothers in her African home country with the lack of it in England. “If it’s back home now, you know, your parents, you give them your children sometimes. Two days, three days, they would be with your parents. You have a little rest. But here there’s nobody,” the woman said.

Other researchers have found the causes of perinatal loneliness include a lack of recognition of the difficulties of being a mother, the burden of childcare, an absence of community and social networks, mismatch between expectations and reality, and stigma attached to breastfeeding difficulties.

A sense of failure among new parents came up time and time again in a 2024 study by Dr Ruth Naughton-Doe, a research fellow at the University of York specialising in loneliness. “But they were not failures,” she says. “They had been failed. Failed by a society that doesn’t value parenting. Failed by under-resourced services.”

Naughton-Doe found that structural drivers of loneliness were varied, and included poverty and inequality, inadequate and unaffordable public transport, poor infrastructure (including unsafe pavements for prams and too few places to feed or change a baby), limited parental leave and low maternity and paternity pay.

Austerity measures have stripped communities of public areas that are safe and welcoming to young children and care-givers. One in three children under nine in Britain don’t live near a playground. Hundreds of libraries have closed. Sure Start children’s centres have had budgets slashed by 60% since 2010 with many closing down. Austerity has also eroded social support: since 2015 in England, health visitor numbers have reduced by 37%.

A common theme in Naughton-Doe’s research is that professional women are a high-risk group for perinatal loneliness. “They are used to being busy, being valued and it’s a huge shock when they take on a role which isn’t valued, you have to learn on the job, it’s messy, and they’re thrown into this traditional role.”

A compounding factor in loneliness today is the intensity of contemporary mothering norms in a child-hostile society, which are, in some ways, more punishing than they were for previous generations. For my grandmothers’ generation, children were let out of the back door and told to come back at dinner time. For my generation, this would be unthinkable.

Academics such as O’Reilly chart the development of today’s “intensive mothering” ideology to the late 1980s. It is defined by sociologists as a number of beliefs. First, that mothering is natural and instinctual to women and that the mother should be the primary caregiver. Children need copious time, energy and material resources. Mothers must be attuned to their children’s cognitive and emotional needs. The mother must be satisfied, happy and calm.

Unsurprisingly, “intensive mothering”, which is now studied and practised across the industrialised world, is associated with poor maternal mental health, stress and burnout.

O’Reilly raised her children in the 80s and 90s in Ontario, Canada where baby and toddler groups were held mainly for mothers to have social interactions and a cup of coffee while their children played. At some point around the turn of the century, an “obsession with child development” eclipsed the mother and the nature of these gatherings changed, she says.

The problem is not a focus on the needs and requirements of babies and young children. It is having to strive for this ideal within societal conditions that make meeting it impossible. John Bowlby, the architect of attachment theory and its role in child development, emphasised that this was “no job for a single person” and that a caregiver needed “a great deal of assistance”.

And yet, mothers today spend twice as much time looking after their children every day compared with in the 1950s, while also working more.

So what could help? Certainly, grassroots movements to support and connect new parents are growing. Naughton-Doe’s research found an emergence of creative health interventions for new mothers including music, singing, yoga, art and nature-based groups.

From groups offering peer-led creative workshops for mothers, including parents of children with special educational needs and disabilities (Mothers Uncovered in Brighton) to the Motherhood Group which supports the Black maternal experience; forest school groups for LGBTQ+ families (Muddy Puddle Club) to wellbeing circles (Stroud Motherhood Collective); writing groups (New Mothers’ Writing Circle) and zines (Motherlore); and walking groups (Blaze Trails) to the first Matrescence festival, held in Exeter in June, numerous different types of gatherings and organisations are emerging around the UK.

We also need structural change, investment in services, fairer economic policies and parental leave. Naughton-Doe’s research suggests that more parental leave would make a huge difference to maternal loneliness. And solo parents, she says, should be able to nominate someone else to take leave when they go into labour (if they have other children, and no one to look after them, the children go into temporary care by social services).

The words matrescence and patrescence (which mean the transition to motherhood and fatherhood respectively) are useful, because they can make visible the health, wellbeing and vulnerabilities of those experiencing pregnancy, birth and new parenthood in a culture that disavows so much of this developmental stage.

When I realised I was going through something significant – that it is kind of a big deal to become a mother – it eroded some shame and confusion, and enabled me to talk to others, to leave the house more, to learn about the conditions of modern motherhood, and to feel less alone.

• Matrescence: On the Metamorphosis of Pregnancy, Childbirth and Motherhood by Lucy Jones is out now (Penguin, £10.99). To support the Guardian and the Observer, order your copy at guardianbookshop.com. Delivery charges may apply

• This article was amended on 21 August 2024 to clarify that Mothers Uncovered offers workshops for mothers, not only for those with children who have special educational needs and disabilities.