I used to have a neighbour I would do almost anything to avoid. An elderly lady, she was given to jam-making, church-going and patrolling the local streets. If I saw her coming, I would dive back inside my front door or seize my son's hand and gallop to our car. I once twisted my ankle in an effort to escape her. Whenever she saw me she would utter the words, "If you don't get your skates on and give that child a brother or sister he's going to be a lonely only." Even now, it's hard to say what was more loathsome. Was it the boisterous intrusion of her tone, the inexcusability of the phrase "lonely only", or the idea of strapping on skates as a euphemism for – what exactly? Unprotected sex?

Many people have never heard the term "secondary infertility"; fewer understand it. I discovered it a year into my struggle to conceive a second child and fell on it, amazed. What I was undergoing had a name! I wrote it down and immediately felt better, as if the phrase exuded a talismanic power that might protect me from the likes of my neighbour.

Secondary infertility (SI) is defined by doctors as the inability to conceive or carry to term a second or subsequent child. You may not have heard of it but you probably soon will, because it's on the increase. A US study revealed that, in 1995, 1.8 million women suffered from secondary infertility; in 2006, it was 3.3 million. SI now accounts for six out of 10 infertility cases.

Fertility expert Zita West has noticed this increase at her London clinic. "The main reason," she explains, "is age. Women are having babies later." Exhaustion also plays a part. "The sleeplessness of life with a small child can't be underestimated," she says. "You might still be breastfeeding, you might be sharing a bed with a toddler, you might be holding down a job at the same time. Basically, there's not a lot of sex happening."

There can be medical causes of SI, says West. "The thyroid is always something we check. Birth can put the thyroid out of kilter." Anwen, a woman in her 40s, tried for five years to have a second child. "I had my daughter when I was 30," she says, "which, at the time, seemed very early. I was the first among all our friends to have a baby." She decided to try for a second when her daughter was three. "But a year went by and nothing happened. I went to the GP and he told me not to worry. 'If you've already given birth, there won't be a problem,' he said. But my daughter turned five and I still wasn't pregnant." Eventually, Anwen persuaded her GP to refer her to a fertility consultant who, after some simple investigations, informed her she had polycystic ovarian syndrome. "He said I'd probably had it all my life and that my daughter was an amazing one-off. I had no idea that anything was wrong." She was given a prescription for the fertility drug Clomid; two months later, she conceived her son.

While women with SI are looked on, quite correctly, by those with primary infertility as lucky, the condition has its own heartaches. It is, I think, a less visible bind. Most people think that if you can have one child you can have another – a misconception which results in the kind of knife-through-the-heart comments of my neighbour.

Infertility is, in all its forms, a most private, hidden anguish. Nobody wants to discuss the finer points of their reproductive system in public. And the desire to have a child can, if thwarted, be so overwhelming that just a glimpse from a car window of someone with a BabyBjorn can be enough to produce such a flood of tears that you have to pull over. How can you possibly articulate that level of pain to the casual inquirer?

People who have suffered primary infertility tell me that the only way they can get by is to avoid everything and anything to do with babies. But for the secondary infertility sufferer, this is not an option. You are confronted on a daily basis at the school gates by pregnant women, people with babies, large families squashed into multiple buggies. School drop-off becomes a terrible tableau of everything you want but cannot have.

I conceived my first child, a son, with no trouble at all. When he was two, we thought we might have another. A year or so later, when nothing had happened, we saw a fertility specialist, who gave us every test there was. We passed each one, as the consultant put it, "with flying colours". Which left us relieved but also confounded. All I had was a new adjective to add to my diagnosis: Unexplained Secondary Infertility.

When I think back to that time, I recall an almost constant sense of grief. Every 28th day represented another failure, another loss. All around me my friends were having their second and then third babies. My son looked at his classmates with their siblings wistfully, he wished with every birthday candle for a brother or sister and one day he asked, "Can you play tig on your own?"

West states that the "hardest thing about secondary fertility issues is that you want a sibling for your child." Fiona, who has a son of five and has been trying to conceive a second child for two years, says she can no longer look out of the window at her son playing in the garden. "It breaks my heart. He just looks so alone out there. All I want is a sibling for him but I don't think it's going to happen."

I found that I couldn't avoid the sense that we were not yet all here, that there was a person missing. In one of those strange confluences, I was, at the same time, writing a novel about a woman who had just given birth. I was spending my days at the fertility clinic and my evenings writing about the strange, shadowy world of early motherhood. My husband, coming into my study and finding me in tears again, laid his hand gently on the manuscript and said, "Do you ever think that writing this book might not be helping?" But you don't choose the books; they choose you. And if I couldn't bring a baby into being in real life, I was damn well going to do it in fiction.

The grief and anxiety of SI is, of course, self-perpetuating. You find yourself in a double-bind: you're constantly told that the chances of conceiving are maximised if you can relax and eliminate stress, but it's hard to let go of something so all-consuming, so elemental, as infertility. People were always saying to me: "If you just forgot about it, you'd get pregnant straight away." For the record, this is the most unhelpful thing you can say to someone with fertility problems. West explains that "couples become more and more anxious about the gap [between children]".

The NHS recommends that, after trying and failing to get pregnant for a year, you should see your doctor; if you are over 35, you should go after six months. Help is out there, if you want it, and takes many forms. West stresses the importance of investigating both the women and the men, "even if they have previously had a healthy sperm analysis because situations and lifestyles can change". There is also the alternative therapy route: acupuncture, hypnotherapy, reflexology, meditation. Or, if all else fails, you could, like me, go for in-vitro fertilisation (IVF).

I was shocked when our consultant first suggested IVF. If there was nothing wrong with us, why did we need something so invasive? But I was also, as I've said, desperate. My baby girl was born last year and sometimes I still can't quite believe she's here.

Secondary infertility is a secret club and one, I've discovered, with permanent membership. I was in a supermarket the other day and ahead of me in the cereal aisle was a woman with a boy of about nine and twin babies in the trolley. As I passed, she turned and looked at us. I saw her clocking my children and their age-gap and she saw I was doing the same with hers. We looked at each other for a moment; she smiled and I smiled back and then we walked on.

• Maggie O'Farrell's new novel, The Hand that First Held Mine, is published by Headline Review.