I had always been unusually small for my age, and people often thought I was three or four years younger. Although I initially liked being “different”, towards the end of primary school I began to tire of it and underwent tests to find out if there was a reason for my short stature. They didn’t reveal the real cause.

As my early secondary years rolled by and I watched my friends turn into women one by one, I remained defiantly prepubescent. I was at boarding school, which didn’t make things easier – frequently teased, I became increasingly self-conscious. In my GCSE year, having still not had a growth spurt or entered puberty, I had further tests, and at last I was given a diagnosis. As a result of a rare and little-known condition called Swyer syndrome, I had been born with male chromosomes.

As a result, my body had been unable to produce oestrogen, without which puberty can’t take place in girls. Although I had a uterus and fallopian tubes, my ovaries hadn’t developed properly and could not produce eggs. There was a big risk of them becoming cancerous, so they had to be removed – this was in the days before keyhole surgery, so it was a major procedure – and I would require years of treatment to prompt the changes that should have occurred naturally.

I was distraught and found it very hard to talk about, to either my peers or my family. But learning that I was infertile and would never be able to have children naturally didn’t have the impact then that it would later. At 15, I was more concerned with fitting in at school. Still only 4ft 8in, I had to inject myself with growth hormone every day for two years, and for the first year of doing that I was unable to take the hormone replacement drugs that would kickstart puberty.

When it finally happened I felt a huge relief. But my adolescent friends didn’t know how to deal with my situation. Their reactions were a mixture of awkwardness, sympathy and clumsy teasing, and I remained guarded. After years of bullying, I found it impossible to reinvent the way I perceived myself, even as I changed physically.

That opportunity came when I started university. I had never had a boyfriend before, but I started a relationship during freshers’ week. Then in my second year I replaced my entire wardrobe, having finally found the confidence to buy more feminine and even revealing clothes. Throughout my teenage years, my priority had been remaining invisible; it had never occurred to me that I could be sexually attractive.

I was in my early 20s before I slept with a man. When he later broke up with me, he told me it was because I couldn’t have children – but that was simply never an option. I don’t think it’s entirely unconnected that I’m still single at 40. I tend to find myself with people who aren’t really interested in wholly committing.

Virtually all my friends have children, which has put a strain on some relationships, and I was hit particularly hard when my younger sister had a baby. My mum didn’t contact me for a fortnight around the birth – she simply didn’t know how to handle it. I ended up confiding in my dad, who meant well but was very clumsy and started listing things I can do that my sister can’t, such as playing the piano. I asked how he would have reacted if he’d had to have his testicles removed as a teenager and somebody had said, “But don’t worry, because you’re very good at technical drawing.” That led to us not speaking for more than a year.

A few years ago, I was diagnosed with osteoporosis, caused by the lack of oestrogen in my teens. It was another blow, but the treatment effectively led to a second puberty and I became more curvaceous. I now feel much more confident about myself physically – at 40, I’m lucky to look much younger than my age.

Swyer syndrome is a very rare condition, affecting just one in 30,000. I’m yet to meet anyone else who has it and I’m still coming to terms with it. Spending years being made to feel a freak has never left me, but it’s no longer always at the front of my mind.

• As told to Chris Broughton

Do you have an experience to share? Email experience@theguardian.com