Tits. The name was given to me by a bully shortly after I started Year 6. I had been a fat kid since the age of nine, but as puberty began to kick in, parts of me started growing differently than expected. The doctors said I had gynecomastia. "Man boobs" or "moobs" in the jeering parlance of our popular culture.

But my bully simply called them "tits". And so this became my name in the school hallways.

I was Tits.

He would pass me in the corridor and catcall, "Hey, Tits!" and his friends would laugh. Sometimes, if he was feeling bold, he might actually grab one of my breasts and squeeze it in front of the other kids. Not everyone laughed. But many did.

As direct as this bullying was, growing up with gynecomastia was characterised by other, smaller insults, too. Most kids would just say, "Why don't you wear a bra?" Even adults could be cruel. "Are you a boy or a girl?" I was often asked.

When wearing shirts, it was crucial that they be loose fitting. If a T-shirt had shrunk in the dryer, I would spend hours stretching it out, so it didn't cling to my body. You can see fat boys do this every day. Pulling at their shirts to hide the shape of their bodies, and particularly their breasts.

As a fat kid, and one who hated competition, I learned to loathe sports and, especially, physical education. The one form of exercise I enjoyed was swimming. Unfortunately, as my breasts grew, so did my shame about removing my shirt. I knew that taking it off would bring ridicule. So I pretended that I was above swimming – that I was too cool for the pool.

By my teenage years, I had developed remarkable powers of verbal self-defence. I absorbed cruelty and learned how to mete it back out in sharp doses. There's no doubt that this shaped the person I became, for better and for worse. At school, I managed to carve out a social niche for myself. The bullying stopped. But the shirts stayed loose fitting. I rarely went swimming.

The doctors thought that perhaps I suffered from low testosterone. I found this funny, since my sex drive had been in high gear since the age of 14. I assured them that this was not the case. Finally, they suggested that my excess breast tissue was probably just a result of being fat. Lose the weight and the breasts will go away.

So I lost weight. By 17, I was slender. Girls were starting to talk to me. I was more confident. And I still had breasts. In some ways, my slimmer body only accentuated the contours of my chest. So I continued to wear baggy shirts and the idea of being topless in front of a woman or acting on that newfound female attention seemed remote. The doctors noticed, too. After graduation, they congratulated me on my thin body. Now it was time, they said, to get rid of my breasts.

Though I had always been squeamish around doctors, there was little question about whether to have the surgery. They said it would take only a few months to heal and that the only side-effect would be a permanent loss in nipple sensitivity. Could a decade of bullying end with a simple outpatient procedure? We quietly scheduled a date, sharing the decision only with close members of the family.

In the first surgery, I was placed under general anaesthesia. The doctor made a half-moon incision under each nipple and cut out the excess breast tissue, finishing the job with some liposuction. Unfortunately, the surgery wasn't a complete success. My breasts were smaller, but lumpy, and my nipples were puckered. This was disappointing. My chest looked worse than it had before I'd gone under the knife. It took a second surgery to make everything look "normal".

I was 19. On New Year's Eve, I went to a party and got drunk for the first time in my life. There, I met a girl who took my virginity. She didn't insist on taking my shirt off. This was a relief, because under my shirt was a sports bra, and under that layers of gauze. My chest was still healing from the second surgery. In many senses of the word, I was still becoming a man.

I'm reminded of my experiences whenever one of those "humorous" stories pops up on websites such as the Huffington Post and Mail Online. Perhaps you saw the photo making the rounds late last year, of New York Democrat Barney Frank's "moobs". The photo spread throughout the web and inspired mocking headlines, even on politically liberal websites.

This fixation on "man boobs" reveals our culture's obsession with binary gender, but we have all the evidence we need that biological sex and gender are not as rigid or fixed as we imagine. There are intersexed people. There are transgender people and genderqueer people. There are millions of men and boys like me with gynecomastia, a medically harmless (though socially lethal) condition. The prevalence of gynecomastia in adolescent boys is estimated to be as low as 4% and as high as 69%. As one article notes, "These differences probably result from variations in what is perceived to be normal." You think?

We're so entrenched, we can't accept bodies that don't fall on either extreme of the gender continuum. Transgender men and women encounter these attitudes in direct and sometimes life-threatening ways. And, given the misogyny that pervades society, these pressures are even harder for women and girls, whether they're cisgender or transgender. Their bodies are hated and desired in equal measure. When my bully grabbed my breasts and called me "Tits", he was taking what he wanted. He was also reminding me that I was no better than a girl. I was beneath him.

With the explosion of social media and the surveillance society, body policing has become far more intense. We live in an age of crowdsourced bullying. I cannot imagine what it would be like to grow up as a boy with breasts in 2012. I suppose I'd spend hours in Photoshop, digitally sculpting my body to remove fat from my face, belly and chest before uploading my profile photos. I would probably become vigilant about removing tags from unflattering photos and obsess over remarks people made about me on comment threads.

Perhaps because of my early struggles to accept my body, I've found a measure of freedom in appearing naked on stage as a performance artist. And now, 20 years after my surgeries, I find I miss my breasts. Looking through childhood photos, I was astonished to find a picture of myself at 12, dressed for Halloween in full drag as Dolly Parton. In the photo (left), I have a big smile and my boobs have been pushed up and exaggerated. The photo touches me, because it suggests that even while I was facing intense bullying and social stigma, I was already using my body to comment on gender with humour and strength.

As a fat man, I still have breasts. The ones I have now are smaller, but still capable of riling the body police. I once scandalised a fancy pool party simply by taking off my shirt. I realise that, as a man, it is my privilege to do so. In most parts of our society, it is either illegal or strongly frowned upon for a woman to go topless. (Female breasts are either for maternity or for male sexual pleasure, not for baring at polite parties.) Perhaps my breasts, which remind people of this prohibition, invite a similar kind of censure.

Now, I don't just use my sharp tongue for self-defence. I also use my body itself, as an argument and as a provocation.

I am Tits. Got a problem with that?