Steve Silberman hasn’t slept: a combination of jetlag, excitement and genuine surprise at winning the Samuel Johnson prize for non-fiction. These morning-after interviews with happy, sleepy authors can be rather ritualised, but Silberman is a delight: a bulky (think John Goodman) American whose black braces just about keep his baggy trousers up, and who talks non-stop about his book – a history of changing perceptions of autism over the past 80 years – and the curious art-meets-science life that produced it.

Neurotribes is the first popular science book to win the prize – an important breakthrough, he says, not least because “science is under attack in America, particularly from Republican party presidential candidates. Ben Carson, who is himself a doctor, touts his disbelief in evolution.” Silberman reckons the book won because it combines “history, science and real-time interactions with people”, and that the judges responded to its optimistic conclusion that people with autism, marginalised for so long despite having much to offer, were being embraced at last.

“There’s a tremendous amount of human suffering in the book,” he says, “but it also offers the possibility of redemption. It’s an important story that had never been told. That’s had a terrible effect on global public health. Since parents don’t know why the number of autism diagnoses went up so startlingly in the 1990s, they are afraid of any number of things.”

The “anti-vaccine hysteria” this led to reflected the fact that people were unaware that autism had always been around – it’s just that it was only now being properly defined. “Autistic people have always made contributions to society, though often not acknowledged. Many forms of digital life we now take for granted were developed by people with autistic traits. Twenty years ago, you would call someone who communicated with their friends only through a keyboard disabled. Now they’re just a teenager. I wanted to look at autism as a human community, instead of a condition or a disorder.”

Neurotribes began life in 2001 as an article for Wired magazine headlined “The geek syndrome”. Examining what seemed to be an epidemic of autism in Silicon Valley, it caused a stir and brought offers from agents suggesting he develop it into a book. “I felt it was too early. I didn’t have much more to say. I’d had a good career at Wired and won awards, but I felt it was late to take such a big chance.”

He sat on the story for most of the following decade, still getting feedback from the original article and collecting material. Then in 2009 – with the encouragement of neurologist Oliver Sacks – he made the leap. He thought it would take 18 months. It took him five years. “It’s the most personal thing I’ve ever written,” he says. There’s a lot of science in the book – but also a lot of his soul.

Silberman was born in New York, the son of two teachers who were communists and anti-war activists. “I was raised to be sensitive to the plight of the oppressed. One of the things I do is frame autism not purely in a clinical or self-help context, but in a social justice context. I came to it thinking I was going to study a disorder. But what I ended up finding was a civil-rights movement being born.”

He says the fact he is gay also conditioned his approach. “My very being was defined as a form of mental illness in the diagnostic manual of disorders until 1974. I am not equating homosexuality and autism – autism is inherently disabling in ways that homosexuality is not – but I think that’s why I was sensitive to the feelings of a group of people who were systematically bullied, tortured and thrown into asylums.”

Silberman studied psychology at Oberlin, a liberal arts college in Ohio; did a course at the Buddhist-inspired Naropa University in Colorado, where he worked closely with the poet Allen Ginsberg; and did a masters in English literature at Berkeley, California, where his supervisor was another poet, Thom Gunn. In 1979, he moved to San Francisco to live what he calls “a gay life without fear”. He wrote cultural pieces for the San Francisco Chronicle and co-authored a book on the Grateful Dead, but gradually gravitated towards writing about science, a lifelong interest, first for the nascent internet and then for Wired.

In 1999, he discovered the work of Oliver Sacks. “It blew my mind. It was so humane, so precisely observed and so aware of the tiniest resonances in language. It was exploring possibilities of human awareness as Ginsberg had done, but because the people he was studying had rare conditions, it was extending the possibility of consciousness.”

Three years later, Silberman wrote a profile of Sacks for Wired, and through that got to know him well. “I figured out in the course of writing the profile that he was a closeted gay man. He was traumatised by a remark his mother had made when he came out to her – ‘You are an abomination, I wish you had never been born’ – and he hadn’t had a love affair in 30 years. So I became his gay coach, talking about his work and his feelings about being gay.”

Sacks, in turn, encouraged Silberman to write a book, and one of the last things the neurologist wrote before his death in August was a foreword praising what he called “a sweeping and penetrating history” that “will change how you think about autism”.

Silberman, now 57, says the advance for the book was six figures, but he only got a quarter of it upfront. He had given up his journalism job and the money soon ran out. “We were broke, broke, broke,” he says, “and broke in San Francisco is not fun. My schoolteacher husband supported us, while I would literally sell used records at the record store.”

It was worth it, though, he says. “It used to be that all discussion of autism was, in a sense, behind the backs of autistic adults, as if they were children, as if these adults were invisible. In fact, they’re all around us.”

• Neurotribes is published in the UK by Allen & Unwin, £16.99. Buy it for £12.60 with free UK p&p at bookshop.theguardian.com or call 0330 333 6846