The number of gay men being diagnosed with HIV has reached a record high in the UK, according to the Health Protection Agency (HPA).

One in 20 gay men and men who occasionally have sex with men are HIV positive in the UK – and in London, the ratio is as high as one in 12. Part of the reason for the observed rise is increased testing, but, says the HPA, it is also clear that too many men are unaware that they have HIV and are unknowingly infecting others.

"About 8,000 to 10,000 gay men are HIV positive and unaware of their status," said Dr Valerie Delpech, the HPA's head of HIV surveillance.

According to the HPA's annual report, released on Thursday in advance of World Aids Day on Saturday, 3,010 men who have sex with men were newly diagnosed with HIV in 2011 – 47.9% of all new diagnoses in the UK. The numbers have remained high since 2007.

Men who have tested positive and been put on drug treatment, which can keep them healthy and give them a normal lifespan, are unlikely to be infectious. Trials have shown that treatment has a role to play in preventing the spread of the epidemic – the drugs reduce the levels of virus in somebody with HIV to such a low level that they are unlikely to transmit the infection to a sexual partner.

The HPA and organisations for those diagnosed with HIV are all advocating regular testing for anybody at risk. Gay men and other men who have sex with men should take an annual test, they say – and if they have new or casual partners, they should be tested every three months.

Living a long and healthy life with HIV depends on starting treatment early. "People are still starting late: they are infected for three to five years before they are diagnosed," said Delpech. People who are diagnosed as HIV positive late are at 10 times the risk of dying within a year of discovering they have the infection. They are also more likely to infect people while they remain unaware of their status.

The black African community in Britain also faces a higher risk than average, with 37 out of every 1,000 living with HIV last year. Far more men and women in the black African community are diagnosed late than gay men – 68% and 61% respectively, compared with 35%.

The HPA is recommending safe-sex programmes promoting condom use and annual HIV testing as a priority for this community as well as for men who have sex with men. They want NHS clinicians to take every opportunity to offer testing to those at higher risk.

The total number of people in the UK living with HIV climbs steadily every year because treatment is keeping more people alive. Including both diagnosed and undiagnosed cases, it has now reached 96,000, with a total of 6,280 new diagnoses in 2011.

Nearly half of all new diagnoses were acquired heterosexually. More than half of all new UK infections were acquired while the subject was in Britain, compared with 27% in 2002: the small drop in new infections last year, from 6,400 in 2010 to 6,280 last year, was because of the drop in the number of people who had been infected abroad.

Deborah Jack, chief executive of the National Aids Trust, said: "What is striking about the HPA's data is how it really shows both our successes and our shortcomings in tackling HIV in the UK. On the one hand, we can hail treatment as a real success story. Treatment is effective, people diagnosed with HIV can access it easily and it is working in keeping the virus under control.

"However, when it comes to increasing the uptake of testing – the gateway to treatment – our services are patchy, inconsistent and ultimately we are still failing to make any significant headway in tackling the high rates of undiagnosed HIV.

"A quarter of people living with HIV are unaware they have the virus. As long as this figure remains high, new infections will continue to occur. We must increase our efforts in encouraging people to test and making sure that the health service is taking advantage of every single avenue in offering an opportunity to test – something that isn't happening at the moment."

Sir Nick Partridge, the chief executive of the HIV/Aids charity the Terrence Higgins Trust, spoke up for safe sex and said that testing could add 40 years to a person's life. "HIV is an entirely preventable condition, yet each year we see thousands more people across the UK receive this life-changing diagnosis," he said. "While there is still no cure and no vaccine, that doesn't mean we need to accept its continuing march."