The government has confirmed that there are now 124 fewer Sure Start centres for children than there were when the coalition formed last year.

Lord Hill of Oareford, the education minister, released figures that showed there were 3,631 Sure Start centres in April last year, but by 8 September this year there were only 3,507.

Labour said the figures raised questions about David Cameron's claim that he was committed to an initiative of the last government that is credited with helping to reduce child poverty.

Stephen Twigg, the shadow education secretary, said: "The truth behind David Cameron's claim that he is committed to Sure Start has been exposed by his government's own figures. With 124 centres closed already since he became prime minister, families will be worried that even more will close in future.

"Sure Start is one of Labour's proudest achievements, but this out of touch Tory-led government is putting it at risk."

Hill said that just six Sure Start centres had closed and that most of the net reduction was caused by streamlining services. The minister told peers: "Of the reduction of 124 children's centres, six are outright closures; the remainder are accounted for by local re-organisations such as the merger of two or more centres.

"The government attaches high importance to the role that Sure Start children's centres play, which is why through the early intervention grant we have put in the funding to maintain a national network of Sure Start children's centres."

Lady Hughes, Labour's education spokeswoman in the lords, said: "For the first time ever we have the prospect, through Sure Start, of a universal integrated service for the under-fives and their parents. It is clear, however, that local authorities are not only closing centres but are cutting their budgets dramatically – 11% this year, 21% next year – in response to the government's significant cuts in the early intervention grant and the removal of the ringfence."

Cameron said on the eve of last year's general election that he backed Sure Start centres and attacked Gordon Brown for claiming that the Tories were not committed to the initiative.

"Yes, we back Sure Start," he said in a question and answer with readers of the Independent. "It's a disgrace that Gordon Brown has been trying to frighten people about this. He's the prime minister of this country but he's been scaring people about something that really matters. Not only do we back Sure Start, but we will improve it, because at the moment the people who need Sure Start the most – disadvantaged families – are not getting enough of the benefit."

Naomi Eisenstadt, the first director of Sure Start when it launched in 1998, blamed the closures on the removal of the last government's requirement for local authorities to spend a defined amount on the centres. "Sure Start is being cut, anyone who says otherwise is wrong, but everything is being cut and the real issue is the removal of the ringfence," she said.