Government plans to reform mental health law, including the introduction of compulsory treatment in the community and preventive detention of people with dangerous personality disorders, have been thrown into turmoil by an official advisory group's warning that the measures are unworkable without extra resources.

A bill had been expected to be unveiled about now. But the advisory group's report to ministers, a copy of which has been seen by the Guardian, is so critical that mental health groups believe the government has no option but to shelve the legislation.

The development puts further pressure on the health secretary, Patricia Hewitt, already in difficulties over NHS restructuring and smoking curbs. Extra parliamentary time needed to drive through the latter may be cited as a reason to quietly defer the mental health bill.

The Department of Health denies that the bill has been stalled and says it will be introduced "when parliamentary time allows". But Andrew McCulloch, the chief executive of the Mental Health Foundation charity and a former senior civil servant in the department, said yesterday: "You can't brush aside a report like this. They need a fundamental rethink."

Earlier this year, a scrutiny committee of MPs and peers backed criticisms that the draft bill erred too far on the side of enforcement at the expense of civil liberties. In July, ministers accepted some of the committee's points but declared that the central measures would go ahead.

The government set up an advisory group to recommend a workable model for a new system capable of processing an anticipated 40,000 mental health tribunal hearings a year - double the present number. Under the bill, tribunals would cover compulsory community treatment and care plans and would be initiated automatically after 28 days. Ministers told the committee: "The bill will not be introduced until this review of the tribunal model and the assessment of the workforce have been completed."

The advisory group, chaired by a senior health department official, has now told ministers that the system could be made to work only by "a massive injection of additional resources" and that there is a real risk that it may "only work after a long delay and at disproportionate expense". The report concludes: "[We] cannot predict with any confidence that the proposed new mental health tribunal is viable."

Sophie Corlett of the mental health charity Mind said: "We have known from the start that it would be unworkable, because the current system is unworkable. The whole thing is complete chaos."

To try to cut projected costs, the health department is considering one-person tribunals or decisions being made without a hearing. But Mr McCulloch said: "You are talking then about banging somebody up in a mental hospital on the basis of some guy looking through the paperwork. It would be a grotesque step backwards."