Just days after Housing Minister Caroline Flint, MP for the Don Valley constituency, admitted the number of new homes built this year is almost certain to fall, the Government is now referring to the three million as an "ambition" rather than a firm commitment.

The use of the phrase – in contrast to previous talk of a "programme" or "plans" – in this week's draft Queen's Speech comes amid growing doubts about the chances of meeting the goal.

It has led to accusations Ministers are preparing to admit defeat, although last night the Government insisted the goal is still "challenging but very achievable".

Shadow housing minister Grant Shapps said: "They're clearly changing the language now – they've been saying up until now it's a target they're definitely going to meet and we've been saying it's unachievable.

"Perhaps they're getting ready to concede they're not going to meet that target."

Liberal Democrat housing spokesman Lembit Opik said: "It's been downgraded. The language is clear, it's not a target any more."

The pledge to build three million homes is one of Gordon Brown's flagship policies, aimed at tackling a housing shortage and offering hope to first-time buyers struggling to get on the property ladder.

At Labour's summer conference last year, with the new Prime Minister riding high in the polls, Mr Brown was clear in pledging "we will increase housebuilding to 240,000 a year", the annual figure needed to meet the target.

He told the employers' organisation the Confederation of British Industry later last year: "We will stick to our plans to build three million more homes by 2020, making housing more affordable and providing homes for the workforce of the future."

And at Labour's spring conference this year, he referred to "our programme for three million more homes".

But the draft legislative programme, announced by Mr Brown this week, refers only to the Government's "ambition for three million new homes by 2020".

The Tories have always maintained the goal will be missed, because of delays caused by planning appeals and judicial reviews but with the credit crunch biting, the chances of hitting the target appear ever more remote.

Last night a spokesman for the Department for Communities and Local Government dismissed claims the commitment had been watered down.

"There's been absolutely no change in our position," he said. "We've said both ambitions and targets before.

"It would be wrong to say we've watered down our plans. It's a challenging but very achievable target."