Pontefract Hospital's birth centre could eventually be closed for good.
Health bosses have announced a review into the long-term future of the hospital's midwifery-led unit after admitting keeping it open may not be sustainable.
The birth centre was shut for six weeks from the end of September because of a staff shortage and will reopen on November 5 as planned, the Mid Yorkshire NHS Hospitals Trust said.
But the trust says that staff are underworked when the unit at Pontefract is open because it is "not as busy as it should be".
Responding to a suggestion at a health scrutiny committee that the birth centre should remain open if just one woman wanted to have a baby at Pontefract, trust chief executive Martin Barkley said: "I'd have to disagree because that's a completely untenable situation.
"We're dealing with taxpayer's money here and we have to maximise every pound people pay towards the NHS.
"I can't give any long-term assurance over the future of the Pontefract midwifery-led unit.
"The staff are being grossly under-utilised."
Figures show that nearly 6,000 births a year take place at Pinderfields, which is also run by the trust, compared to less than 200 at Pontefract.
Mums-to-be in the district can choose where they give birth, unless they are expected to have difficulties or they suffer complications while in labour. In that scenario, they are taken to Pinderfields because of the extra resources and expertise of doctors there.
Mr Barkley added: "More Pontefract women are choosing to have their babies at Wakefield, for pretty understandable reasons.
"All of these issues are under review at the moment.
"Hand on heart I can't give you that assurance at this moment in time that the midwifery-led unit will remain open long-term."
Scrutiny chair Betty Rhodes described the news was "dispiriting" and suggested that the public would be "concerned".
Any move to close the birth centre will not affect pregnant women who've already chosen to have their babies at Pontefract because of the length of time the review is expected to take.
Coun Charlie Keith said the review would be seen like "an undertaker getting out his tape measure" and claimed that Pinderfields did not have the extra capacity to cope with the closure of the Pontefract unit, though the trust firmly denies this.
Coun Keith said: "If Pinderfields was built to be the district's hospital the lift should have gone another two floors up. It's not big enough.
"We're shutting things down to rob Peter, to pay Paul. I fear Pinderfields won't be able to cope."
The trust has again insisted that the future of the whole of Pontefract Hospital is not under threat, with Mr Barkley saying its urgent treatment centre and surgery theatres were well used.
He added: "Pontefract Hospital is busy, but parts of it are not as busy as we'd like them to be."