McCarthy & Stone want to build 51 apartments on the site of a derelict nursing home on Langholm Close, down a quiet cul de sac, accessed by a private road.
St Mary’s councillor Linda Johnson told the eastern area planning sub committee on Monday no one disagreed that the site needed redeveloping, but its "sheer size" was the problem.
She expressed concern about how it would affect views approaching the town from Beverley Westwood and said newer housing had been sympathetically designed “to make you feel as if it was always meant to be there”.
However the proposed scheme “will be in your face, jump out at you, dominate the skyline”.
Coun Denis Healy suggested the size of the new build would be around treble the footprint of the existing building and the number of windows meant “practically every property in the locality will be affected in some way” and there would be “huge overlooking”.
He added: "It's almost monolithic in its appearance."
Coun Barbara Jefferson said she feared the views from the west over the Racecourse and the Hurn would be "absolutely changed", adding: "To me it is complete overdevelopment".
The agent for the developer said they'd worked with council officers to address various issues, including deleting dormer windows on more sensitive elevations.
And he said the scheme would result in significant benefits, including addressing the identified need for older people's housing.
The average age of those taking up apartments was 83 and such schemes gave people a "new lease of life".
The old building, he added, had been subject to many acts of vandalism and anti-social behaviour.