Emergency services were called at 6.15pm on Wednesday after people in Slaithwaite - a village in the Colne Valley - raised concerns that fresh terrain could be seen falling from the exposed cliff side underneath the new build homes.
One fire fighter from Slaithwaite fire station said crew could "hear the earth moving" as they attended the scene at Manchester Road, backing onto the River Colne.
Surveyors from Kirklees Council are expected to attend the scene today to evaluate the risk.
Watch Commander Matthew Green from West Yorkshire Fire & Rescue Service said: "We attended and assessed the situation last night after local residents called 999. We did the best we could considering it was dark.
"We could actually hear the land falling and, as you can see from the picture, it looks like it could be potentially perilous."
Mr Green said the terrain underneath the homes had partially collapsed due to a minor landslip around 18 months ago when a tree came down, leading to the exposed earth seen in the picture.
The house on the end is currently unoccupied, although the two houses prior were evacuated overnight as a precaution.
"We advised them, if there was somewhere they could stay overnight, to do so, which they did," Mr Green added.
"The next three properties were more comfortable to stay and keep monitoring the situation.
"We didn't notice any particular damage to the house or the structure but we could hear the land moving as we turned up. The council are sending a surveyor today to give a more qualified opinion on the situation."
It comes as flood alerts remain in place across parts of Yorkshire this week due to Storm Christoph.
Major incidents have been declared while some roads have also been closed. Pictures have emerged showing flooding at Tadcaster while flood gates have been closed in York.
Kirklees Council and West Yorkshire Fire & Rescue Service have been contacted for further comment on the situation in Slaithwaite.
Support The Yorkshire Post and become a subscriber today. Your subscription will help us to continue to bring quality news to the people of Yorkshire. In return, you'll see fewer ads on site, get free access to our app and receive exclusive members-only offers. Click here to subscribe.