The council refused plans to demolish Bennett Cottage on Mayfield Road, in Mayfield Valley, and build a bigger and more modern house in its place at a meeting last year.
Councillors on the planning and highways committee narrowly voted to block the development with six votes against and five in favour with two abstentions.
In the meeting, councillor Andrew Sangar, who voted against, said: “I believe that Mayfield Valley is one of the jewels in Sheffield’s crown – it’s used by lots of people from across Sheffield for walking, cycling and running.
“I believe this development is out of keeping with the special character of the area and I believe it’s detrimental to the openness of the green belt and the settings of the listed buildings.
“The loss of the traditional buildings would be significant and would harm the character of the Mayfield Valley and would harm the special architectural and historic interest of the setting.”
Ahead of the meeting around 90 people wrote to the council to object and around 35 wrote in support.
The developer appealed against the decision and this week, council officers confirmed that it was unsuccessful.
They said: “In summary the Inspector concluded that there were no considerations to clearly outweigh the harm by reason of inappropriateness, the harm to the openness of the Green Belt, and the identified heritage and landscape harm and that very special circumstances have not been demonstrated to justify the appeal proposal.
“It was considered that the adverse impacts arising from the appeal proposal would significantly and demonstrably outweigh the benefits of the scheme when assessed against the National Planning Policy] Framework taken as a whole, including the contribution to the supply of housing, and that both the national and local policies provide clear reasons for refusing the development.”