Emergency services were called to the Cliffe block in Stannington Road, Stannington, at 7.15am following the discovery.
Officers sealed off the15-storey block with tape and stood guard while enquiries were carried out.
A crime scene investigation team was also alerted to the discovery.
An investigation into the circumstances surrounding the death is underway but it is not being treated as suspicious.
A South Yorkshire Police spokeswoman said: "Emergency services were called at around 7.15am this morning following reports a man had died in the Stannington area of Sheffield.
"An investigation is underway and in the early stages, but at this time there are not thought to be any suspicious circumstances."