Specialist officers forced entry to two homes in Burngreave, one in Meersbrook and another in Stocksbridge in a series of dawn raids.
A fifth was carried out in Chesterfield.
Three men, aged 22, 36 and 41, were arrested in Sheffield and a 31-year-old man was arrested in Chesterfield as part of a pre-planned counter terrorism operation.
All four men were arrested on suspicion of being concerned in the commission, preparation or instigation of acts of terrorism under section 41 of the Terrorism Act 2000.
They have been taken to a police station in West Yorkshire for questioning.
A spokesman for the North East Counter Terrorism Policing Unit said the raids were 'intelligence led' and 'pre-planned'.
The properties raided this morning remain under police guard while searches are carried out.