A 30-year-old man who was at the property in the Manor area of the city was arrested on suspicion of possession with intent to supply class B drugs and possession of a firearm.
He remains in custody and is being questioned by officers from South Yorkshire Police.
The firearm was the second to be seized in the city this week, as police urge people to help them remove the deadly weapons from our streets.
The warrant was executed by officers from Sheffield's Tasking Team, who discovered cannabis plants with an estimated street value of £6,000 along with the weapon and ammunition.
Temporary Chief Inspector Ian Proffitt, from the team, said: "The seizure this morning follows the recovery of an additional firearm from the Upperthorpe area earlier this week by the team following information received.
“Every firearm we recover is one less in circulation on the streets that can fall into the wrong hands and potentially result in fatal consequences.
"We are committed and we will continue to tackle illegal activity and criminal behaviour. I would encourage people to please call 101, or anonymously make contact with Crimestoppers on 0800 555111, if you have any information or concerns in your area, as we will always act on any information received."