Officers rushed to Hartshead Square at around 2pm this afternoon after a member of the public saw a man carrying a suspected firearm.
Armed police responded immediately and arrested two men in connection with the incident.
One of the men was found to have been carrying a BB gun.
A South Yorkshire Police spokesperson said: "We will always take reports of the nature extremely seriously and deploy officers, both armed and unarmed as soon as possible, to ensure that the public are kept safe"