The item was found at Elford Place West during a routine search of the address in connection with drugs offences.
A cordon has now been put in place and a specialist army team has been called to investigate.
West Yorkshire Police said:
"At 8.30am today, officers executed a drugs warrant at an address in Elford Place West, Harehills.
"Two men were arrested at the address on suspicion of drugs offences. During a search of the property officers found a potentially suspicious item.
"A cordon has been put in place on the street and a number of properties in the area have been evacuated. Officers are liaising with army specialists who are at the scene to assess the item.
"The council’s emergency planning team have been contacted and are putting appropriate measures in place to support the evacuated residents."