The town council has confirmed it seized the permits from the two men after they were detained by South Yorkshire Police.
Both cabbies are being held on suspicion of sexual offences against children between 1994 and 2003.
Karl Battersby, director of development services at Rotherham Council, said: “Public safety is our prime concern and we have to react to information that we receive.
“So far this year we have taken action to prevent six people from continuing to operate as taxi drivers.
“We would ask that the public support us in this, and that they pass on any concerns about the taxi trade to either ourselves by ringing 382121 or South Yorkshire Police on either 101 for non emergencies of 999 in an emergency situation.
“Following consultation with both the public and the licensed trade we will shortly be introducing robust new standards for licensing drivers, vehicles and operators.
“The council is determined to make sure that the public can have complete confidence in the taxi service in Rotherham.”
A spokesman said the men will have right to appeal the council’s decision by applying to Rotherham Magistrates Court.