Police were called to the building site off Portland Crescent shortly after 4.30pm this afternoon.
Inspector Carlton Young said: "By the time we got the call they were already at the top. They came down of their own accord just before 5pm."
He said the pair had been immediately arrested on suspicion of causing a public nuisance.
"There was a big emergency response," he said. "A number of fire engines, ambulances and officers. Luckily there were no injuries."
The site on the junction of Portland Crescent and Woodhouse Lane was due to be the Hilton Arena hotel.