They moved onto the Castlefield car park around 18 days ago.
People living close by lodged a number of complaints with the police and the council claiming homes and cars had been damaged and human waste had been left at the site.
But today Wakefield Council confirmed the group of around 20 caravans had left.
Antony Sadler, Wakefield Council’s service director for communities, said: “We were made aware of an unauthorised encampment of travellers on Castlefield car park in Castleford. We visited the site and started legal action to try and move them on as quickly as possible. The travellers have since vacated the site overnight on Wednesday.”