The building was badly damaged by the fire on Saturday, February 1, which burned for more than 24 hours.
Videos from the scene show clouds of heavy smoke descending across the city, as shops and homes were forced to evacuate.
But demolition is now underway at the site, with videos showing heavy machinery removing cladding from the building.
In a statement, West Yorkshire Fire and Rescue Service said: "Demolition of the site will begin today and continue over the coming days. You may notice an increase of smoke and dust in the area.
"Thornes Park Stadium will remain open as a reception centre for anyone affected by the Fire."
Earlier today, portable cabins were seen arriving on site.
It is understood that more than 100 workers were employed at the factory, which has stood on site for more than 40 years.
A spokesperson for Bakers, Food and Allied Workers Union said staff were continuing to be paid and would not be out of pocket.
Meanwhile, a number of local businesses, including the city's nine screen Cineworld cinema, have been forced to remain closed.