The game got off to a slow start as Millers midfielder Anthony Forde dragged a shot wide before United mustered a chance of their own with Kemar Roofe unable to work a header from a Jack Harrison cross on target.
Paul Warne side packed the middle of the pitch and pressed Leeds high as they attempted to force a mistake from the visitors who were enjoying the majority of the possession.
It was to be the Millers though who would bring the game to life just before the half hour mark as Semi Ajayi struck an effort from 30-yards out which flew into Casilla's top corner which wouldn't have looked out of place at the Bernabeu.
Bielsa's men managed to work themselves one chance before the break at Harrison cut in from the right and drilled a low shot on goal but the effort was straight at Marek Rodak in the Rotherham net.
Five minutes after the break and the Whites were level.
Warne's side again had started the better of the two sides but a mix-up at the back between defender Clark Robertson and goalkeeper Rodak allowed Mateusz Klich to prod home from close-range as the pair left the loose ball for one another inside the six-yard box.
Leeds were buoyed by the leveller and pushed on in search of the advantage.
The Millers almost regained an unlikely lead as Will Vaulks forced Casilla into a brilliant one-handed save with another strike that looked destined for the top corner.
With 20 minutes to go it was all United and Roofe really should've given the visitors the lead as Pablo Hernandez lifted a lovely ball over the top leaving the striker with the goal at his mercy but his header went wide of the mark.
Gjanni Alioski then himself fired past the post after being given the freedom of the Rotherham box as United started to produce attacks in waves.
Leeds pressed on but were limited to balls into the box from afar as the Millers dropped deep into their half as the clock ticked down.
United's pressure finally told five minutes from time as substitute Jerry Yates was caught in possession on the right and the ball worked its way to Klich who coolly slotted into the bottom left corner from inside the box to send the away fans behind the goal delirious.
Rotherham poured forward in the closing stages but failed to muster an attempt on goal in vein and Leeds claimed the spoils after a second half turnaround.
Leeds United XI: Casilla, Ayling, Phillips, Cooper, Alioski, Forshaw, Klich, Hernandez, Harrison, Clarke, Roofe. Subs: Peacock-Farrell, Davis, Shackleton, Stevens, Halme, Pearce, Roberts.
Rotherham United XI: Rodak, Mattock, Vaulks, Ajayi, Forde, Taylor, Towell, Robertson, Smith, Ragget, Jones. Subs: Price, Vyner, Wood, Palmer, Crooks, Vassell, Yates.
Attendance: 11,259 (2,317)