Freddie Ladapo celebrated his 27th birthday by scoring the winning goal for the Millers struck just moments after he had entered the fray from the substitutes’ bench.
His goal gave Rotherham a fifth successive home win and kept them top of the table but manager Paul Warne was far from happy.
The Millers chief said: “I picked the team and got it wrong and I’m disappointed in myself.
“The players are not dancing around in the dressing room. They’re pleased they’ve won but they know they have got away with one.
“I thought we created enough chances to win comfortably. I’m not best pleased. I thought we were poor. We will not keep winning if we keep up that performance level.
“I thought we looked lacklustre. It’s my fault. I think I should have made more changes from Tuesday night. I think it (the 1-0 win against Ipswich) emotionally drained a few of them.
“If you can learn from a victory then it’s a good thing and Freddie is the one who is smiling the most with a goal on his birthday.”
Both sides quickly squandered the lead in this entertaining encounter.
Jamie Murphy’s opener was swiftly cancelled out by Michael Smith in the first half.
Millers striker Smith’s header gave Rotherham a perfect start to the second before Oliver Sarkic silenced the home fans with a powerful finish moments later.
It was left to Ladapo to win it in the 71st minute.
Rotherham United: Iversen, Thompson, Ihiekwe, Wood, Mattock, Ogbene, Crooks, Barlaser (Olosunde 75), Adelakun (Wiles 61), Smith, Vassell (Ladapo 66). Unused substitutes: MacDonald, Lindsay, Clarke, Bilboe.
Burton Albion: O’Hara, Brayford, O’Toole, Buxton, Daniel (Hutchinson 88), Shaughnessy, Quinn, Sarkic (Fraser 77), Edwards (Powell 83), Murphy, Akins. Unused substitutes: Wallace, Sbarra, Nartey, Garratt.
Referee: W Finnie (England).