The Blades took the lead through the impressive Jose Baxter, but Rotherham staged a second-half revival to leave the visitors stunned.
The Millers equalised after the break after Kieran Agard turned to fire in a fierce shot before substitute Daniel Nardiello converted from the penalty spot after being fouled by Tony McMahon - who had earlier smashed a free kick against the crossbar.
Robert Milsom’s low shot secured a home victory and maintained Rotherham’s fine start to the campaign.
The Millers have won three and drawn three of their opening six league games.
”I apologised to the players for the first half for getting it wrong and the shape of the team,” said Millers manager Steve Evans.
“I said to the group don’t blame them (the players) blame me, I got it wrong and in the second half we saw the Rotherham United that’s there.
“We’ve played some super football (in the second half), we stepped up to the plate and the big players became big players.”