Substitute Michael Smith headed home seven minutes into stoppage time to give the promotion-chasing Millers a deserved victory.
“I thought we were pretty good,” said Paul Warne. “I thought we created enough chances to win the game.”
Carlton Morris hit the post from a 10th minute corner as Rotherham enjoyed a good start.
Matt Crooks headed wide from Matthew Olosunde’s cross on 20 minutes, before Morris wasted two further headers.
Crooks was denied by a fine O’Leary save three minute after the restart as United kept up the pressure. And the visitors got their reward on the hour mark when Joe Mattock headed home at the far post after Ihiekwe headed Barlaser’s corner into his path.
But the lead didn’t last long with Aaron Pierre heading home from a well-worked free-kick six minutes later.
United received a massive let-off when Omar Beckles blazed over the bar from a corner with seven minutes to go.
And they made the hosts pay dearly when Smith won it after heading home seven minutes into stoppage time.
Shrewsbury Town: O’Leary, Pierre, Ebanks-Landell, Beckles, Love (Williams 86), Laurent, Walker, Golbourne, Edwards (Giles 79), Okenabirhie (Udoh 68), Whalley. Unused substitutes: Murphy, Fane, Goss, Cummings
Rotherham United: Iversen, Thompson, Ihiekwe, Robertson, Mattock (Wood 90 + 6), Olosunde, Barlaser, Crooks, Ogbene (Vassell 82), Morris (Smith 85), Ladapo. Unused substitutes: Price, Lindsay, Clarke, Hastie.
Referee: M Edwards (England).