Nicolas Pépé opened the scoring from the penalty spot before David McGoldrick levelled in the 87th minute.
However, Arsenal snatched the win and a place in the semi-finals in stoppage time through Dani Ceballos's strike. Here are the match ratings from Bramall Lane.
Dean Henderson, 7: Excellent save from Nicolas Pepe, who gave him no chance with the penalty. Could even have scored himself.
Chris Basham, 6: Put the Blades' best chance wide with a diving header.
John Egan, 7: An important block from Xhaka the highlight of some good defending.
Jack Robinson, 8: His defending has been better than his distribution since the restart but a cross for Chris Basham and a long throw for Oli McBurnie could have created goals, and one for David McGoldrick did.
George Baldock, 6: Usual hard-working performance by the wing-back.
John Fleck, 7: An excellent turn and pass to release McGoldrick in the first half.
Oliver Norwood, 6: Improved showing after his Old Trafford substitution.
John Lundstram, 6: When it's not your day... first a goal disallowed by the video assistant referee, then left with his arm in a sling.
Enda Stevens, 6: Thought he had snuffed out the danger late on, but not so.
David McGoldrick, 7: Goals in consecutive FA Cup games for a striker still to score in the Premier League, he caused Arsenal really problems down their right in the opening quarter.
Oliver McBurnie, 7: Decent performance from the target man.
Substitutes: Sander Berge (for Lundstram, 35), 5: Touches and final balls just did not come off.
K Freeman (for Baldock, 65), 6: Did his job as Baldock's replacement. Became a conventional right-back in the closing stages.
Sharp (for Basham, 76), 6: Forced Martinez into a good save with his legs .
Not used: L Freeman, Jagielka, Mousset, Osborn, Moore, Zivkovic.