Tigers, one man down at the final hooter after referee Gareth Hewer also showed Grant Millington a yellow card, struggled to shake off their dogged visitors until 12 points in three minutes paved the way for a fourth straight win.
Greg Eden – with his second of the game – and a home debut effort from former Rovers player Chris Clarkson took the Tigers from 20-12 up to lead 32-12.
But with McGuire yellow-carded for dissent and James Greenwood and Weller Hauraki also in the sin-bin, the 10 men scored the game’s last try through Chris Atkin.
Rovers have won only two of their opening five fixtures but could have made a closer contest of it with Mose Masoe denied a hat-trick try with Rovers only trailing by eight.
Clarkson, released by Rovers at the end of last season, was making only his second appearance for his new club after recently signing terms until the end of the campaign.
Rovers lined up without ex-Wigan player Joel Tomkins, stood down ahead of a hearing to answer a charge of questioning an official’s integrity.
After a penalty littered, error strewn start, Rovers went in front courtesy of Masoe’s 13th minute barge over try - his first of 2019 - converted by Ryan Shaw.
By half-time, though, the Tigers were leading 14-6 following three tries – including two made on Humberside.
Ex-Hull FC player Jordan Rankin re-gathered his own kick after 15 minutes to touchdown.
Seven minutes later prop Liam Watts – who served both Hull clubs before arriving at the Jungle last season – smashed his way through from close range.
Paul McShane’s only goal of the opening period put Castleford 10-6 up before skipper Michael Shenton bounced the ball over the visitors’ try line.
Rovers lost substitute Ryan Lannon to a leg injury before the hosts strengthened their position with their third try of the night.
Shaw spilled possession, Hauraki talked his way into conceding a penalty and prolific winger Eden sped over in the corner for an unconverted try.
The Tigers advanced their lead to 20-6 with a 45th-minute walk-in try from James Clare and a superb touchline conversion by Rankin.
But McGuire set up man mountain Masoe for his second touchdown of the game and Shaw improved, leaving the Robins trailing just 20-12.
That though was as close as they got with Eden taking Jake Trueman’s kick at full speed to score in the corner.
Clarkson then took three defenders with him on his run to the line.
Rankin kicked both conversions to make the game safe especially after Hewer’s yellow-card frenzy in the final 10 minutes.
Castleford Tigers: Mata’utia, Clare, Minikin, Shenton, Eden, Trueman, Rankin, Watts, McShane, Millington, Sene-Lefao, Clarkson, Massey. Substitutes: Moors, Cook, Egodo, O’Neill.
Hull KR: Dagger, Shaw, J. Keinhorst, Hall, McGuire, Drinkwater, Jewitt, Lee, Masoe, Linnett, Addy, Hauraki, Mulhern. Substitutes: Greenwood, Atkin, Lannon.
Referee: Gareth Hewer (RFL).
IN the Championship, Batley Bulldogs suffered a heavy defeat at the hands of Sheffield Eagles, going down 44-16.