Three-times Challenge Cup winners Featherstone Rovers booked their place in the fifth round with a seven-try win.
The Cumbrians were level at 12-12 late in the first half, but Rovers ran away with the game in the second period.
Prop Andy Bostock put Rovers ahead with a 10th-minute score.
Whitehaven replied with ex-Rovers player Jessie Joe Parker’s try but Featherstone regained the lead when Will Sharp crossed.
Dion Aiye went over and John Paul Brocklebank kicked his second goal to square matters again, but Rovers edged back in front as Sharp darted in for his second.
Featherstone added 20 unanswered second-half points with tries from Ben Blackmore, Will Milner, Matt James and Nathan Chappell.
Paul Sykes kicked three goals and Remy Marginet one.