They have won at Wembley on four of those occasions, the first coming in 1935 but the last being as far back as 1986.
Daryl Powell’s side will look to make that five when they venture there again on Saturday against St Helens.
He was in charge for one of the defeats – 2014 against his former club Leeds Rhinos – but the other loss was by a quality Castleford side in 1992.
Unfortunately, that day, Darryl van der Velde’s classy team, including brilliant Great Britain full-back Graham Steadman, Kiwi stars Tawera Nikau and Richie Blackmore plus inspirational captain Lee Crooks, came up against a mighty Wigan side in the middle of its record-breaking eight-year winning streak in the competition.
Lining up at stand-off for Castleford was Grant Anderson, one of their local players who hailed from the West Yorkshire mining town and who went on to score 82 tries in 234 games for the club after debuting in 1987.
Speaking to The Yorkshire Post, he recalled: “We had a real close-knit side. We had some good overseas players like T (Nikau), some great homegrown lads and a few people like Crooksy.
“There was (Australian second-row) Graeme Bradley as well who struggled early on and then started to pick his form up.
“We had a pretty good team and looking back we always fancied ourselves against Wigan.
“If you look at the results around that era we were probably the team that got closest to them than any.
“We went with confidence.”
However, Wigan took a fifth-minute lead when, in his first Challenge Cup final, prolific Lions winger Martin Offiah grubbered down his left wing and capitalised when – with a rare error – Steadman failed to successfully pick up the bobbling ball.
By the interval, they had built up a 19-0 lead courtesy of a try from Great Britain stand-off Shaun Edwards, another from Offiah, three conversions from Frano Botica and Joe Lydon drop-goal.
Castleford scored first in the second period via Blackmore’s try and their Lions prop Keith England would also cross but Wigan – whose entire 15-man squad were internationals – saw Steve Hampson add another try and the result was never in doubt as they finished 28-12 victors.
Anderson, 52, conceded: “Wigan just got the jump on us.
“Steady will look back and think maybe he should have put his foot through it when Martin goes through and scores early on.
“Within no time they were 19 points up. And Wigan’s experience of playing there put them in good stead. We started pushing passes. We got back to them in the second half a little but Wigan did do really well.”
Tragically, two of Castleford’s backline that day have sadly passed away.
David Nelson, the former Sheffield Eagles winger who played outside Blackmore at Wembley, was only 38 when he was shot and murdered at a pub in Seacroft, Leeds in 2001.
St John Ellis, the enigmatic winger who scored 97 tries for Castleford, died at the age of just 41, having collapsed following a training session with Doncaster, where he was coach, on New Year’s Eve in 2005.
“Singe played in the centre against Wigan, up against Gene Miles,” said Anderson, referring to Wigan’s famous Australia star.
“We had Jon Wray on the other wing.
“Singe was the first person who I ever saw react with the crowd. He used to score and run back and jump up on the fence, punching the air and hugging speccies!
“He was a totally different animal to what we were used to; he brought some razzmatazz to the team.
“Richie Blackmore was a world-class centre and it was quite a good backline.
“Up front, you knew what you’d get out of Beefy (England). He never took a backward step.
“We had Graham Southernwood at hooker who, really, should have been a world-beater. He was an absolute talent at 17. And that day he killed it.
“He can’t have been far off man of the match even though I know they don’t often give it to someone on the losing team.
“He had time on the ball, kicking game, great skills, toughness – especially in that era when there were elbows and not as many cameras about as now – and he could do anything.
“He probably didn’t use his brain as much as he should have done but he was a fabulous player.”
Southernwood was one of those, like Tony Smith, who was born and raised in the town.
However, of all the players who featured for Castleford in that era, the fearless Nikau, the world-class No13 with a famous mullet signed from Ryedale York in 1991, stands out for Anderson.
“I’ve been lucky enough to play with some great players,” he said.
“John Joyner is a big friend of mine - he’s coming to my house to watch the final on Saturday - and was a class player.
“But Tawera Nikau, from that time he played at Cas, he influenced every game he played in whether attack or defence.
“He improved the game, was a naturally class player with the ball in his hand, was great in defence and he good bash people with great technique.
“I can’t remember him ever not turning up. We all come off the field at times thinking it’s not happened for some reason.
“But I can’t ever remember T not turning up. What a class lad to have on your side.”
Although Castleford fell short against Wigan in 1992, two years later they did memorably defeat then in the Regal Trophy final at Headingley when Anderson – having switched to centre following the arrival of Kiwi No6 Tony Kemp – was among the try-scorers in a shock and emphatic 33-2 triumph.
But Wembley remained the highlight for the player, who spent all his career with his hometown club aside from a couple of seasons with Halifax.
Anderson explained: “When you look back as a local lad signing on at 17, playing for Redhill and Lock Lane, you don’t really visualise that.
“It’s your dream but sometimes a lot of dreams don’t come true.
“To actually do it and get there was unbelievable.
“If you are lucky enough to play at Wembley once, it’s great. You need to play twice, though, as all of a sudden it becomes a blur.
“Walking out in front of 77,000 people and that noise. The Regal Trophy victory was massive but Wembley is the pinnacle really, even if we didn’t get the result.
“It was a massive event for the town.”
Castleford’s other Challenge Cup wins came back to back in 1969 and 1970 but Anderson believes they can add another one at the weekend.
“Saints aren’t tearing roads up at the minute. Cas have enough quality to do it. I think they have a great shout.”