The Castleford loose-forward played many sports as a youngster but called a halt to his budding football career after discovering he was a better rugby league player.
“My footballing days are long gone now,” he said. “I actually turned Leeds down! I went on a summer camp and they wanted me on trial but I said ‘no, I’m committed to my rugby’.
“I was better at rugby than I was at football so it wasn’t the hardest of decisions.
“My dad’s a big Leeds fan – we watched the Champions League games together – but he wasn’t that bothered.”
Milner has remained a big Leeds fan over the years and, alongside his England team-mate Tom Burgess, recently got the chance to take part in a penalty shoot-out at half-time during a Championship game against Ipswich.
“I actually thought I was just going up there to watch but I ended up going into the hospitality and taking penalties at half-time,” Milner said.
“It felt good. I’m a big Leeds fan so to be involved in their game was quite a special moment.
“That was the first game I’ve been to this year.
“I’m a good armchair supporter at the minute but last year I went to a few games when they were doing well at the start of the season. They’re doing well at the minute, so hopefully they can get to the glory land and get promoted.”
Milner has already experienced a victory at Elland Road this year, having been in the Castleford team that beat Leeds 25-24 there in March during redevelopment work at Headingley.
And he has the chance to experience more glory as England go in search of a Test series whitewash against New Zealand.
“I don’t know the last time that happened but that’s what we’re building towards and putting a marker down to the rest of the world that we’ve got something good in England with our rugby league team,” said Milner.
“It’s good signs for the national game.”
Milner made his debut against France in the opening international and was a substitute in last Sunday’s series-clinching 20-14 win over the Kiwis at Anfield.
“It was a special moment to get out there on Sunday for me and my family,” said Milner who played just 10 minutes as a replacement.
“I’m looking to build on that on Sunday and get more game time. I got a brief introduction to Test football and on my first contact I got bumped off and I came off with a splattered nose.
“That’s what Tests are about but I’ll be ready for it on Sunday, I know what to expect now.”