McPartlin, one half of duo Ant and Dec, previously announced he is stepping back from TV commitments and public appearances following a drink-driving conviction which saw him banned from the road for 20 months and fined Â£86,000.
This Morning host Holly Willoughby will join Declan Donnelly to present the show later this year in Australia.
Pattison, who won the jungle series in 2015 and struck up a friendship with the presenters, told The Sun: "Ant and I are mates, I absolutely love him. I think I echo a nation's sentiment when I say I'm devastated he won't be doing the jungle.
"It's a show that's so synonymous with him and Dec. He will be missed - there's no two ways about it."
Pattison, who rose to fame on reality series Geordie Shore, said she thinks he was "rushed into feeling ready for stuff last year" when he returned to screens after entering a rehabilitation centre due to living with depression, alcohol and substance abuse.
He revealed last year he had been addicted to painkillers since a failed knee operation in 2015.
McPartlin checked into a private rehab clinic again earlier this year after pleading guilty to drink-driving.
Last month he confirmed he had been sober for six months following his public battle with alcohol.
Pattison said: "I support him wholeheartedly. When the story broke I was texting him, telling him I was constantly there for him if he ever needed anything, that we all wanted him to just get better."
Ant and Dec have presented I'm A Celebrity together since it began in 2002.