The 64-year-old, who has gained a reputation for saving relegation-threatened clubs from the drop, believes the situation faced by the Terriers - eight points adrift of safety at the bottom of the Premier League - is too much for even him.
“They are rock-bottom of the Premier League and I don’t think I could change that situation, so for me it is not the right match. It is not the job for me,” said Allardyce, who was the bookmakers’ favourite.
“Huddersfield haven’t got enough goals to get them out of trouble. That has been the one failing.
“No club has got out of such trouble with so few points after this many games.”
Allardyce believes the club have ridden all their luck now and should be planning for a return to the SkyBet Championship.
“For me, and I know the fans won’t want to hear this, it’s got to be about planning for relegation and then planning to get back into the Premier League,” he added on TALKSPORT.
“Huddersfield’s trouble is not their performances, it’s a lack of goals and we all know what it means when you have a goalscorer.
“I’ve watched a lot of their games and they’ve always been very close, they’re never been thrashed really, but because of the lack of quality in front of goal they’ve ended up drawing instead of winning and losing instead of drawing.
“They’ve only scored 13 goals in 22 matches and, while I’ve managed to pull teams out of the bottom end, at Sunderland I had Jermain Defoe scoring 18 and at Crystal Palace I had Christian Benteke who scored 15 goals.”