After a goalless first half at the CNG Stadium, Halifax took the lead through Jordan Sinnott’s 69th minute strike.
Harrogate handed Bradford City loanee Reece Webb-Foster a debut in search of an equaliser but were left to fight with ten men as Simon Ainge collected a second yellow card.
And Halifax made them pay when Jake Hibbs finished off a counter attack in the final moments to send the West Yorkshire side through.
Elsewhere, National League pair York City and Guiseley both face midweek replays after draws in the fourth qualifying round.
Guiseley played out a bore draw at Lincoln City while managerless York let a lead slip to draw 1-1 against Curzon Ashton.
Jackie McNamara took caretaker control of York despite his sacking last week and his side took an early lead through Richard Brodie’s penalty.
However, Niall Cummins squeezed home an equaliser on 28 minutes and neither side could force a winner in the second half.
North Ferriby United exited the competition in a 4-1 loss at home to Macclesfield despite scoring the fastest goal of the fourth qualifying round.
Vinny Mukendi put Ferriby ahead inside a minute however their National League counterparts levelled on 43 minutes through Danny Whitaker.
Chris Holroyd gave the visitors the lead ten minutes after the restart before goals from Jack Mackreth and Whitaker sealed an away win.