Harrogate went ahead midway through the first half when Tom White fouled George Thomson in the penalty area and Dominic Knowles converted from the spot.
Gateshead drew level as Fraser Kerr rose highest at the back post to head home Greg Olley’s corner.
Steven Rigg then gave the hosts a 52nd-minute lead, heading in Scott Boden’s cross after a neat move.
But four minutes later Harrogate drew level as Jack Muldoon found space in the box and fired home a low effort.
Aaron Williams struck the winner five minutes from time after reacting quickest when Warren Burrell’s shot deflected into the box.
Elsewhere, FC Halifax Town are fourth after an excellent point in a hard-fought 0-0 draw at Wrexham.
Town survived a fourth-minute thunderbolt from Mike Fondop-Talom that crashed back off the bar to create the better early chances.
Nathan Clarke had a header from a corner ruled out for a foul, and Matty Kosylo, Clarke and Jonathan Edwards all fired off target.
Fondop-Talom was Wrexham’s biggest threat, shooting over and heading a free-kick towards the top left corner before Johnson reached high to save.
Luke Summerfield was forced to clear a Town header off the line shortly before the break.
By the hour mark, Fondop-Talom and Rekeil Pyke had both gone close with headers inside the box.
Halifax had lost some of their earlier composure, with Edwards’s shot at Rob Lainton from a tight angle against the run of play midway through the second-half. Dayle Southwell went close to winning it late on with an excellent run and low shot that was tipped behind.